Greetings @ShardulC,
Generally it is intended and recommended to run all application code in a container. The base TorizonCore OS has no package manager to install additional packages you may need. The only alternative would be to use the Yocto build system to build your own TorizonCore OS with Python and whatever packages you require.
With containers it’s a fairly simple matter to use a Debian based container, then utilize the apt package manager in Debian to install whatever you need. For further guidance you can consult our documentation on this topic here: Python development on TorizonCore | Toradex Developer Center
Best Regards,
Jeremias