We have been getting OnDemand rolled out and find it to be a great software so far and are now starting to work on integrating Jupyter notebook via an interactive app but have some questions on the best way to implement this. Currently our users use Jupyter notebook by installing the package inside their own conda environments and launching from there.
However this does not seem to work too well with the way Jupyter notebook is launched via OnDemand since the launch command does not activate the conda environment where it is installed. This means the launch fails due to not finding the command.
My question is whether launching an interactive app from a conda environment is possible or if OnDemand expects an installed version loaded via module. If it is possible with conda environments, is there documentation that can be referenced to work this out?
Launching from a conda environment is definitely possible, just probably not advisable. OnDemand doensât expect anything. It comes down to the whatâs in the template/script.sh.erb. You can do all sorts of user environement setup in that shell script, because it is going to be running in your scheduler as that user so it has access that userâs home directory.
Have you seen the jupyter we run at OSC? So we load the module python/3.6-conda5.2 and that gives us /usr/local/python/3.6-conda5.2/bin/jupyter where someone has presumably pip installed it as root (Iâm not 100% sure how our module installations happen)?
Iâd suggest doing the same/similar. It gives the best âit just worksâ experience to your users. Of course you can always configure the script to boot $HOME/.local/bin/jupyter, but again, I think it âjust workingâ without users having to install things and do prep work is the best experience.
Now you may notice a lot of searching conda environments in how we load kernels, and I can (sorta) answer questions on all that - but essentially that provides functionality to load different python/IJUlia kernels if the user has installed them.
Hi Jeff, I took your advice and did a pip installation with one of our python modules.
This then allowed the app to start as expected and I was able to open up notebook. I will continue testing but wanted to report back and thank you for your help.
I have done a bunch of customization and javascript to load various anaconda installs (available as environment modules/lmod) then allows users to specify the path to a specific conda env if needed.
Its not the cleanest so let me know if you have questions on it!
We do Jupyter from Conda and it works great. We package the Python environment into an Lmod module, and stay away from Virtual Environments for several reasons listed below.
In case of OOD, weâd have to have 2 Lmod modules, one to load the base Anaconda and other to load the virtual environment. I consider that to be a hassle (the VE initialization does plug into the module syntax nicely), our users would need to get educated to write the VE module, or they would have to activate the VE in the OOD webform. Plus, I have seen problems with that before which details I donât remember, so, my recommendation is to have multiple Anaconda (miniconda) installations, one for each VE youâd want to run, each loadable by separate Lmod module, https://www.chpc.utah.edu/documentation/software/python-anaconda.php#mi. The miniconda install is simple as is a modification of its Lmod module.
Although it is not directly related to Jupiter notebook, may I know the way to launch Rstudio with conda environment? I will need to use specific R packages that were installed in the conda environment (thus have better version control). I am not sure how to install specific packages without conda. Thanks!
