Jupyter app tutorial: how to build in additional kernel support?

Get Help
1

Hi all,

I stood up our OOD-Jupyter app a while back using the interactive apps tutorial, with minimal modifications, and am quite happy with the results. We’re now ramping up to replace our jupyterhub with OOD, but it looks like I need some guidance getting Jupyter to “see” the/any kernels available to it.

If I’m not mistaken, lines 231-271 of script.sh.erb from the bc_osc_jupyter repo demonstrate what’s necessary to include additional kernels (e.g. bash, matlab, etc) into Jupyter, but I think I may be missing something as I struggle to interpret the erb/ruby.

Could I beg of someone to outline the bare necessities of prepping/using additional kernels, that one might need to built into the tutorial version of the Jupyter app, please and thank you? (…very, very, very much!)

If it helps, kernels used by our jupyterhub sit in our nfs /packages Environment Modules library, and we’re at OnDemand version: v1.6.20 | Dashboard version: v1.35.3.

(For completeness, this issue looks relevant but also looks like it’s specific to the aforementioned OSC-specific erb file in use at Owens.)

2

Jason,

we have a handful of other kernels (R, Matlab), and all that’s needed is to

  • install the kernel into the Jupyter - each has its own way of doing that it usually boils down to installing a Python module and modifying a Jupyter config file
  • load the needed environment (lmod) modules (e.g. R, Matlab) in the script.erb file

I don’t think we needed to do anything more than that.

As for Virtual Environments, I just replied on that the other day, I am not a big fan of those - prefer independent miniconda installations loadable with separate environment modules. I describe our Jupyter setup in that reply.

HTH,
MC

1 Like
3

Thanks for your response, @mcuma. With your verification of how things were supposed to work, I went looking more carefully at my setup and found the problem :slight_smile:

…which I’ll share here in case anyone else runs into the issue, in the future:

  # /var/www/ood/apps/sys/jupyter/form.yml
  
  # Set the corresponding modules that need to be loaded for Jupyter to run
  #
  # @note It is called within the batch job as `module load <modules>`
  #   if defined
  #
  # JB note: qiime's own python will take precendence
  #          and ignore the explicit anaconda3 kernels
  #          unless you load qiime before anaconda3

  modules: "qiime2/2019.10 anaconda3"

We wanted our jupyter app to have qiime available by default, so I had listed qiime2 after anaconda on the modules: line… but that ordering made $PATH find qiime’s python before anaconda’s… and so the kernels within anaconda’s python weren’t found.

Thanks again!