No environment variables for jobs submitted through the job composer (myjobs)

Get Help
,
1

Hi Open Ondemand Discourse :slight_smile:
I have installed open ondemand 2.0.0 on ubuntu 20, and have it connected to a cluster running slurm 21.08.8-2.
While the job composer does work, and can submit jobs to the cluster, I am having some issues with the environment variables not being passed. I was hoping to have it setup, such that jobs launched with the job composer would know the PATH and USER of the job (similar to what it would be if a user logged in through ssh to the login node). This would also make it possible to issue commands like module load, which users usually use to access software. And also let people use srun and sbatch from within the main sbatch template.

However the environment variables for the job seems very limited and only seems to contain some slurm specific variables, and no $PATH at all.

I was able to get the interactive desktop to work by exporting the HOME variable, and setting it based on the $SLURM_JOB_USER variable. This was done in the batch connect vnc section of /etc/ood/config/clusters.d/ yml file for the cluster.

However none of the commands present under either the vnc or basic sections of the clusters.d yml file seems to be executed when submitting jobs through the job composer, so it doesn’t seem possible to build the environment variables from there.

Based of some posts in the discourse, i have tried to add
SLURM_EXPORT_ENV=“ALL”
to the file /etc/ood/config/app/myjobs/env
and i have also tried to
copy_environment: true
to the job section of the /etc/ood/config/clusters.d yml file.
and also to source /etc/profile
but none of these things seemed to help.

Does anyone know if it is possible to load environment variables for the job composer application?

2

Hi and welcome! Unfortunately I think you’ll need a patch for this issue.

It’s currently patched in the next version, but I’m going to backport it to 2.0 (2.0.29) shortly.

3

Hi jeff.orhstrom
Thanks a lot for responding. And so quickly! Tomorrow I will try to install the newer version from here:
https://apt.osc.edu/ondemand/latest/ondemand-release-web_2.1.0_all.deb

and let you know if it works out.

4

That is the nightly version which is a little unstable. I don’t believe there are known bugs with it, but again, it’s in active development.

That said - I hope to get 2.0.29 out by say the end of the month (and will submit the patch for your issue today).

5

Hi Again
Sorry for taking a while to follow up. I have been temporarily distracted by something else.
However, with the new release 2.0.29 this issue is completely resolved, and we can now pass environment variables through the job composer. Thanks a lot for your help.

Also, I have to say. Open Ondemand is a really cool platform. And you are a cool guy for making it.

1 Like
6

This topic was automatically closed 180 days after the last reply. New replies are no longer allowed.