No environment variables for jobs submitted through job composer

Get Help
1

Hi all,

OnDemand 3.03.

Failed to run MPI jobs, turns out SLURM_EXPORT_ENV is set to “none” by OOD by default.

Tried many ways as below, did not work:

Added a new file /etc/ood/config/app/myjobs/env, add line of:
SLURM_EXPORT_ENV=“ALL”

Also added:
copy_environment: true
to /etc/ood/config/clusters.d.yml

No success.

Right now I have added SLURM_EXPORT_ENV=“ALL” in the default job script template, so users can run MPI jobs without add it manually.

Is there any way to fix this inside OOD?

Thanks,

Feng

2

Slurm as job scheduler.

3

Yes, we provide this config for our production desktop applications for the same reason.

Additionally you can set this configuration at the cluster level if you like, so it’ll affect all VNC applications - not just your desktops.

https://osc.github.io/ood-documentation/latest/reference/files/submit-yml-erb.html#setting-batch-connect-options-globally

4

Thanks, Jeff!

Is there’s any way to set this only for the Job Composer(myjob) itself( like a config file in /etc/ood/config/app/myjobs/)? Not from the Job script template? Since it sounds better to hide this from regular users.

The documentation of OOD says you can change OOD-MAX-SCRIPT-SIZE_KB in /etc/ood/config/app/myjobs/env, but in my installation, there is no such file. I added this file there, and added the SLURM_EXPORT_ENV, but it seems the OOD at least ignores the SLURM_EXPORT_ENV.

Add SLURM_EXPORT_ENV to Job script template does the work, but it seems not a perfect solution?

Best
Feng

5

In 3.0.3 there should be a checkbox to Copy Environment in the Job Composer. Check that box and we’ll use ALL instead of NONE.

6

Thanks, Jeff.

Yes I see it is in “Job Options” windows after click the “Job Options” button.

Once I click to select “Copy Environemnt”. The job failed with:

job169935/slurm_script: line 23: module: command not found

The "module " is the linux env module command to set runtime env.

Looks like this option clears all the env? The PATH is set to a simple one as:

PATH=/opt/ood/ondemand/root/usr/share/gems/3.0/bin:/opt/ood/ondemand/root/usr/bin:/opt/ood/ondemand/root/usr/sbin:/sbin:/bin:/usr/sbin:/usr/bin

7

Hnnn, I checked the myjobs/cinfig/configuration_singleton.rb, it seems it reads the “.env” file setting, not “env”. Made the change and it seems it works.

8

This is why we use NONE. It doesn’t clear the env, we’re submitting the job from an environment that doesn’t have the module system, so you can see why there’s a weird PATH.

9

Thanks, Jeff!

I added a “.env” file to myjobs folder with the SLURM_EXPORT_ENV=ALL, and it seems it works… Need to do some more tests to see if it really fix it and won’t cause any issue.

Best,

Feng