# When checking the copy\_environment field of job composer module: command not found

**URL:** <https://discourse.openondemand.org/t/when-checking-the-copy-environment-field-of-job-composer-module-command-not-found/3522>\
**Category:** Get Help\
**Tags:** question\
**Created:** [May 28, 2024, 3:24pm UTC](https://discourse.openondemand.org/t/when-checking-the-copy-environment-field-of-job-composer-module-command-not-found/3522 "2024-05-28T15:24:36Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![asus-ocis](https://sea1.discourse-cdn.com/flex015/user_avatar/discourse.openondemand.org/asus-ocis/32/2275_2.png) [@asus-ocis](https://discourse.openondemand.org/u/asus-ocis)\
**Post date:** [May 28, 2024, 3:24pm UTC](https://discourse.openondemand.org/t/when-checking-the-copy-environment-field-of-job-composer-module-command-not-found/3522/1 "2024-05-28T15:24:36Z")

</div>

Hi

We install ood v3.1. In order to solve the following problems, check the copy\_environment field of the job option.

An ORTE daemon has unexpectedly failed after launch and before  
communicating back to mpirun. This could be caused by a number  
of factors, including an inability to create a connection back  
to mpirun due to a lack of common network interfaces and/or no  
route found between them. Please check network connectivity  
(including firewalls and network routing requirements).

When I submit the job, I get the following error. It seems that the environment variables have been cleared. How to solve this problem?

/var/spool/slurmd/job30416/slurm\_script: line 15: module: command not found  
/var/spool/slurmd/job30416/slurm\_script: line 16: module: command not found  
/var/spool/slurmd/job30416/slurm\_script: line 27: mpiexec.hydra: command not found

Thank you

---

<div class="post-metadata">

**Author:** ![travert](https://sea1.discourse-cdn.com/flex015/user_avatar/discourse.openondemand.org/travert/32/3074_2.png) [@travert](https://discourse.openondemand.org/u/travert)\
**Post date:** [May 28, 2024, 3:54pm UTC](https://discourse.openondemand.org/t/when-checking-the-copy-environment-field-of-job-composer-module-command-not-found/3522/2 "2024-05-28T15:54:51Z")

</div>

Hello and welcome!

Could you post the content of the `submit.yml` you are using. I’m assuming here you’ve set `copy_environment: true`? Is that correct?

---

<div class="post-metadata">

**Author:** ![asus-ocis](https://sea1.discourse-cdn.com/flex015/user_avatar/discourse.openondemand.org/asus-ocis/32/2275_2.png) [@asus-ocis](https://discourse.openondemand.org/u/asus-ocis)\
**Post date:** [May 28, 2024, 5:43pm UTC](https://discourse.openondemand.org/t/when-checking-the-copy-environment-field-of-job-composer-module-command-not-found/3522/3 "2024-05-28T17:43:02Z")

</div>

Hi travert

Sorry, where can I find myjob’s submit.yml? (/var/www/ood/apps/sys/myjobs/)

I created a job in job composer, and then checked Copy environment in the job option (as shown in the picture).  
 ![image](https://us1.discourse-cdn.com/flex015/uploads/osc/original/2X/0/0786e9fd464d1d26a271c0510aaa8b450961395b.png)

Next, I executed the submit job, an error occurred: command not found

 ![image](https://us1.discourse-cdn.com/flex015/uploads/osc/original/2X/4/4b54f4df8855201c4108521924d1f5c11d7640eb.jpeg)

I did not set copy\_environment: true in /etc/ood/config/clusters.d/xxx.yml, I don’t know where the setting is wrong.

Thank you!

---

<div class="post-metadata">

**Author:** ![mjbludwig](https://sea1.discourse-cdn.com/flex015/user_avatar/discourse.openondemand.org/mjbludwig/32/684_2.png) [@mjbludwig](https://discourse.openondemand.org/u/mjbludwig)\
**Post date:** [May 28, 2024, 5:51pm UTC](https://discourse.openondemand.org/t/when-checking-the-copy-environment-field-of-job-composer-module-command-not-found/3522/4 "2024-05-28T17:51:06Z")

</div>

That looks like an issue with either the compute node that the job was allocated or even your users .bashrc but not related to OOD.

The Job Composer is just a GUI interface for writing standard sbatch jobs.

---

<div class="post-metadata">

**Author:** ![asus-ocis](https://sea1.discourse-cdn.com/flex015/user_avatar/discourse.openondemand.org/asus-ocis/32/2275_2.png) [@asus-ocis](https://discourse.openondemand.org/u/asus-ocis)\
**Post date:** [May 29, 2024, 5:55am UTC](https://discourse.openondemand.org/t/when-checking-the-copy-environment-field-of-job-composer-module-command-not-found/3522/5 "2024-05-29T05:55:42Z")

</div>

Hi

In the SLURM adapter (`ood_core-0.25.0/lib/ood_core/job/adapters/slurm.rb`), there is a function `export_arg` that takes `env` and `copy_environment` as arguments. Here is the relevant code:

```ruby
          # we default to export NONE, but SLURM defaults to ALL.
          # we do this bc SLURM setups a new environment, loading /etc/profile
          # and all giving 'module' function (among other things shells give),
          # where the PUN did not.
          # --export=ALL export the PUN's environment.
          def export_arg(env, copy_environment)
            if !env.empty? && !copy_environment
              env.keys.join(",")
            elsif !env.empty? && copy_environment
              "ALL," + env.keys.join(",")
            elsif env.empty? && copy_environment
              # only this option changes behaivor dramatically
              "ALL"
            else
              "NONE"
            end
          end

```

The comments in the code mention that when `copy_environment` is set to `true`, the PUN’s (Portal User Environment) environment is exported. The comments also mention that the PUN does not run `/etc/profile`.

I have an `LMOD` variable set in `/etc/profile.d/`. Based on the comments and my understanding, if `copy_environment` is enabled, this `LMOD` variable will not be available in my SLURM job environment because the PUN environment does not run `/etc/profile`.

In my experiments, I found that by adding `source /etc/profile` to my script, the `LMOD` variable works as expected in the SLURM job environment. Here is what I added to my script:

```bash
#!/bin/bash
#SBATCH -J openmpi-hello-world # Job name
#SBATCH --partition development
#SBATCH --nodes=1
#SBATCH --ntasks-per-node=112
#SBATCH -o %j_openmpi-4.1.6-gcc-10.4.0.log # Path to the standard output and error files relative to the working directory
#SBATCH -e %j_openmpi-4.1.6-gcc-10.4.0.err # Path to the standard output and error files relative to the working directory

source /etc/profile
env
export UCX_TLS=ud,dc,rc,self
export OMPI_MCA_btl=tcp,self
export OMPI_MCA_pml=ucx
export UCX_NET_DEVICES=mlx5_0:1

GCC_VERSION="10.4.0"
OPENMPI_VERSION="4.1.6"

module purge
module load gcc/"$GCC_VERSION" openmpi/"$OPENMPI_VERSION"

mpirun hello-world_openmpi-"$OPENMPI_VERSION"-gcc-"$GCC_VERSION"

```

I would appreciate if someone could confirm my understanding or correct me if I’m wrong. Any additional insights or suggestions are also welcome.

Thank you in advance for your help.

Best regards,  
Frank

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/flex015/uploads/osc/original/2X/b/bae70bd0ed39a3ae769c2108155f4cb3e9da8385.png) [@system](https://discourse.openondemand.org/u/system)\
**Post date:** [November 25, 2024, 5:56am UTC](https://discourse.openondemand.org/t/when-checking-the-copy-environment-field-of-job-composer-module-command-not-found/3522/6 "2024-11-25T05:56:11Z")

</div>

This topic was automatically closed 180 days after the last reply. New replies are no longer allowed.