Bash interpreter help needed setting up interactive desktop/apps on ood 3.1.1/Univa Grid Engine

Get Help
1

I’m trying to set up an interactive app in this case desktop with xfce. Everything appears to start up normally until it hits:

[websockify]: started successfully (proxying 53130 ==> localhost:5901)
Scanning VNC log file for user authentications…
/opt/sge/crc/spool/d8civy030/job_scripts/110: line 249: syntax error near unexpected token <' /opt/sge/crc/spool/d8civy030/job_scripts/110: line 249: done < <(tail -f --pid=${SCRIPT_PID} “vnc.log”) &’

I set the following in clusters.d yml file:

batch_connect:
  basic:
    script_wrapper: |
      #!/bin/bash
      set +o posix
      . ~/.bashrc
      module purge
      %s
  vnc:
    script_wrapper: |
      #!/bin/bash
      set +o posix
      . ~/.bashrc
      module purge
      export PATH="/opt/TurboVNC/bin:$PATH"
      export WEBSOCKIFY_CMD="/usr/local/bin/websockify"
      %s
  script:
    queue_name: "ondemand"
    accounting_id: "crc"
    job_name: "desktop_interactive"
    native:
      - "-pe"
      - "sm"
      - "4"
      - "-S"
      - "/bin/bash"

But job_script_options.json shows:

{
“job_name”: “sys-dashboard-sys-bc_desktop-crc”,
“workdir”: “/project01/crcsupport/dbarstis/ondemand/data/sys/dashboard/batch_connect/sys/bc_desktop/crc/output/9fb3ae4b-920b-4f16-a7e9-9491fce4afe4”,
“output_path”: “/project01/crcsupport/dbarstis/ondemand/data/sys/dashboard/batch_connect/sys/bc_desktop/crc/output/9fb3ae4b-920b-4f16-a7e9-9491fce4afe4/output.log”,
“shell_path”: “/bin/bash”,
“wall_time”: 3600,
“queue_name”: “debug”,
“email_on_started”: false
}

How can I get UGE to enforce the interpreter?

2

Looking at the docs - the set +o posix I believe is all you should need.

The script portion of what you’ve given though doesn’t do anything. The other portions will, but the script portion won’t be read/used. I think you’d need to duplicate that portion in all your apps’ submit.yml files.

I’d also ask for you to spot check to see if set +o posix and the others made it to the resulting file job_script_content.sh when you run the jobs.

Another thing you can check is your /var/log/ondemand-nginx/$USER/error.log for execve lines. These will show the actual qsub (? or similar I forget the UGE batch submit command off hand) command being issued.

3

set +o posix does not appear in job_script_content.sh. I’ll see what I can find in error.log.

4

Found the following execve lines in error.log

App 1396 output: [2024-04-02 11:02:26 -0400 ] INFO “execve = ["git", "describe", "–always", "–tags"]”
App 1396 output: [2024-04-02 11:02:29 -0400 ] INFO “execve = [{}, "/opt/sge/bin/lx-amd64/qsub", "-wd", "/project01/crcsupport/dbarstis/ondemand/data/sys/dashboard/batch_connect/sys/bc_desktop/crc/output/9fb3ae4b-920b-4f16-a7e9-9491fce4afe4", "-N", "sys-dashboard-sys-bc_desktop-crc", "-o", "/project01/crcsupport/dbarstis/ondemand/data/sys/dashboard/batch_connect/sys/bc_desktop/crc/output/9fb3ae4b-920b-4f16-a7e9-9491fce4afe4/output.log", "-q", "debug", "-l", "h_rt=01:00:00"]”
App 1396 output: [2024-04-02 11:02:29 -0400 ] INFO “execve = [{}, "/opt/sge/bin/lx-amd64/qstat", "-r", "-xml", "-j", "110"]”
App 1396 output: [2024-04-02 11:02:39 -0400 ] INFO “execve = [{}, "/opt/sge/bin/lx-amd64/qstat", "-r", "-xml", "-j", "110"]”

5

My point with the error logs is that if you wished to supply -S /bin/bash to the job, you’d see it here. I’ll try on my side to see what’s what with the job_script_content.sh.

6

I just confirmed that those settings you have in your cluster definition file should show up in job_script_content.sh. Would you mind sharing the entire file? Maybe there’s some issue there with YAML formatting that we’re overlooking.

7

Here’s the original file:

v2:
metadata:
title: “CRC Cluster”
login:
host: “ood-dev.crc.nd.edu”
job:
adapter: “sge”
cluster: “crc”
bin: “/opt/sge/bin/lx-amd64”
conf: “/opt/sge/util/install_modules/crc_configuration_node.conf”
#conf: “/opt/sge/crc”
sge_root: “/opt/sge”
libdrmaa_path: “/opt/sge/drmaa/lib/lx-amd64/libdrmaa.so”
batch_connect:
basic:
script_wrapper: |
#!/bin/bash
set +o posix
. ~/.bashrc
module purge
%s
vnc:
script_wrapper: |
#!/bin/bash
set +o posix
. ~/.bashrc
module purge
export PATH=“/opt/TurboVNC/bin:$PATH”
export WEBSOCKIFY_CMD=“/usr/local/bin/websockify”
%s
script:
queue_name: “ondemand”
accounting_id: “crc”
job_name: “desktop_interactive”
native:
- “-pe”
- “sm”
- “4”
- “-S”
- “/bin/bash”

  #bin_overrides:
  #qsub: "/opt/sge/bin/lx-amd64/qsub_wrapper"
  # qstat: ""
  # qhold: ""
  # qrls: ""
  # qdel: ""
  #
# submit_host: "ood-dev.crc.nd.edu"

I’ve started pulling out the script section and /bin/bash references to mirror the results from a post from Apr 2020 so it would look like:

v2:
metadata:
title: “CRC Cluster”
login:
host: “ood-dev.crc.nd.edu”
job:
adapter: “sge”
cluster: “crc”
bin: “/opt/sge/bin/lx-amd64”
conf: “/opt/sge/util/install_modules/crc_configuration_node.conf”
#conf: “/opt/sge/crc”
sge_root: “/opt/sge”
libdrmaa_path: “/opt/sge/drmaa/lib/lx-amd64/libdrmaa.so”
batch_connect:
basic:
script_wrapper: |
set +o posix
. ~/.bashrc
module purge
%s
vnc:
script_wrapper: |
set +o posix
. ~/.bashrc
module purge
export PATH=“/opt/TurboVNC/bin:$PATH”
export WEBSOCKIFY_CMD=“/usr/local/bin/websockify”
%s

  #bin_overrides:
  #qsub: "/opt/sge/bin/lx-amd64/qsub_wrapper"
  # qstat: ""
  # qhold: ""
  # qrls: ""
  # qdel: ""
  #
# submit_host: "ood-dev.crc.nd.edu"

Nothing has been implemented yet.

8

I think you need to reformat those because really I’m looking for format errors.

You can use ` characters like so:

```yaml
my_content:
string: ‘yaml conent’
with: |
more yaml content
```

shows up as

my_content:
  string: 'yaml conent'
  with: |
    more yaml content
9 
v2:
  metadata:
    title: "CRC Cluster"
  login:
    host: "ood-dev.crc.nd.edu"
  job:
    adapter: "sge"
    cluster: "crc"
    bin: "/opt/sge/bin/lx-amd64"
    conf: "/opt/sge/util/install_modules/crc_configuration_node.conf"
      #conf: "/opt/sge/crc"
    sge_root: "/opt/sge"
    libdrmaa_path: "/opt/sge/drmaa/lib/lx-amd64/libdrmaa.so"
    batch_connect:
      basic:
        script_wrapper: |
          #!/bin/bash
          set +o posix
          . ~/.bashrc
          module purge
          %s
      vnc:
        script_wrapper: |
          #!/bin/bash
          set +o posix
          . ~/.bashrc
          module purge
          export PATH="/opt/TurboVNC/bin:$PATH"
          export WEBSOCKIFY_CMD="/usr/local/bin/websockify"
          %s
      script:
        queue_name: "ondemand"
        accounting_id: "crc"
        job_name: "desktop_interactive"
        native:
          - "-pe"
          - "sm"
          - "4"
          - "-S"
          - "/bin/bash"

      #bin_overrides:
      #qsub: "/opt/sge/bin/lx-amd64/qsub_wrapper"
      # qstat: ""
      # qhold: ""
      # qrls: ""
      # qdel: ""
      #
    # submit_host: "ood-dev.crc.nd.edu"
10

Welp there it is - batch_connect has the same indentation as job. So you need to shift that section left.

But again, the script section does nothing here.

11

Yup. That was it. I looked at that for days and I never would have caught it.
Thank you so much for your help! I pulled out the script section and the /bin/bash lines.

12

This topic was automatically closed 180 days after the last reply. New replies are no longer allowed.