dbarstis
April 2, 2024, 3:21pm
1
I’m trying to set up an interactive app in this case desktop with xfce. Everything appears to start up normally until it hits:
[websockify]: started successfully (proxying 53130 ==> localhost:5901)<' /opt/sge/crc/spool/d8civy030/job_scripts/110: line 249: done < <(tail -f --pid=${SCRIPT_PID} “vnc.log”) &’
I set the following in clusters.d yml file:
batch_connect:
basic:
script_wrapper: |
#!/bin/bash
set +o posix
. ~/.bashrc
module purge
%s
vnc:
script_wrapper: |
#!/bin/bash
set +o posix
. ~/.bashrc
module purge
export PATH="/opt/TurboVNC/bin:$PATH"
export WEBSOCKIFY_CMD="/usr/local/bin/websockify"
%s
script:
queue_name: "ondemand"
accounting_id: "crc"
job_name: "desktop_interactive"
native:
- "-pe"
- "sm"
- "4"
- "-S"
- "/bin/bash"
But job_script_options.json shows:
{
How can I get UGE to enforce the interpreter?
Looking at the docs - the set +o posix I believe is all you should need.
The script portion of what you’ve given though doesn’t do anything. The other portions will, but the script portion won’t be read/used. I think you’d need to duplicate that portion in all your apps’ submit.yml files.
I’d also ask for you to spot check to see if set +o posix and the others made it to the resulting file job_script_content.sh when you run the jobs.
Another thing you can check is your /var/log/ondemand-nginx/$USER/error.log for execve lines. These will show the actual qsub (? or similar I forget the UGE batch submit command off hand) command being issued.
dbarstis
April 3, 2024, 3:30pm
3
set +o posix does not appear in job_script_content.sh. I’ll see what I can find in error.log.
dbarstis
April 3, 2024, 5:53pm
4
Found the following execve lines in error.log
App 1396 output: [2024-04-02 11:02:26 -0400 ] INFO “execve = ["git", "describe", "–always", "–tags"]”
My point with the error logs is that if you wished to supply -S /bin/bash to the job, you’d see it here. I’ll try on my side to see what’s what with the job_script_content.sh.
I just confirmed that those settings you have in your cluster definition file should show up in job_script_content.sh. Would you mind sharing the entire file? Maybe there’s some issue there with YAML formatting that we’re overlooking.
dbarstis
April 3, 2024, 6:36pm
7
Here’s the original file:
v2:#conf: “/opt/sge/crc”
#bin_overrides:
#qsub: "/opt/sge/bin/lx-amd64/qsub_wrapper"
# qstat: ""
# qhold: ""
# qrls: ""
# qdel: ""
#
# submit_host: "ood-dev.crc.nd.edu"
I’ve started pulling out the script section and /bin/bash references to mirror the results from a post from Apr 2020 so it would look like:
v2:#conf: “/opt/sge/crc”
#bin_overrides:
#qsub: "/opt/sge/bin/lx-amd64/qsub_wrapper"
# qstat: ""
# qhold: ""
# qrls: ""
# qdel: ""
#
# submit_host: "ood-dev.crc.nd.edu"
Nothing has been implemented yet.
I think you need to reformat those because really I’m looking for format errors.
You can use ` characters like so:
```yaml
shows up as
my_content:
string: 'yaml conent'
with: |
more yaml content
dbarstis
April 3, 2024, 6:44pm
9
v2:
metadata:
title: "CRC Cluster"
login:
host: "ood-dev.crc.nd.edu"
job:
adapter: "sge"
cluster: "crc"
bin: "/opt/sge/bin/lx-amd64"
conf: "/opt/sge/util/install_modules/crc_configuration_node.conf"
#conf: "/opt/sge/crc"
sge_root: "/opt/sge"
libdrmaa_path: "/opt/sge/drmaa/lib/lx-amd64/libdrmaa.so"
batch_connect:
basic:
script_wrapper: |
#!/bin/bash
set +o posix
. ~/.bashrc
module purge
%s
vnc:
script_wrapper: |
#!/bin/bash
set +o posix
. ~/.bashrc
module purge
export PATH="/opt/TurboVNC/bin:$PATH"
export WEBSOCKIFY_CMD="/usr/local/bin/websockify"
%s
script:
queue_name: "ondemand"
accounting_id: "crc"
job_name: "desktop_interactive"
native:
- "-pe"
- "sm"
- "4"
- "-S"
- "/bin/bash"
#bin_overrides:
#qsub: "/opt/sge/bin/lx-amd64/qsub_wrapper"
# qstat: ""
# qhold: ""
# qrls: ""
# qdel: ""
#
# submit_host: "ood-dev.crc.nd.edu"
Welp there it is - batch_connect has the same indentation as job. So you need to shift that section left.
But again, the script section does nothing here.
dbarstis
April 3, 2024, 7:08pm
11
Yup. That was it. I looked at that for days and I never would have caught it.