Hi,
I’m pretty sure that once again the problem is quite stupid but i’m completly lost in the multiple config and options and i can’t seems to make sense of any of it so i’m resolving to post here
I have a working cluster in a kind of working openondemand instance on a debian 12 machine. I’m trying to start a notebook on a remote ubuntu 22.04LTS via the Linux Host adapter and by following the jupyter interactive app doc.
Actually the jupyter interactive app immediatly finish in Completed without any logs in the session folder
On the PUN logs i can see a successfull connection followed by what i interpret as a tentative to attach to a tmux session ? But it seems to miss some parameter ?
App 431257 output: [2026-02-09 10:06:52 +0000 ] INFO "execve = [{}, \"ssh\", \"-t\", \"-p\", \"22\", \"-o\", \"BatchMode=yes\", \"-o\", \"UserKnownHostsFile=/dev/null\", \"-o\", \"StrictHostKeyChecking=no\", \"soullard@kafka-stage\", \"tmux\", \"list-panes\", \"-aF\", \"\\\\#\\\\{session_name\\\\},\\\\#\\\\{session_created\\\\},\\\\#\\\\{pane_pid\\\\}\"]"
App 431257 output: [2026-02-09 10:06:52 +0000 ] INFO "execve = [{}, \"ssh\", \"-t\", \"-p\", \"22\", \"-o\", \"BatchMode=yes\", \"-o\", \"UserKnownHostsFile=/dev/null\", \"-o\", \"StrictHostKeyChecking=no\", \"soullard@<server ip>\", \"tmux\", \"list-panes\", \"-aF\", \"\\\\#\\\\{session_name\\\\},\\\\#\\\\{session_created\\\\},\\\\#\\\\{pane_pid\\\\}\"]"
App 431257 output: [2026-02-09 10:06:52 +0000 ] INFO "method=GET path=/pun/sys/dashboard/batch_connect/sessions format=html controller=BatchConnect::SessionsController action=index status=200 allocations=18438 duration=522.52 view=10.93"
App 431257 output: [2026-02-09 10:06:52 +0000 ] WARN "Announcement file not found: /etc/ood/config/announcement.md"
App 431257 output: [2026-02-09 10:06:52 +0000 ] WARN "Announcement file not found: /etc/ood/config/announcement.yml"
App 431257 output: [2026-02-09 10:06:52 +0000 ] INFO "method=GET path=/pun/sys/dashboard/batch_connect/sessions format=turbo_stream controller=BatchConnect::SessionsController action=index status=200 allocations=6677 duration=5.94 view=2.96"
On the remote server, all i can see is a user slice briefly starting before dying almost immediatly
For my configuration i used the following files
cluster.yml
v2:
metadata:
title: "Ubuntu Remote"
url: "this-is-a-remote-test"
hidden: false
login:
host: "<remote ip>"
job:
adapter: "linux_host"
submit_host: "<remote ip>"
ssh_hosts:
- "kafka-stage"
- "<remote ip>"
site-timeout: 7200
debug: true
singularity_bin: /usr/bin/singularity
singularity_image: /opt/ood/linuxhost_adaptater/ubuntu.sif
strict_host_checking: false
tmux_bin: /usr/bin/tmux
batch_connect:
set_host: "host= $(echo 'kafka-stage')" # this is ignored by linux-host_adapter anyway
conn_file: "connection.yml"
singularity_bindpath: /fs,/home
singularity_container: /usr/opt/ood/linuxhost_adapter/ubuntu.sif
Even though i added singularity options as specified in the doc (and singularity is installed on the remote host) i never found any tentative to run the singularity executable on the remote host. I also precise that the cluster shell access is working fine with this config. Removing the singularity_ options from the batch_connect does not help either.
For the jupyter app files I add the following values
diff --git a/form.yml b/form.yml
index 3aaf67b..2f53b4c 100644
--- a/form.yml
+++ b/form.yml
@@ -8,7 +8,7 @@
# under /etc/ood/config/clusters.d/*.yml
# @example Use the Owens cluster at Ohio Supercomputer Center
# cluster: "owens"
-cluster: ""
+cluster: "ubuntu_test"
# Define attribute values that aren't meant to be modified by the user within
# the Dashboard form
@@ -25,11 +25,11 @@ attributes:
# modules: "python/3.5"
# @example Using combination of modules
# modules: "python/3.5 cuda/8.0.44"
- modules: "python"
+ modules: "" #since module is not installed on the host at least hoping for a clear crash once module will be called
# Any extra command line arguments to feed to the `jupyter notebook ...`
# command that launches the Jupyter notebook within the batch job
- extra_jupyter_args: ""
+ extra_jupyter_args: "--ip 157.136.254.73 --port 8080 --no-browser" #because all other ports are closed
# All of the attributes that make up the Dashboard form (in respective order),
# and made available to the submit configuration file and the template ERB
diff --git a/template/before.sh.erb b/template/before.sh.erb
index 6b89042..f4caf02 100755
--- a/template/before.sh.erb
+++ b/template/before.sh.erb
@@ -27,10 +27,14 @@
# The plain text password used to authenticate to the web server with
# Export the module function if it exists
+#
+
+touch /tmp/imhere
+
[[ $(type -t module) == "function" ]] && export -f module
# Find available port to run server on
-port=$(find_port)
+port=8080 #$(find_port)
# Generate SHA1 encrypted password (requires OpenSSL installed)
SALT="$(create_passwd 16)"
@@ -46,7 +50,7 @@ export CONFIG_FILE="${PWD}/config.py"
umask 077
cat > "${CONFIG_FILE}" << EOL
c.NotebookApp.ip = '*'
-c.NotebookApp.port = ${port}
+c.NotebookApp.port = 8080 #${port}
c.NotebookApp.port_retries = 0
c.NotebookApp.password = u'sha1:${SALT}:${PASSWORD_SHA1}'
c.NotebookApp.base_url = '/node/${host}/${port}/'
diff --git a/template/script.sh.erb b/template/script.sh.erb
index 718951c..a926db3 100755
--- a/template/script.sh.erb
+++ b/template/script.sh.erb
@@ -6,6 +6,8 @@ echo "TIMING - Starting main script at: $(date)"
# Set working directory to home directory
cd "${HOME}"
+touch imhere #this has never been executed in any of my tests
+
#
# Start Jupyter Notebook Server
#
Any help or hint would be appreciated. Pretty sure it is a simple configuration i missed or not set correctly but I can’t seems to find it.
Sorry for this wall of text.Thanks a lot in advance. Do not hesitate if i need to put anymore details.
PS : Just to precise an other problem which may be unrelated but it seems that the batch job will not start without rclone installed on the ondemand machine but it never seems to be called anywhere and the ondemand home directory never seems to be forwarded to the remote host. Is that a normal behavior ? Or did I (once again) missed an option ?