Timed out waiting for RStudio Server to open port with Rocker RStudio

Hello

I’m trying to install RStudio app into OOD. I clone the OSC RStudio Server repo, GitHub - OSC/bc_osc_rstudio_server: Batch Connect - OSC RStudio Server and modified it for our cluster.

I pull singularity image from rocker singularity.

singularity pull docker://rocker/rstudio:4.4.2

This is my template/script.sh.erb

#!/usr/bin/env bash

<%-
  session_dir = session.staged_root
  tutorial_dir = session.staged_root.join('ParallelR')

  match = /^app_rstudio_server\/(.+)$|.+ R\/(.+)/.match(context.version)
  r_version = match[1] unless match[1].nil?
  r_version = match[2] unless match[2].nil?

  all_modules = if r_version >= '4.2' && ['kubernetes', 'kubernetes-test', 'kubernetes-dev'].include?(context.cluster)
                  "gnu/11.2.0 mkl/2021.3.0 #{context.version}"
                else
                  context.version
                end
%>

# Load the required environment
setup_env () {
  module purge
  module load singularity
  module use "/iss/home/s.park/ondemand/dev/RStudio/lmodfiles"
  module load rocker_rstudio/4.4
}

setup_env

export WORKING_DIR="<%= session_dir %>"

#
# Start RStudio Server
#
# PAM auth helper used by RStudio
export RSTUDIO_AUTH="$WORKING_DIR/bin/auth"

# Generate an `rsession` wrapper script
export RSESSION_WRAPPER_FILE="$WORKING_DIR/rsession.sh"
(
umask 077
sed 's/^ \{2\}//' > "$WORKING_DIR/rsession.sh" << EOL
  #!/usr/bin/env bash

  # Log all output from this script
  export RSESSION_LOG_FILE="$WORKING_DIR/rsession.log"

  exec &>>"\${RSESSION_LOG_FILE}"
  set -x

  # rsession.sh doesn't share the same env as the outside script, so these
  # need to be set explicitly
  export R_LIBS_SITE="${R_LIBS_SITE}"
  export TZ="Europe/Paris"
  export HOME="$HOME"
  export MODULEPATH_ROOT="$MODULEPATH_ROOT"
  export MODULEPATH="$MODULEPATH"
  export LMOD_PKG="$LMOD_PKG"

  env

  # Launch the original command
  echo "Launching rsession..."
  exec rsession --r-libs-user "${R_LIBS_USER}" "\${@}"
EOL
)
chmod 700 "$WORKING_DIR/rsession.sh"

# Set working directory to home directory
cd "${HOME}"

# Output debug info
module list
hostname

# server log directory in this job's working directory
mkdir -p "$WORKING_DIR/logs"

# use the system installed rserver
export PATH="$PATH:/usr/lib/rstudio-server/bin"

set -x
# Launch the RStudio Server
echo "Starting up rserver... at $(date)"

#bwrap --dev-bind / / --tmpfs /tmp \
#       --bind $WORKING_DIR/etc /etc/rstudio \
#       --setenv LD_LIBRARY_PATH "$LD_LIBRARY_PATH" \
#       rserver \
#       --www-port "${port}" \
#       --auth-none 0 \
#       --auth-pam-helper-path "${RSTUDIO_AUTH}" \
#       --auth-encrypt-password 0 \
#       --rsession-path "${RSESSION_WRAPPER_FILE}" \
#       <%- if r_version > '4.1' -%>
#       --server-data-dir='/tmp/run' \
#       --server-user=$(whoami)
#       <%- end -%>

singularity exec --contain \
        -B $WORKING_DIR/etc:/etc/rstudio \
        "$RSTUDIO_SERVER_IMAGE" \
        rserver \
        --www-port "${port}" \
        --auth-none 0 \
        --auth-pam-helper-path "${RSTUDIO_AUTH}" \
        --auth-encrypt-password 0 \
        --rsession-path "${RSESSION_WRAPPER_FILE}" \
        --server-data-dir='/tmp/run' \
        --server-user=$(whoami)

and my modulefile is here, lmodfiles/rocker_rstudio/4.4.lua.

--
help([[ rstudio - loads rstudio with singularity environment for ondemand apps ]])
whatis([[loads rstudio with singularity environment for ondemand]])
setenv("RSTUDIO_SERVER_IMAGE","/iss/home/s.park/ondemand/dev/RStudio/image/rocker-rstudio.simg")
setenv("SINGULARITY_BIND","/etc,/dev,/media,/mnt,/opt,/srv,/var")
prepend_path("PATH", "/usr/lib/rstudio-server/bin",":")

I failed to open RStudio on the OOD interface with this output.log.

Script starting...
Waiting for RStudio Server to open port 12805...

Currently Loaded Modules:
  1) proxy/1.0.0                17) zlib/1.3.1-aqk3zlk
  2) singularity/4.2.1          18) icu4c/74.2-rbeu5gf
  3) ncurses/6.5-7sn3yj4        19) libtiff/4.7.0-higewyn
  4) readline/8.2-aypt7cv       20) libiconv/1.18-svwat2m
  5) glibc/2.28-7rs64fv         21) libxml2/2.10.3-4p4hiah
  6) gcc-runtime/8.5.0-4ihak4k  22) pigz/2.8-uobpsza
  7) bzip2/1.0.8-ian4lcl        23) zstd/1.5.7-w5ywklb
  8) xz/5.8.0-6l3tplv           24) tar/1.34-67uuyj2
  9) pcre2/10.45-2hgtsjn        25) gettext/0.25-ifh7uj4
 10) nghttp2/1.65.0-6bei7wb     26) berkeley-db/18.1.40-pmlmbu5
 11) zlib-ng/2.2.4-z3von4t      27) gdbm/1.25-cpxuto2
 12) openssl/3.5.0-gkftdzj      28) perl/5.40.2-jlj34nt
 13) curl/8.12.1-j7as5hi        29) texinfo/7.2-hbutatj
 14) openjdk/17.0.11_9-tha6p3c  30) R/4.4.2
 15) libjpeg/9f-aahyifh         31) rocker_rstudio/4.4
 16) libpng/1.6.39-wzsztji


s-cpu05
++ date
+ echo 'Starting up rserver... at Tue Aug 19 16:20:39 CEST 2025'
Starting up rserver... at Tue Aug 19 16:20:39 CEST 2025
++ whoami
+ singularity exec --contain -B /iss/home/s.park/ondemand/data/sys/dashboard/batch_connect/dev/RStudio/output/c87c60f7-c470-4d76-9d4c-da5dd61d4e7a/etc:/etc/rstudio /iss/home/s.park/ondemand/dev/RStudio/image/rocker-rstudio.simg rserver --www-port 12805 --auth-none 0 --auth-pam-helper-path /iss/home/s.park/ondemand/data/sys/dashboard/batch_connect/dev/RStudio/output/c87c60f7-c470-4d76-9d4c-da5dd61d4e7a/bin/auth --auth-encrypt-password 0 --rsession-path /iss/home/s.park/ondemand/data/sys/dashboard/batch_connect/dev/RStudio/output/c87c60f7-c470-4d76-9d4c-da5dd61d4e7a/rsession.sh --server-data-dir=/tmp/run --server-user=s.park
Timed out waiting for RStudio Server to open port 12805! on Tue Aug 19 16:22:41 CEST 2025
Cleaning up...

There are simillar discussion but not exactly. I tried to find many discussion but I couldn’t.

Does anyone have this issue?

PARK

1 Like

Looking at the output log, are you able to log what port is being set to? I’m wondering if the port is somehow in use. We provide a built-in function called find_portthat takes some parameters and returns a free port for you:

Thank you for your answer.

There is a file template/before.sh.reb and that file includes sourcing find_host_port but I couldn’t find where it is..

template/before.sh.erbis as below.

# rstudio 1.4+ needs a csrf token
csrf_token=<%= SecureRandom.uuid %>

# Export the module function if it exists
<% if context.cluster =~ /kubernetes/ -%>
exec &> >(tee -a "pod.log")

source /etc/profile.d/lmod.sh

source /bin/find_host_port
source /bin/save_passwd_as_secret
password="$PASSWORD"
host="$HOST_CFG"
port="$PORT_CFG"
export SINGULARITY_HOME="$HOME"

/bin/bash /opt/open_ondemand/helpers/k8_helper csrf_token "$csrf_token"

<% else -%>
[[ $(type -t module) == "function" ]] && export -f module

# Find available port to run server on
port=$(find_port ${host})

# Define a password and export it for RStudio authentication
password="$(create_passwd 16)"

<% end -%>

export RSTUDIO_PASSWORD="${password}"

I don’t see either of the port variables being exported though, so the application likely doesn’t know which port was opened. You’ll need to at least export those port entries and see where things go from there.

Hello,

I think it’s a little bit strange. When I submit a job, before.sh.erb is run and get an available port that is used in script.sh.erb script.

As above output.log file, the second line says RStudio Server to open port 12805 that is set in before.sh.erb script. I checked the port variable is exported, then it worked well. Also, the bottom of the output.log, on the singularity executed image and rserver executed with proper port –-www-port 12805.

I couldn’t understand you can’t see port variable. Is there any other port variable for ood and RStudio?

Sorry about the earlier confusion. I was mistaken in thinking the issue was with the ${port} variable not being exported. OOD is passing that in correctly as you said.

Looking at this again from the start, it doesn’t look like any of the rsession.sh script ever gets mounted into the container for you at launch. I’d bet that because of that, the rstudio application fails within the container, but OOD just waits and never sees anything and it times out.

I think you’ll want to make sure to include a -B $WORKDING_DIR:$WORKING_DIR argument along with what you have when you call singularity in the script.sh you have to start.

Oh, no. You don’t say sorry. I’m very thankful to you and community.

Unfortunately, I tried to cal singularity in the script.sh with -B $WORKING_DIR:$WORKING_DIR but it failed as same error. Timed out …

As you said before, rsession.sh seems not to be executed properly.

Here before.sh is as follows.

# Export the moduddle function if it exists
[[ $(type -t module) == "function" ]] && export -f module

# Find available port to run server on
port=$(find_port ${host})

# Define a password and export it for RStudio authentication
password="$(create_passwd 16)"
export RSTUDIO_PASSWORD="${password}"

And here is my script.sh

#!/usr/bin/env bash

<%-
  session_dir = session.staged_root
%>

# Load the required environment
setup_env () {
  module purge
  module load singularity
  module use /iss/home/s.park/ondemand/dev/RStudio/lmodfiles
  module load rocker_rstudio/4.4
}
setup_env

export WORKING_DIR="<%= session_dir %>"

#
# Start RStudio Server
#
# PAM auth helper used by RStudio
export RSTUDIO_AUTH="$WORKING_DIR/bin/auth"

# Generate an `rsession` wrapper script
export RSESSION_WRAPPER_FILE="$WORKING_DIR/rsession.sh"
(
umask 077
sed 's/^ \{2\}//' > "$WORKING_DIR/rsession.sh" << EOL
  #!/usr/bin/env bash

  # Log all output from this script
  export RSESSION_LOG_FILE="$WORKING_DIR/rsession.log"

  exec &>>"\${RSESSION_LOG_FILE}"
  set -x

  # Launch the original command
  echo "Launching rsession..."
  exec rsession --r-libs-user "${R_LIBS_USER}" "\${@}"
EOL
)
chmod 700 "$WORKING_DIR/rsession.sh"

# Set working directory to home directory
cd "${HOME}"

# Output debug info
module list

# server log directory in this job's working directory
mkdir -p "$WORKING_DIR/logs"

# use the system installed rserver
export PATH="$PATH:/usr/lib/rstudio-server/bin"

set -x
# Launch the RStudio Server
echo "Starting up rserver... at $(date)"

singularity exec -B "$WORKING_DIR:$WORKING_DIR" "$RSTUDIO_SERVER_IMAGE" rserver \
	--www-port="${port}" \
	--auth-none=0 \
	--auth-pam-helper-path="${RSTUDIO_AUTH}" \
	--auth-encrypt-password=0 \
	--rsession-path="${RSESSION_WRAPPER_FILE}" \
	--server-user=$(whoami)

Due to the script.sh, rsession.sh is generated in the $WORKING_DIR.

#!/usr/bin/env bash

# Log all output from this script
export RSESSION_LOG_FILE="/iss/home/s.park/ondemand/data/sys/dashboard/batch_connect/dev/RStudio/output/2828bf43-3b3e-4185-bd3b-21816b948f36/rsession.log"

exec &>>"${RSESSION_LOG_FILE}"
set -x

# Launch the original command
echo "Launching rsession..."
exec rsession --r-libs-user "" "${@}"

This is output.log.

before.sh starting
Script starting...
Waiting for RStudio Server to open port 61395...

Currently Loaded Modules:
1) proxy/1.0.0   2) singularity/4.2.1   3) rocker_rstudio/4.4 

++ date
+ echo 'Starting up rserver... at Fri Aug 22 13:08:25 CEST 2025'
Starting up rserver... at Fri Aug 22 13:08:25 CEST 2025
++ whoami
+ singularity exec -B /iss/home/s.park/ondemand/data/sys/dashboard/batch_connect/dev/RStudio/output/2828bf43-3b3e-4185-bd3b-21816b948f36:/iss/home/s.park/ondemand/data/sys/dashboard/batch_connect/dev/RStudio/output/2828bf43-3b3e-4185-bd3b-21816b948f36 /iss/home/s.park/ondemand/dev/RStudio/image/rocker-rstudio.simg rserver --www-port=61395 --auth-none=0 --auth-pam-helper-path=/iss/home/s.park/ondemand/data/sys/dashboard/batch_connect/dev/RStudio/output/2828bf43-3b3e-4185-bd3b-21816b948f36/bin/auth --auth-encrypt-password=0 --rsession-path=/iss/home/s.park/ondemand/data/sys/dashboard/batch_connect/dev/RStudio/output/2828bf43-3b3e-4185-bd3b-21816b948f36/rsession.sh --server-user=s.park
Timed out waiting for RStudio Server to open port 61395! on Fri Aug 22 13:10:30 CEST 2025
Cleaning up...

I wonder if at this point it is a matter of running the rsession.sh script now that it is mounted.

Can you look at the error logs from the container at all? I’m wondering if the scirpt is mounted in, but hitting some kind of permission issue in the container when it goes to execute it after the mount. I know you are setting it to 077 outside, but I’m just not sure when it is bind mounted if that remains true.

Actually, aren’t you then chmod 700 the file after you write it? Yeah, I’d bet it’s something with permissions at this point and doing some logging in the container and seeing what errors happen will be the best guide.

Finally, I opened RStudio UI with rocker rstudio image.

This is the last part of my script.sh

# Launch the RStudio Server
echo "Starting up rserver... at $(date)"

singularity exec \
    --bind $WORKING_DIR/etc:/etc/rstudio \
    "$RSTUDIO_SERVER_IMAGE" \
    rserver --www-port "${port}" \
    --auth-none 0 \
	--auth-pam-helper-path "${RSTUDIO_AUTH}" \
	--auth-encrypt-password 0 \
	--rsession-path "${RSESSION_WRAPPER_FILE}" \
	--server-data-dir='/tmp/run' \
	--server-user=$(whoami)

I failed if I setup setenv(“SINGULARITY_BIND”,“/etc,/dev,/media,/mnt,/opt,/srv,/var”)in the ~/ondemand/dev/RStudio/lmodfiles/rocker-rstudio/4.4.luathat is module file for Lmod.

without mounting /etc of host system, I couldn’t use some features such as module, proxy, etc. on the RStudio terminal. It’s dilemma..

Environment:

  • OS: Rocky Linux 8.8
  • Fork OSC bc_rstudio_server repo
  • Container image: rocker rstudio singularity build rocker-rstudio.sif docker://rocker/rstudio:4.4

Thank you for your help travert.