Osc_bc_rstudio_server streamlined submit.yml.erb

I’d guess it’s the DB it’s trying to create. I see you have one, but I’d guess it’s a mounting discrepancy.

What scheduler are you on? Slurm gives job specific TMPDIR, so using $TMPDIR in the script, will evaluate to /tmp/some_job_id but you’ve mounted it in as just /tmp in the container (line 100).

So you need to reference /tmp in lines 111 and 112 because that’s where the files are in the container when the server starts.

100 -B “$TMPDIR:/tmp” 
...
111 --database-config-file “/tmp/db.conf”
112 --server-data-dir “/tmp/rstudio-server” 

Thanks – understanding to reference “inside” the container paths was important. That allows the server to start running when launching ‘script.sh’ from the command line in the ood working directory.

When I adapt those changes into the ‘script.sh.erb’ and launch the app, the rsession.log is still not generated into the working directory. The operational state is that rserver is started on a compute node:
mrd20 868792 868767 0 17:38 ? 00:00:00 bash /home/mrd20/ondemand/data/sys/dashboard/batch_connect/dev/RStudio_Pioneer/output/a4e2a6d1-970f-4dfa-ab4b-39aa549d5eb0/script.sh
mrd20 868855 868792 0 17:38 ? 00:00:00 Singularity runtime parent
mrd20 868869 868855 0 17:38 ? 00:00:00 rserver --www-port 21673 --auth-none 0 --auth-pam-helper-path /home/mrd20/ondemand/data/sys/dashboard/batch_connect/dev/RStudio_Pioneer/output/a4e2a6d1-970f-4dfa-ab4b-39aa549d5eb0/bin/auth --auth-encrypt-password 0 --database-config-file /tmp/db.conf --server-data-dir /tmp/rstudio-server --server-user=mrd20 --rsession-path /home/mrd20/ondemand/data/sys/dashboard/batch_connect/dev/RStudio_Pioneer/output/a4e2a6d1-970f-4dfa-ab4b-39aa549d5eb0/rsession.sh

The ‘Connect to RStudio Server’ radio button launches a new tab, in which a ‘Signin to Rstudio’ is presented. Is one meant to manually use the ‘connection.yml’ info to signin? I can’t negotiate this page, and again, there is no ‘rsession.log’ to refer to.

And yet…Progress!!
ps: from script.sh.erb
89 # Launch the RStudio Server
90
91 ## Prepare temp directory for sqlite db and conf
92
93 echo “Temp directory for db: $TMPDIR”
94 echo “create var directory under WORKING_DIR”
95 mkdir “$WORKING_DIR/var”
96
97 echo “Starting up rserver… at $(date)”
98
99 #
100 singularity run
101 -B “$TMPDIR:/tmp”
102 -B “$WORKING_DIR”
103 -B “$WORKING_DIR/var:/var/lib/rstudio-server”
104 -B /home
105 “$RSTUDIO_SERVER_IMAGE”
106 --www-port “${port}”
107 --auth-none 0
108 --auth-pam-helper-path “${RSTUDIO_AUTH}”
109 --auth-encrypt-password 0
110 --database-config-file “/tmp/db.conf”
111 --server-data-dir “/tmp/rstudio-server”
112 --server-user=$(whoami)
113 --rsession-path “${RSESSION_WRAPPER_FILE}”
114
115 echo ‘Singularity as exited…’

Maybe you need to initialize some empty directories?

Also is this file being created somewhere? Every file in the template directory will be copied to $WORKING_DIR.

That’s how I mount configs like the db.conf.

Plus, since every file is templated, I can configure the log directory to the effectively the same directory as WORKING_DIR.

Thanks, Jeff – The local script.sh.erb remains pretty faithful to the osc repo version.
What I’ve done is attempted to strip out the ruby conditionals, to aid my understanding.

I’m still confused over how the reverse proxy is supported with only the use of a “basic” template, from the submit.yml.erb.

And the rserver signin: how does that work? I’m hoping that knowing the procedure will help understand the bindings required to implement the server. Particularly, why am I still not finding the rserver.log?

Thanks

#!/usr/bin/env bash
<%-
session_dir = session.staged_root
%>

Load the required environment

setup_env () {

export WORKING_DIR="<%= session_dir %>"

module purge

The rserver container module should set these environment variables:

SINGULARITY_BINDPATH="/etc,/media,/mnt,/opt,/run,/srv,/usr,/var,/users"

RSTUDIO_SERVER_IMAGE="/usr/local/project/ondemand/singularity/rstudio/rstudio_launcher_centos7.simg"

PATH="$PATH:/path/to/R:/path/to/rstudio/rserver"

export RSTUDIO_SERVER_IMAGE="/home/mrd20/hpcdemo/rstudio/rstudio-launcher-centos-8.sif"
export SINGULARITY_BINDPATH="/scratch,/usr/local"
export PATH="$PATH:/usr/local/easybuild/software/RStudio-Server/1.4.1717-foss-2021a-Java-11-R-4.1.0/bin"

SINGULARITY_BINDPATH is being used to bind all RStudio’s requirements from

the host into the guest, and so those values may vary between sites.

module load ondemand-vnc/2.0

module load singularity/3.8.6
module load RStudio-Server/1.4.1717-foss-2021a-Java-11-R-4.1.0
which rsession
export LD_LIBRARY_PATH=/usr/lib:$LD_LIBRARY_PATH
export SINGULARITYENV_LD_LIBRARY_PATH="$LD_LIBRARY_PATH"
}
setup_env

Start RStudio Server

PAM auth helper used by RStudio

export RSTUDIO_AUTH="$WORKING_DIR/bin/auth"

Generate an rsession wrapper script

export RSESSION_WRAPPER_FILE="$WORKING_DIR/rsession.sh"
(
umask 077
sed ‘s/^ {2}//’ > “$WORKING_DIR/rsession.sh” << EOL
#!/usr/bin/env bash

Log all output from this script

export RSESSION_LOG_FILE="$WORKING_DIR/rsession.log"

exec &>>"${RSESSION_LOG_FILE}"
set -x

rsession.sh doesn’t share the same env as the outside script, so these

need to be set explicitly

export R_LIBS_SITE="${R_LIBS_SITE}"
export TZ=“US/Eastern”
export HOME="$HOME"
export MODULEPATH_ROOT="$MODULEPATH_ROOT"
export MODULEPATH="$MODULEPATH"
export LMOD_PKG="$LMOD_PKG"

env

Launch the original command

echo “Launching rsession…”
wrsession=which rsession
echo “which rsession? $wrsession?”
exec rsession --r-libs-user “${R_LIBS_USER}” “${@}”
EOL
)
chmod 700 “$WORKING_DIR/rsession.sh”

Set working directory to home directory

echo “Home check:\nHOME is: $HOME\nSHOME is $SINGULARITY_HOME”

cd “${SINGULARITY_HOME}”

Output debug info

hostname
whoami
echo “$USER”

rstudio runtime stuff

echo -e “provider=sqlite\ndirectory=$TMPDIR/db” >> $TMPDIR/db.conf
mkdir “$TMPDIR/run”
mkdir “$TMPDIR/lib”

server log directory in this job’s working directory

mkdir -p “$WORKING_DIR/logs”

set -x

Launch the RStudio Server

Prepare temp directory for sqlite db and conf

echo “Temp directory for db: $TMPDIR”
#echo “create var directory under WORKING_DIR”
#mkdir “$WORKING_DIR/var”

echo “Starting up rserver… at $(date)”

#SINGULARITYENV_LD_LIBRARY_PATH="$LD_LIBRARY_PATH"
singularity run
-B “$TMPDIR:/tmp”
-B “$WORKING_DIR”
-B “$TMPDIR/lib:/var/lib/rstudio-server”
-B /home
“$RSTUDIO_SERVER_IMAGE”
–www-port “${port}”
–auth-none 0
–auth-pam-helper-path “${RSTUDIO_AUTH}”
–auth-encrypt-password 0
–rsession-path “${RSESSION_WRAPPER_FILE}”
–database-config-file “/tmp/db.conf”
–server-data-dir “/tmp/run”
–server-user=$(whoami)
–secure-cookie-key-file “/tmp/rstudio-server/secure-cookie-key” \

echo ‘Singularity as exited…’

Does this need to be static /tmp because that’s the value inside the container?

This is part of how auth works.

The other bit that’s an update from 1.2 to 1.3 where we create a CSRF token in the script.sh.erb, passit to the view in submit.yml.erb (in conn_params) then use it in the view.html.erb.

If you’re still not booting correctly, take advantage of that set -x there. It telling you what command it’s issuing.

You have a working version correct? What’s are the differences between the working version and the version you can submit and generate now?

Sorry for not being more direct in reponse to your questions about the database files:

[mrd20@compt351 be955dbb-a98e-45ad-9454-ca476df1d8e3]$ ls -R /tmp/job.11010.hpc
/tmp/job.11010.hpc:
db.conf job.11010.hpc lib rstudio-server run

/tmp/job.11010.hpc/job.11010.hpc:
db

/tmp/job.11010.hpc/job.11010.hpc/db:
rstudio.sqlite

/tmp/job.11010.hpc/lib:

/tmp/job.11010.hpc/rstudio-server:
secure-cookie-key

/tmp/job.11010.hpc/run:
rstudio-rsession

/tmp/job.11010.hpc/run/rstudio-rsession:

From: ```
-B $WORKING_DIR/etc:/etc/rstudio \


Do I understand correctly from the line above that you place the db.conf in the $WORKING_DIR/etc, and then bind that into the container at /etc/rstudio?

So the session launches, rserver is running on the compute node, and the view presents

Connect to RStudio Server then

In the code that I’m working with ‘before.sh’ is generating the password and the csrf-token.

# Export the module function if it exists
[[ $(type -t module) == "function" ]] && export -f module

# Find available port to run server on
port=$(find_port ${host})

# Define a password and export it for RStudio authentication
password="$(create_passwd 16)"

# rstudio 1.4+ needs a csrf token
csrf_token=<%= SecureRandom.uuid %>

export RSTUDIO_PASSWORD="${password}"
template/before.sh.erb (END)

In the script.sh, we have:

export RSTUDIO_AUTH="$WORKING_DIR/bin/auth"
<..snip..>
--auth-pam-helper-path "${RSTUDIO_AUTH}"

So this appears that the flag “–auth-pam-helper-path” executes the script to pull the RSTUDIO_PASSWORD from the environment variable. before.sh also generates the csrf_token, but only as a local variable. So for authentication, the only binding that seems relevant is for the $WORKING_DIR, where before.sh and bin/auth are to be found.

Unfortunately, I’m still not able to see rsession.log, which as you earlier pointed out, would be quite helpful with some of the detailed troubleshooting. Could you please review what needs done to gain access to the rsession.log? Thanks

How to get the log directory?

I used this configuration file with a mount point to etc. Looking at the rserver help output I don’t believe there’s an equivalent for this configuration like there is for the --database-config-file. So I was likely forced to mount it because there’s no other entrypoint to change it.

What could be wrong with your auth

As for what’s going on with your authentication there are these changes you need to make

  1. First the auth file needs updated
# template/bin/auth
# this used to be ne. Now it needs to be lt.
if [[ $# -lt 1 ]]; then
  1. Generate the csrf_token. You have that.
  2. pass the csrf_token back to the view. Do this by using conn_params in your submit.yml.erb
---
batch_connect:
  template: "basic"
  conn_params:
    - csrf_token
  1. Use the csrf_token in the view. That’s all the javascript addition and new hidden form field in this git diff I’d used in my original reply.