# Unable to launch fully containerized Rstudio

**URL:** <https://discourse.openondemand.org/t/unable-to-launch-fully-containerized-rstudio/948>\
**Category:** Get Help\
**Tags:** question\
**Created:** [June 3, 2020, 3:34pm UTC](https://discourse.openondemand.org/t/unable-to-launch-fully-containerized-rstudio/948 "2020-06-03T15:34:23Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![charles8ronson](https://sea1.discourse-cdn.com/flex015/user_avatar/discourse.openondemand.org/charles8ronson/32/446_2.png) [@charles8ronson](https://discourse.openondemand.org/u/charles8ronson)\
**Post date:** [June 3, 2020, 3:34pm UTC](https://discourse.openondemand.org/t/unable-to-launch-fully-containerized-rstudio/948/1 "2020-06-03T15:34:23Z")

</div>

We are trying to fully containerize Rstudio however we run the image we get the following error. Is there something we are missing here? Ref:

Jun 03 08:09:53 pdlhpc1cn001 rserver[13797]: ERROR Could not change permissions for specified ‘server-data-dir’ - the directory (/var/run/rstudio-server) must be writeable by all users and have the sticky bit set; LOGGED FROM: int main(int, char\* const\*) src/cpp/server/ServerMain.cpp:608

Jun 03 08:09:53 pdlhpc1cn001 rserver[13797]: ERROR system error 30 (Read-only file system) [path: /var/run/rstudio-server]; OCCURRED AT rstudio::core::Error rstudio::core::system::changeFileMode(const rstudio::core::FilePath&, mode\_t) src/cpp/server/ServerMain.cpp:62; LOGGED FROM: int main(int, char\* const\*) src/cpp/server/ServerMain.cpp:609

Jun 03 08:14:57 pdlhpc1cn001 rserver[15512]: ERROR system error 13 (Permission denied) [path: /var/run/rstudio-server, target-dir:]; OCCURRED AT rstudio::core::Error rstudio::core::FilePath::createDirectory(const string&) const src/cpp/shared\_core/FilePath.cpp:634; LOGGED FROM: int main(int, char\* const\*) src/cpp/server/ServerMain.cpp:555

---

<div class="post-metadata">

**Author:** ![mario](https://sea1.discourse-cdn.com/flex015/user_avatar/discourse.openondemand.org/mario/32/662_2.png) [@mario](https://discourse.openondemand.org/u/mario)\
**Post date:** [June 3, 2020, 3:52pm UTC](https://discourse.openondemand.org/t/unable-to-launch-fully-containerized-rstudio/948/2 "2020-06-03T15:52:41Z")

</div>

@charles8ronson

Can you share the `script.sh.erb` that launches RStudio?

---

<div class="post-metadata">

**Author:** ![eisenefaust](https://sea1.discourse-cdn.com/flex015/user_avatar/discourse.openondemand.org/eisenefaust/32/470_2.png) [@eisenefaust](https://discourse.openondemand.org/u/eisenefaust)\
**Post date:** [June 3, 2020, 3:58pm UTC](https://discourse.openondemand.org/t/unable-to-launch-fully-containerized-rstudio/948/3 "2020-06-03T15:58:01Z")

</div>

We figured out that we needed to mount /tmp to /var/run in the container  
but now it’s not adhering to the authorization portion of the script:

````auto

# Load the required environment
setup_env () {
  # Additional environment which could be moved into a module
  # Change these to suit
  export RSTUDIO_SERVER_IMAGE="/depot/tmp/singularity-images/de-gmorto/rstudio-server-centos7r4.sif"
#don't change paths in the container. everything should work since it's all contained inside the container above
  export SINGULARITY_BINDPATH="/tmp:/var/run,/active,/archive"
  export PATH="/R/4.0.0/bin:/usr/lib/rstudio-server/bin:$PATH"
  export SINGULARITYENV_PATH="$PATH"
  export MODULEPATH="/depot/Modules/modulefiles"
  # In Singularity 3.5.x it became necessary to explicitly pass LD_LIBRARY_PATH
  # to the singularity process
  # export SINGULARITYENV_LD_LIBRARY_PATH="$LD_LIBRARY_PATH:/usr/lib:/usr/lib64"
}
setup_env

#
# Start RStudio Server
#
module use /depot/Modules/modulefiles
module load python

# PAM auth helper used by RStudio
export RSTUDIO_AUTH="${PWD}/bin/auth"

# Generate an `rsession` wrapper script
export RSESSION_WRAPPER_FILE="${PWD}/rsession.sh"
(
umask 077
sed 's/^ \{2\}//' > "${RSESSION_WRAPPER_FILE}" << EOL
  #!/usr/bin/env bash

  # Log all output from this script
  export RSESSION_LOG_FILE="${PWD}/rsession.log"

  exec &>>"\${RSESSION_LOG_FILE}"

  # Launch the original command
  echo "Launching rsession..."
  set -x
  exec rsession --r-libs-user "${R_LIBS_USER}" "\${@}"
EOL
)
chmod 700 "${RSESSION_WRAPPER_FILE}"

# Set working directory to home directory
cd "${HOME}"

export TMPDIR="$(mktemp -d)"
export TMPDIR="$(mktemp -d)"

mkdir -p "$TMPDIR/rstudio-server"
python -c 'from uuid import uuid4; print(uuid4())' > "$TMPDIR/rstudio-server/secure-cookie-key"
chmod 0600 "$TMPDIR/rstudio-server/secure-cookie-key"

module unload python

set -x
# Launch the RStudio Server
echo "Starting up rserver..."

singularity run -B "$TMPDIR:/tmp" "$RSTUDIO_SERVER_IMAGE" \
 --www-port "${port}" \
 --auth-none 0 \
 --auth-pam-helper-path "${RSTUDIO_AUTH}" \
 --auth-encrypt-password 0 \
 --rsession-path "${RSESSION_WRAPPER_FILE}"

echo 'Singularity has exited...'```
````

---

<div class="post-metadata">

**Author:** ![alanc](https://sea1.discourse-cdn.com/flex015/user_avatar/discourse.openondemand.org/alanc/32/2524_2.png) [@alanc](https://discourse.openondemand.org/u/alanc)\
**Post date:** [June 3, 2020, 4:12pm UTC](https://discourse.openondemand.org/t/unable-to-launch-fully-containerized-rstudio/948/4 "2020-06-03T16:12:09Z")

</div>

Have you looked at any of the other threads on here about Rstudio? A lot of people have been able to get it working, but there always seem to be certain containerization or authentication issues that crop up. Most of the threads have solutions posted. For example:

> [@Rserver in the Singularity container vs rstudio-server installed from rpm](https://discourse.openondemand.org/t/rserver-in-the-singularity-container-vs-rstudio-server-installed-from-rpm/312):
>
> Hi. We are trying to install the RStudio app following the successful model of Jupyter. After starting a session the output log has: FATAL: exec /.singularity.d/actions/run failed: no such file or directory The singularity container was created using “singularity pull”. It seems to call “rserver” but we installed RStudio on each node using the rpm which uses “rstudio-server” as the executable. Am I missing something obvious? Thanks Chris

> [@Rstudio app works BUT each session begins with a single auth failure that must be ignored](https://discourse.openondemand.org/t/rstudio-app-works-but-each-session-begins-with-a-single-auth-failure-that-must-be-ignored/660):
>
> Hiya everyone, We’re running centos 6 and slurm 19, and are one OOD release behind. I just setup the RStudio interactive app but am quite perplexed by this reproducible and ignorable authentication error that follows immediately after clicking “Connect to RStudio Server” for each session: “Error: Incorrect or invalid username/password” One can simply go back to the OOD tab and click the button again (with or without closing this goofy error tab) and ev…

> [@RStudio without Singularity?](https://discourse.openondemand.org/t/rstudio-without-singularity/773):
>
> I thought I’d read that OOD was now able to run RStudio without relying on Singularity but can’t find documentation to support that notion. Was I dreaming?? Thanks - Susan Litzinger PSC

---

<div class="post-metadata">

**Author:** ![eisenefaust](https://sea1.discourse-cdn.com/flex015/user_avatar/discourse.openondemand.org/eisenefaust/32/470_2.png) [@eisenefaust](https://discourse.openondemand.org/u/eisenefaust)\
**Post date:** [June 3, 2020, 11:29pm UTC](https://discourse.openondemand.org/t/unable-to-launch-fully-containerized-rstudio/948/5 "2020-06-03T23:29:10Z")

</div>

Hello, that was not the solution. It was part of the solution. Authorization is still not functioning. I get to the login and the username/password aren’t getting validated to allow me to log in.when I type them in manually, it also doesn’t work.  
This is following the button Connect to RStudio Server:

 ![image](https://us1.discourse-cdn.com/flex015/uploads/osc/original/1X/cd6949719668eba541bae718ff2f632d8a46f031.png)

---

<div class="post-metadata">

**Author:** ![eisenefaust](https://sea1.discourse-cdn.com/flex015/user_avatar/discourse.openondemand.org/eisenefaust/32/470_2.png) [@eisenefaust](https://discourse.openondemand.org/u/eisenefaust)\
**Post date:** [June 3, 2020, 11:29pm UTC](https://discourse.openondemand.org/t/unable-to-launch-fully-containerized-rstudio/948/6 "2020-06-03T23:29:53Z")

</div>

This is where it fails after I launch it.  
 ![image](https://us1.discourse-cdn.com/flex015/uploads/osc/original/1X/ba22529b0e555c217d499f3aafe25dfa1b429814.png)

I can’t add new posts to this topic so I’m editing this post:

I mounted /home  
I updated the template/bin/auth to include the changes for rstudio server 1.3 support  
still failing at authorization phase.

I installed within the container:  
R-4.0.0.tar.gz to /R/4.0.0  
rstudio-server-rhel-1.3.959-x86\_64.rpm to default location: /usr/lib/rstudio-server/bin

Still have auth issues. Any other ideas would be greatly appreciated.

---

<div class="post-metadata">

**Author:** ![jeff.ohrstrom](https://sea1.discourse-cdn.com/flex015/user_avatar/discourse.openondemand.org/jeff.ohrstrom/32/136_2.png) [@jeff.ohrstrom](https://discourse.openondemand.org/u/jeff.ohrstrom)\
**Post date:** [June 4, 2020, 1:19pm UTC](https://discourse.openondemand.org/t/unable-to-launch-fully-containerized-rstudio/948/7 "2020-06-04T13:19:07Z")

</div>

I’d ensure that HOME is mounted too. You seem to be passing credentials OK, but can the container find this file `--auth-pam-helper-path "{RSTUDIO\_AUTH}"` (It ends up being ~/ondemand/data/sys/dashboard/batch\_connect/sys/rstudio/output//bin/auth)?

---

<div class="post-metadata">

**Author:** ![maflister](https://sea1.discourse-cdn.com/flex015/user_avatar/discourse.openondemand.org/maflister/32/139_2.png) [@maflister](https://discourse.openondemand.org/u/maflister)\
**Post date:** [June 4, 2020, 2:30pm UTC](https://discourse.openondemand.org/t/unable-to-launch-fully-containerized-rstudio/948/8 "2020-06-04T14:30:12Z")

</div>

What version of RStudio Server are you using? We noticed that the new 1.3 version fails to authenticate with “Error: Incorrect or invalid username/password”. We just didn’t get around to posting the issue yet.

---

<div class="post-metadata">

**Author:** ![dugan](https://avatars.discourse-cdn.com/v4/letter/d/eada6e/32.png) [@dugan](https://discourse.openondemand.org/u/dugan)\
**Post date:** [June 4, 2020, 4:39pm UTC](https://discourse.openondemand.org/t/unable-to-launch-fully-containerized-rstudio/948/9 "2020-06-04T16:39:05Z")

</div>

Hi,

The authentication issue in 1.3 is due to a change in how the auth-pam-helper script is invoked by rserver. Previously, it was called with just one argument, $USERNAME.  
In 1.3 it is called with three arguments, “$USERNAME rstudio 1”. The test in line 4 of the bin/auth script:

```auto
#!/usr/bin/env bash

# Confirm username is supplied
if [[$# -ne 1]]; then
  echo "Usage: auth USERNAME"
  exit 1
fi
...

```

fails and needs to be modified to handle the extra arguments. I just replaced “-ne” with “-lt” to get authentication to work.

---

<div class="post-metadata">

**Author:** ![mario](https://sea1.discourse-cdn.com/flex015/user_avatar/discourse.openondemand.org/mario/32/662_2.png) [@mario](https://discourse.openondemand.org/u/mario)\
**Post date:** [June 4, 2020, 5:18pm UTC](https://discourse.openondemand.org/t/unable-to-launch-fully-containerized-rstudio/948/10 "2020-06-04T17:18:00Z")

</div>

@dugan Thanks for this!

I opened an issue to get this merged into `bc_osc_rstudio_server` here [https://github.com/OSC/bc\_osc\_rstudio\_server/issues/34](https://github.com/OSC/bc_osc_rstudio_server/issues/34)

---

<div class="post-metadata">

**Author:** ![charles8ronson](https://sea1.discourse-cdn.com/flex015/user_avatar/discourse.openondemand.org/charles8ronson/32/446_2.png) [@charles8ronson](https://discourse.openondemand.org/u/charles8ronson)\
**Post date:** [June 4, 2020, 5:44pm UTC](https://discourse.openondemand.org/t/unable-to-launch-fully-containerized-rstudio/948/11 "2020-06-04T17:44:23Z")

</div>

Onbehalf of eisenfaust (can only do 3 posts atm):

I mounted /home  
I updated the template/bin/auth to include the changes for rstudio server 1.3 support  
still failing at authorization phase.

I installed within the container:  
R-4.0.0.tar.gz to /R/4.0.0  
rstudio-server-rhel-1.3.959-x86\_64.rpm to default location: /usr/lib/rstudio-server/bin

Still have auth issues

---

<div class="post-metadata">

**Author:** ![maflister](https://sea1.discourse-cdn.com/flex015/user_avatar/discourse.openondemand.org/maflister/32/139_2.png) [@maflister](https://discourse.openondemand.org/u/maflister)\
**Post date:** [June 4, 2020, 8:07pm UTC](https://discourse.openondemand.org/t/unable-to-launch-fully-containerized-rstudio/948/12 "2020-06-04T20:07:33Z")

</div>

Thanks for this. We’ve now fully remove Singularity from our RStudio Server app.

---

<div class="post-metadata">

**Author:** ![maflister](https://sea1.discourse-cdn.com/flex015/user_avatar/discourse.openondemand.org/maflister/32/139_2.png) [@maflister](https://discourse.openondemand.org/u/maflister)\
**Post date:** [June 4, 2020, 8:57pm UTC](https://discourse.openondemand.org/t/unable-to-launch-fully-containerized-rstudio/948/13 "2020-06-04T20:57:40Z")

</div>

Try pointing RStudio Server to your /tmp.

```auto
singularity run -B "$TMPDIR:/tmp" "$RSTUDIO_SERVER_IMAGE" \
--www-port "${port}" \
--auth-none 0 \
--auth-pam-helper-path "${RSTUDIO_AUTH}" \
--auth-encrypt-password 0 \
--rsession-path "${RSESSION_WRAPPER_FILE}" \
--server-data-dir /tmp

```

---

<div class="post-metadata">

**Author:** ![eisenefaust](https://sea1.discourse-cdn.com/flex015/user_avatar/discourse.openondemand.org/eisenefaust/32/470_2.png) [@eisenefaust](https://discourse.openondemand.org/u/eisenefaust)\
**Post date:** [June 4, 2020, 10:10pm UTC](https://discourse.openondemand.org/t/unable-to-launch-fully-containerized-rstudio/948/14 "2020-06-04T22:10:59Z")

</div>

Unfortunately that didn’t solve my issue for the authorization issue I’m currently having.  
It does seem to be a potentially good way to not have to map /tmp:/var/run. If we end up doing this, I think we will need to also supply:  
`--server-pid-file arg (=/var/run/rstudio-server.pid)`  
probably something like:  
`--server-pid-file /tmp/rstudio-server.pid`

The issue with the /var/run may be due to write permissions since it was built with fakeroot:  
`singularity build -f`

Would love to work out this Authorization issue though.

---

<div class="post-metadata">

**Author:** ![holtgrewe](https://sea1.discourse-cdn.com/flex015/user_avatar/discourse.openondemand.org/holtgrewe/32/474_2.png) [@holtgrewe](https://discourse.openondemand.org/u/holtgrewe)\
**Post date:** [June 4, 2020, 10:45pm UTC](https://discourse.openondemand.org/t/unable-to-launch-fully-containerized-rstudio/948/15 "2020-06-04T22:45:14Z")

</div>

Hi, are you certain that you need to have this bind?

We’re using the Singularity Hub RStudio image

- [https://singularity-hub.org/collections/463](https://singularity-hub.org/collections/463)

with this configuration

- [https://github.com/bihealth/ood-bih-rstudio-server](https://github.com/bihealth/ood-bih-rstudio-server)

In this configuration, I’m only binding /fast which is our GPFS file system mount which contains the user’s homes.

---

<div class="post-metadata">

**Author:** ![ssivy](https://sea1.discourse-cdn.com/flex015/user_avatar/discourse.openondemand.org/ssivy/32/106_2.png) [@ssivy](https://discourse.openondemand.org/u/ssivy)\
**Post date:** [June 10, 2020, 2:08pm UTC](https://discourse.openondemand.org/t/unable-to-launch-fully-containerized-rstudio/948/16 "2020-06-10T14:08:04Z")

</div>

Did you try using the --writable-tmpfs option when launching singularity?

---

<div class="post-metadata">

**Author:** ![eisenefaust](https://sea1.discourse-cdn.com/flex015/user_avatar/discourse.openondemand.org/eisenefaust/32/470_2.png) [@eisenefaust](https://discourse.openondemand.org/u/eisenefaust)\
**Post date:** [June 11, 2020, 6:28pm UTC](https://discourse.openondemand.org/t/unable-to-launch-fully-containerized-rstudio/948/17 "2020-06-11T18:28:12Z")

</div>

Without the bind, it fails to open the port.

The `--writable-tmpfs` is interesting to me but it fails to open a port with that added to the singularity run in the script.

version 1.1.463 works fine for me when I use R 3.6.3  
But rstudio server version 1.3.959 fails to authenticate. I’m using R 4.0.0 in this image

---

<div class="post-metadata">

**Author:** ![jeff.ohrstrom](https://sea1.discourse-cdn.com/flex015/user_avatar/discourse.openondemand.org/jeff.ohrstrom/32/136_2.png) [@jeff.ohrstrom](https://discourse.openondemand.org/u/jeff.ohrstrom)\
**Post date:** [June 11, 2020, 7:03pm UTC](https://discourse.openondemand.org/t/unable-to-launch-fully-containerized-rstudio/948/18 "2020-06-11T19:03:09Z")

</div>

I would say next steps are answering these two questions. Can the job find the `$RSTUDIO_AUTH` in the container (if you’re not mounting home as an example)? Then is the `$RSTUDIO_AUTH` working as it’s expected?

1. can it find the `$RSTUDIO_AUTH` file. To debug this, I’d write a simple wrapper like this. This ensures that the file exists in the container and you’re correctly mounting HOME.

```auto
#!/bin/bash

# load any modules you may need
module purge
module restore
module load ondemand
module load app_rstudio_server/3.6.3 rstudio_launcher/centos7

TMPDIR=$(mktemp -d)

# replace with your file location
RSTUDIO_AUTH="/users/PZS0714/johrstrom/ondemand/data/sys/dashboard/batch_connect/sys/bc_osc_rstudio_server/output/72ca052a-6af0-47dd-a521-0b2ed966170e/bin/auth"

# notice I'm using exec here (so i can execute ls), but it's the same arguments we use 
SINGULARITYENV_LD_LIBRARY_PATH="$LD_LIBRARY_PATH" singularity exec --contain -B "$TMPDIR:/tmp" \
 "$RSTUDIO_SERVER_IMAGE" \
 ls -lrt $RSTUDIO_AUTH

```

1. The right env variables are being populated checked in the RSTUDIO\_AUTH file. I’d say modify the `$RSTUDIO_AUTH` file to redirect output to a log file. It looks like stdout and stderr from this script are being eaten, so let’s redirect it to a file to see where exactly it’s failing. The `$RSTUDIO_PASSWORD` is created in the `before.sh` and exported from there. And $USER, is well, the system $USER.

---

<div class="post-metadata">

**Author:** ![mcuma](https://sea1.discourse-cdn.com/flex015/user_avatar/discourse.openondemand.org/mcuma/32/171_2.png) [@mcuma](https://discourse.openondemand.org/u/mcuma)\
**Post date:** [February 25, 2021, 8:45pm UTC](https://discourse.openondemand.org/t/unable-to-launch-fully-containerized-rstudio/948/19 "2021-02-25T20:45:19Z")

</div>

Agh, thanks @dugan, I spent way too much time on this silly thing. How did you figure the extra auth-pam-helper arguments out? strace?

Did anyone succeed with getting the RStudio 1.4 to work in user space? For me, up to 1.3 works fine, but 1.4 throws another annoying error:  
[rserver] ERROR system error 13 (Permission denied) [path: /var/lib/rstudio-server, target-dir:]

I am not successful in finding a runtime option that would steer this directory somewhere writeable.

---

<div class="post-metadata">

**Author:** ![dugan](https://avatars.discourse-cdn.com/v4/letter/d/eada6e/32.png) [@dugan](https://discourse.openondemand.org/u/dugan)\
**Post date:** [February 25, 2021, 10:25pm UTC](https://discourse.openondemand.org/t/unable-to-launch-fully-containerized-rstudio/948/20 "2021-02-25T22:25:13Z")

</div>

@mcuma, I have not been able to get 1.4 working in a way that is suitable for users yet. The problem you identified happens because it wants to create a sqlite database in that directory.  
There is a new commandline option “–database-config-file path\_to\_config\_file”, and the directive  
to put in that config file to change the location of the database is:

directory=“some\_directory”

With these changes to script.sh.erb the server will start without error but when I try to connect I get the dreaded rstudio login screen. If I enter my username and the “passwd” from the connection.yml file I almost get in. This process added “AppUri” to the end of the url and I get a “page not found” error. If I remove that bit from the url in the address bar and type return I do get in.

The real problem here is that they added some extra “security” features in 1.4. In particular they  
added a csrf-token to the login screen. We need to figure out what the value is and pass that in along with the username and passwd in order to get past the login screen. I don’t know how to do  
this automatically. This is where I’m stuck. I added this line to view.html.erb:

I still get the login screen when I connect but I can just look at the page source and see what  
csrf-token is set to and then manually add a line to connection.yml:

csrfToken: …

then when I connect I get in without hitting the login screen.

So that’s where I’m at.

[Next page](https://discourse.openondemand.org/t/unable-to-launch-fully-containerized-rstudio/948.md?page=2)