RStudio when launched without Singularity is having strange troubles with authentication

… Okay so can confirm replacing ‘-ne’ with ‘-lt’ in the template/bin/auth totally fixed our issue. I absolutely love that this worked and thanks for the call out… but dang SO much time for that one issue.

Thanks!

1 Like

Full info + Solution: Thanks to @dugan and @maflister
For those who run into this in the future here is the changes that had to be implemented:
Installed rstudio 1.3.1093 natively with the rpm (will test with source later)
yum install -y rstudio-server-rhel-1.3.1093-x86_64.rpm
In thetemplate/script.sh.erb ensure you use the following :

rserver \
 --www-port "${port}" \
 --auth-none 0 \
 --auth-pam-helper-path "${RSTUDIO_AUTH}" \
 --auth-encrypt-password 0 \
 --rsession-path "${RSESSION_WRAPPER_FILE}" \
 --server-data-dir "${TMPDIR}" \
 --secure-cookie-key-file "${TMPDIR}/rstudio-server/secure-cookie-key"

Ref: @maflister specifically https://github.com/mcw-rcc/bc_rcc_rstudio_server

If you haven’t already, you need to update the template/bin/auth and change the ‘-ne’ flag to ‘-lt’

#!/usr/bin/env bash

# Confirm username is supplied

if [[ $# -lt 1 ]]; then

echo "Usage: auth USERNAME"

exit 1

fi

USERNAME="${1}"

# Confirm password environment variable exists

if [[ -z ${RSTUDIO_PASSWORD} ]]; then

echo "The environment variable RSTUDIO_PASSWORD is not set"

exit 1

fi

# Read in the password from user

read -s -p "Password: " PASSWORD

echo ""

if [[ ${USERNAME} == ${USER} && ${PASSWORD} == ${RSTUDIO_PASSWORD} ]]; then

echo "Successful authentication"

exit 0

else

echo "Invalid authentication"

exit 1

fi

Ref: @dugan specifically https://github.com/OSC/bc_osc_rstudio_server/issues/34

What type of logging or error feedback would have helped reduce the amount of time it took to debug this issue? Perhaps there is something we can change in OnDemand to help with debugging problems like this in the future.

Not sure how to accomplish it, but some kind of logging on wether the pam auth found in template/bin/auth was actually being invoked or if it failed out because of syntax would be nice. It was really hard to pinpoint what the actual problem was because as far as we could see there wasn’t anything “erroring”.
If that level of error logging does exist we didn’t have a good place to reference that in the documentation.

I have this same issue with Rstudio server 1.4 container, but not 1.2 or 1.3. Has anyone been able to find the magic to fix it for 1.4 please?
Inspecting the “connect to” part of the web page shows the user/pass being stored correctly, but i have to manually enter them on the next screen in order to enter rstudio

I have issues as well with RStudio server 1.4, it seems authentication request from OOD app is not accepted anymore.
Can anyone help with this?

I think RStudio Server 1.4 still has an issue. Suggest to use RStudio Server 1.3.

I feel something was changed since in the 1.3.x if I enter the “auth-do-sign-in” url I get:

  1. Redirected to “auth-sign-in?error=1”
  2. An error: “Error: Incorrect or invalid username/password”

and that’s expected since I’m just entering it in the browser (GET request without parameters)

But if I do the same in the version 1.4.x I get:

  1. Redirected to “auth-sign-in?appUri=appUri&error=2”
  2. An error: “Error: Temporary server error, please try again”

So either this authentication method got deprecated (maybe it there’s a different url to send the request) or that’s not working “by default” and needs some configuration in “rserver.conf” file.
I’m trying to understand more but if there’s anyone with more expertise which may help it could be useful.
@maflister are you saying this is a known bug which RStudio folks are working on?

@fenz I think might be related to this:

I think it is related (since I feel that’s something they are removing) but the helper itself still works since if you use your userID and the password generated in the “before” script (stored in the connection.yml file) it lets you in.
This is also still documented by “rocker” team in the “singularity” section for their rstudio:4.0.4 (which has rstudio server 1.4.x): Rocker Project
So I’m pretty sure pam authentication still works.

Not sure if this can be the issue but from here: Shiny Server v1.5.20 Configuration Reference

7.3.1.2 Custom login templates now must include a CSRF token field

Shiny Server Pro 1.4.6 includes mitigations against CSRF attacks. For the most part, this should not affect your installation of Shiny Server Pro. The exception is if you are using a custom template for your login page. In that case, you’ll need to add a new field to your login <form> tag:

<input id="csrfToken" type="hidden" name="_csrf" value="{{csrf_token}}"/>

I tried to copy the request from the login page and that seems to be a difference (together with the appUri value):

cURL request

rocker/rstudio:4.0.2 (rstudio server 1.3.x):


–data-raw ‘username=maffiaa&password=test&csrf-token=MISSING+VALUE&appUri=’

rocker/rstudio:4.0.4 (rstudio server 1.4.x):


–data-raw ‘username=maffiaa&password=test&csrf-token=db0dc73d-2964-4325-85cf-664274bffd39&appUri=%2F’

But I’m not sure how to define the token and not even 100% sure that’s the issue.

Edit:
I could not find any option to disable this check but is seems that’s the issue:

This works:

$ curl ‘URL:PORT/auth-do-sign-in’ -H ‘Cookie: csrf-token=TOKEN’ --data-raw ‘username=USERID&password=PASSWORD&csrf-token=TOKEN&appUri=’

This doesn’t work:

$ curl ‘URL:PORT/auth-do-sign-in’ --data-raw ‘username=USERID&password=PASSWORD&csrf-token=TOKEN&appUri=’

I really used “TOKEN” as token and it worked but it needs to have it as cookie (I couldn’t find if it is stored on the server side).
I’ll stop here and see if anyone can help from this point.

Edit 2:
I just figured out this was mentioned already here: Unable to launch fully containerized Rstudio - #20 by dugan

I think I found the solution. The “view.html.erb” should look like:

<%-
 csrftoken="MYTOKEN"
-%>
<script defer src="https://use.fontawesome.com/releases/v5.8.2/js/brands.js" integrity="sha384-GtvEzzhN52RvAD7CnSR7TcPw555abR8NK28tAqa/GgIDk59o0TsaK6FHglLstzCf" crossorigin="anonymous"></script>
<script defer src="https://use.fontawesome.com/releases/v5.8.2/js/fontawesome.js" integrity="sha384-Ia7KZbX22R7DDSbxNmxHqPQ15ceNzg2U4h5A8dy3K47G2fV1k658BTxXjp7rdhXa" crossorigin="anonymous"></script>
<script>
  var hostButton=$( "a.btn.btn-primary.btn-sm.fas.fa-terminal" );
  var hostname=hostButton.text();
  hostButton.replaceWith(hostname);
  document.cookie = "csrf-token=<%= csrftoken %>; path=/rnode/<%= host %>/<%= port %>/; secure";
</script>
<form action="/rnode/<%= host %>/<%= port %>/auth-do-sign-in" method="post" target="_blank">
  <input type="hidden" name="username" value="<%= ENV["USER"] %>"/>
  <input type="hidden" name="password" value="<%= password %>"/>
  <input type="hidden" name="staySignedIn" value="1"/>
  <input type="hidden" name="appUri" value=""/>
  <input id="csrfToken" type="hidden" name="csrf-token" value="<%= csrftoken %>"/>
  <button class="btn btn-primary" type="submit">
    <i class="fab fa-r-project"></i> Connect to RStudio Server
  </button>
</form>

So I declare a variable in ruby to be used as TOKEN and, in the “script” I set it as cookie for the rstudio page.
This can then be specified as “csrf-token” in the POST request. The rest of the file stays the same.

This is an “hello world” of the solution. The TOKEN can clearly be a better one.
Let me know if this works for you as well.

Edit:
Maybe something like this for the token:

<%-
  require 'securerandom'

  csrftoken=SecureRandom.uuid
-%>

Very cool, @fenz! I mistakenly thought that we had to use the random cookie generated by the server but you show that the server will accept whatever the client sets. It does make me wonder what additional security this csrf-token is providing. Thanks.

I just figured this:

should be without “/” at the end of the path, like:
document.cookie = "csrf-token=<%= csrftoken %>; path=/rnode/<%= host %>/<%= port %>; secure";

Since with the “/” RStudio seems to keeping reloading (still not sure why), without it works.

@dugan I agree. I was also looking for the token on the server side and in principle that’s true;
When you GET the login page RStudio server sets the csrf-token in the cookie which will be used for the POST request. But, when it gets the request, RStudio just checks that the csfr-token you use is the same present in the cookie (and not if it was generated by the server).
At the moment this solution is simpler but in case RStudio changes the check we may need to “GET” the login page in the “script” section and parse the cookie to get the csrf-token generated by RStudio. The token is generated for the “session”, so it should work. But, as said, for the moment it seems not needed.

I can confirm we had the same problem and this worked like a charm.

Apologies for this issue. There was a bug that got fixed in the 2.0 release where URLs without a trailing / did not redirect to the backend correctly. So this should at least alleviate that 1 particular issue.

Thank @fenz. For some reason when I tried passing a csrf token in the past it did not work well (plus a lot of issues with Rstudio, especially when using R 4.1.0). Created a new container with the latest version and your solution worked great. Wanted to try setting RSTUDIO_DATA_HOME to get fully isolated user cache locations (since we have a paid instance Rstudio as well, this would likely clash if they run from the same default of .local/share/rstudio). This worked really well when passed to rserver.

Thanks, @fenz for finding this solution. One thing I noticed is that the second time I click the “Connect to RStudio Server” button, the auth-do-sign-in lands me on the regular URL, but with an “appUri” at the end and I get a “page not found” error. If I knock off the appUri or blow away all the cached items for this server, I’m back to good. Has anyone else encountered this / found a way around it?

Yes, @rena, I’ve noticed this too. I think the problem has to do with generating the cookie value in the
view.html.erb file:

<%-
  require 'securerandom'

  csrftoken=SecureRandom.uuid
-%>

This changes the cookie value each time you click on the launch button. We want to generate it once
for the session and store it in the connection.yml file for the job. So I removed the above bit from view.html.erb and added this to to my script.sh.erb:

# add csrftoken to connection.yml file
(
csrftoken=$(python -c 'from uuid import uuid4; print(uuid4())')
connection=${PWD}/connection.yml
while true; do
  if [ -s $connection ]; then
     echo "csrftoken: $csrftoken" >>$connection
     exit 0
  fi
  sleep 1
done
) &

It needs to go before the "cd “${HOME}” so $PWD still refers to the job directory.
This seems to correct the problem for me. It is a bit of a hack but I couldn’t find a simpler way to
add additional elements to connection.yml.

We are using an isolated home, it works quite well. The user specify the home in the app form and we mount it with “-H” option in Singularity (together with -c option). This way the user can manage different “workspaces”.