RStudio Server in OnDemand with Singularity 3.5.x

Folks,

I wanted to update the community on some work and issues that have recently come up with RStudio Server and OnDemand.

At OSC we recently upgraded our Singularity installation to version 3.5.1 which had an undocumented change in behavior regarding LD_LIBRARY_PATH. The newer version requires an explicit export of SINGULARITYENV_LD_LIBRARY_PATH=$LD_LIBRARY_PATH to the Singularity process where prior versions did not. I have updated the documentation, OSC’s implementation, and the generic RStudio Server example on Github. Given the friction with the Singularity based implementation, I thought that it would be a good time to look into an alternative implementation with unshare.

unshare is utility that uses kernel features to allow unprivileged users to disconnect from some of the parent process namespaces. Where unshare becomes relevant for RStudio Server is we can create process-private mount points that shadow system directories like /tmp.

So how can we use unshare to solve the /tmp problem? From within our unshared process, if we unshare the mount namespace (-m), we can then mount a new tmpfs file system at /tmp which the host context will never see, but to do that we need to be root inside the unshared context. Easy enough, just pass the -r or --map-root-user flag to unshare and away we go.

Good so far, but this is where things start going sideways. Within the unshared context whoami will return ‘root’, and because we are not really root we cannot sudo su to our real user, presumably because of a technical issue that would otherwise allow us to become any user thus creating security problems. RStudio itself behaves differently when launched as root, including backgrounding itself. In the example that I have working RStudio crashes/exits(?) almost immediately after launch. Assuming that we solve the crashing problem there is still the issue that files created by our mapped root user show up in the host context as owned by nfsnobody, at least in the VM I use for development. While we can have our script change ownership after the fact, possibly doing so on the fly using by doing something clever (or fragile) with a file system watcher, it is far from ideal. Until I can get RStudio to actually work I am not sure what the per-user RStudio process will do in terms of file ownership.

If any one is interested in viewing or contributing to the work I have done thus far then have a look at the example branch : OSC/bc_example_rstudio#unshare.

Thanks for reading, and please let me know if there’s something I missed.

- Morgan