Thanks for the context. First, Ruby is an absolutely splendid language and I hope you enjoy it as much as I do.
Secondly; the TLDR of this answer is: We have never deployed OOD to Kubernetes so we don’t have that recipe available, and indeed I suspect it will be hard to do and get right.
Though, you may not be the first to blaze this trail, I don’t know how has other than from inference. This is the only one I could find.
Here’s why this is difficult: We really expect you to be you and we’ve setup some infrastructure around that and the assumption that OOD is not contained at all and indeed on real OS. That is, you to have a UID/GID(s) of a normal regular non-privileged user. You may see Per User NGINX (PUN) in the documentation and we utilize NGINX’s feature to fork a process and set it’s effective UID & GID.
Here’s the ps -elf output I just pulled from our test system. This is my (user johrstrom’s) PUN process actually are. A couple take aways are the master process is being ran by root. Real root because it’s a real OS. Because it’s real root, it can make those system calls to set effective UID & GID to me (uid=30961(johrstrom) gid=5515(PZS0714)).
The other take away is that some process’ parent Id is 1, systemd. Systemd doesn’t like being in a container, but there are ways around that. The important bit really is that systemd is doing the heavy lifting in terms of forking processes’ and it is very good at that on real systems. I always podman for unprivileged users - plus those containers don’t interact with external systems (they’re really only for development) so I don’t know how good that forking process is without systemd.
4 S johrstr+ 54856 1 0 80 0 - 100183 poll_s 13:11 ? 00:00:00 Passenger watchdog
0 S johrstr+ 54859 54856 0 80 0 - 378357 poll_s 13:11 ? 00:00:00 Passenger core
5 S root 54878 1 0 80 0 - 30614 sigsus 13:11 ? 00:00:00 nginx: master process (johrstrom) -c /var/lib/ondemand-nginx/config/puns/johrstrom.conf
5 S johrstr+ 54879 54878 0 80 0 - 33309 ep_pol 13:11 ? 00:00:00 nginx: worker process
So that’s the big hurdle with having everything bundled in 1 container - Having all those real LDAP users like johrstrom in a container. What’s more is, you’ll likely have to boot the container as real root so that it can continue to boot PUNs with the right UIDs. And that gives me a a bit of pause - because the containers have to be privileged in this way.
Conversly - you may be wondering, well why not boot all the PUNs in their own kuberentes PODs and have Apache route to that k8s Pod instead of a local PUN. We currently do this routing though a simple file check. I’m the REMOTE_USER named johrstrom (we got that from our Open Id Connect provider) and apache is using a simple file structure scheme to find the unix socket (a local file) that my PUN is listening to, namely /var/run/ondemand-nginx/johrstrom/passenger.sock. So if you were to try to break up these components (Apache and Nginx) you’d have to solve how to route requests.
Easiest way is to deploy is on a VM. Sorry if that’ not what you’re looking for. It’s probably not impossible to fully containerize Open OnDemand, you’d just be forging that path where we haven’t yet.
Hope that helps!