(I asked this question at SC and was diverted here)
We have a static service (cryoSPARC) running within our cluster on a host named db1, port 50000. One can access this port either via port forwarding / socks proxy, but we would like to know if there is a way we can use ondemand to streamline the process for our users.
Currently, our ondemand instance is configured to submit jobs to our cluster. To use cryoSPARC, we have added an interactive application which spawns a linux desktop environment. This then launches a firefox browser inside an Apptainer container pointing to db1:50000.
The current setup works, but we were hoping that there was a better approach? Our initial thoughts were to configure db1 as a “compute” host and submit directly to that host, or have a web frontend spawn on the compute host when the cryoSPARC application is launched, and proxy all traffic from :50000 to db1:50000.
When we connect to other services using /rnode/host/port this works nicely when a service is running on a specific port on the compute host, but /rnode/db1/50000 won’t work here, as the service is running on db1, not the compute host running the job.
I don’t think we can support proxying back to cyrosparc directly yet. Which is to say - what you have may be the best bet.
To correctly use /rnode/host/port to connect to Cryosparc - we’d need this feature below implemented and having just checked it out, it would require an upgrade to apache/httpd (2.5) that lots of operating systems don’t even have support for yet.
As the Engineer for that case study, I can answer directly. The mod_substitute approach is the best Jeff and I have come up with as a potential avenue as the Cryosparc devs have told both us and many other folks they will not fix this limitation. Which in fairness would probably be an overhaul of a ton of the structure of the app…some very unfortunate technical debt effectively.
For that case study, I just have Cryosparc running as normal and I just redirect back to that port directly with a dumb simple flask app.
I do effectively the same as you for other clients HPC systems which is start Cryosparc up on the compute node via an Interactive Desktop app where I then also spawn Firefox in kiosk mode directly to localhost:port that Cryosparc is listening on. For reference, to make sure I do not get port clashes, my installation script for Cryosparc (I install it for each user uniquely so I am license compliant) does some math based off the users UID to find a unique port range to give the Cryosparc installer. I then use the same math when the server is starting up on the compute node so it finds the on the fly started up Cryosparc.
To get back to mod_substitute, as Jeff mentions, it appears features that are needed are not supported in apache 2.4 and instead will need to use apache 2.5. As far as the OOD dev side of things are concerned, this is a pretty hard stop until 2.5 gets more wide adoption across distro’s. I do plan on spending some time trying to get it working with 2.5 and publishing my results but time is of course always limiting.
I think for the time being, we will keep our current setup and sit tight for when apache 2.5 (and mod_substitute) become supported across the main distros.
