Thanks to yesterday call, I was finally able to simplified user connection to Tensorflow and Tensorboard on the same app launch. Users still need to launch Tensorboard internally on Jupyter’s builtin terminal, but the Tensorboard Connect acts as a placeholder to serve.
Tensorboard is protected by adapting the security from Utah’ RShiny app (https://github.com/CHPC-UofU/bc_osc_example_shiny). An NGINX server is setup to protect, and tensorboard is launched with the default options (localhost/6006), which are then hardcoded into the nginx.conf file. I have tested and confirmed that external access to the compute node on port 6006 fails. There can still potentially be issues if two people are placed on the same node, but this can be avoided if the tensorflow/tensorboard user asks for one entire node.
I was able to demonstrate how to get the working If your users have network namespaces enabled on the kernels.
Here I setup a proxy that does the authentication (credit to Stanford on that IIRC), it’s external. But it then connects to the internal namespace to tensorflow.
We’ve actually enabled network namespaces for all users on our clusters - we just haven’t taken full advantage of them yet.