Blank page when connecting to a Jupyter interactive session over HTTPS

Get Help
1

Dear OOD community,

we’re working on our installation of Open Ondemand, using the Ansible role provided by OSC ( GitHub - OSC/ood-ansible: An ansible role for Open Ondemand ). In a first step, we tested everything using HTTP, and our interactive sessions and dashboard worked perfectly. Now we’re moving to HTTPS: the dashboard works as well as some interactive apps (e.g. VSCode), and we’re able to login and run jobs but we’re having issues with some other applications; in this thread, we’d like to ask your help for the Jupyter app.

The Jupyter interactive sessions appear to run without errors but when we connect to them we get a blank page. This seems a similar behavior than the one described in the issue Blank screen, weird errors with Jupyter - #4 by joshfactorial .

We are using the following before.sh.erb, which is a slightly modified version of the file from the example in GitHub - OSC/bc_example_jupyter: Batch Connect - Example Jupyter Notebook Server . The script.sh.erb we are using uses the command jupyter lab instead of the default jupyter notebook. Rolling back to the original version of the files did not solve the problem.

# before.sh.erb

# Export the module function if it exists
[[ $(type -t module) == "function" ]] && export -f module

# Find available port to run server on
port=$(find_port)

# Generate SHA1 encrypted password (requires OpenSSL installed)
password="$(create_passwd 16)"

# The `$CONFIG_FILE` environment variable is exported as it is used in the main
# `script.sh.erb` file when launching the Jupyter Notebook server.
export CONFIG_FILE="${PWD}/config.py"

# Generate Jupyter configuration file with secure file permissions
(
umask 077
cat > "${CONFIG_FILE}" << EOL
c.ServerApp.ip = '*'
c.ServerApp.port = ${port}
c.ServerApp.port_retries = 0
c.IdentityProvider.token = '${password}'
c.ServerApp.base_url = '/node/${host}/${port}/'
c.ServerApp.open_browser = False
c.ServerApp.allow_origin = '*'
c.ServerApp.root_dir = '${HOME}'
c.ServerApp.disable_check_xsrf = True
c.ServerApp.log_level = "DEBUG"
EOL
)

This is the log in the output.log:

The system default contains no modules
  (env var: LMOD_SYSTEM_DEFAULT_MODULES is empty)
    No changes in loaded modules

    Script starting...
    Waiting for Jupyter Notebook server to open port 49754...
    TIMING - Starting wait at: Fri Sep 26 02:37:29 PM CEST 2025
    TIMING - Starting main script at: Fri Sep 26 02:37:29 PM CEST 2025
    No modules loaded
    TIMING - Starting jupyter at: Fri Sep 26 02:37:29 PM CEST 2025
    + jupyter lab --config=/home/<my username>/ondemand/data/sys/dashboard/batch_connect/sys/jupyter/openhpc/output/bad2a958-5436-4e85-bc9b-849e68cdab11/config.py
    [I 2025-09-26 14:37:31.779 ServerApp] jupyter_lsp | extension was successfully linked.
    [D 2025-09-26 14:37:31.783 ServerApp] Config changed: {'ServerApp': {'config_file': '/home/<my username>/ondemand/data/sys/dashboard/batch_connect/sys/jupyter/openhpc/output/bad2a958-5436-4e85-bc9b-849e68cdab11/config.py', 'ip': '*', 'port': 49754, 'port_retries': 0, 'base_url': '/node/ondemand-poc-node-00/49754/', 'open_browser': False, 'allow_origin': '*', 'root_dir': '/home/<my username>', 'disable_check_xsrf': True, 'log_level': 'DEBUG', 'jpserver_extensions': <LazyConfigValue value={'jupyterlab': True, 'jupyter_lsp': True, 'jupyter_server_terminals': True, 'notebook': True, 'notebook_shim': True}>}, 'IdentityProvider': {'token': 'ackaqi8Ys78471xa'}}
    [I 2025-09-26 14:37:31.783 ServerApp] jupyter_server_terminals | extension was successfully linked.
    [I 2025-09-26 14:37:31.787 ServerApp] jupyterlab | extension was successfully linked.
    [I 2025-09-26 14:37:31.791 ServerApp] notebook | extension was successfully linked.
    [I 2025-09-26 14:37:32.025 ServerApp] notebook_shim | extension was successfully linked.
    [D 2025-09-26 14:37:32.026 ServerApp] Config changed: {'IdentityProvider': {'token': 'ackaqi8Ys78471xa'}, 'NotebookApp': {}, 'ServerApp': {'config_file': '/home/<my username>/ondemand/data/sys/dashboard/batch_connect/sys/jupyter/openhpc/output/bad2a958-5436-4e85-bc9b-849e68cdab11/config.py', 'ip': '*', 'port': 49754, 'port_retries': 0, 'base_url': '/node/ondemand-poc-node-00/49754/', 'open_browser': False, 'allow_origin': '*', 'root_dir': '/home/<my username>', 'disable_check_xsrf': True, 'log_level': 'DEBUG', 'jpserver_extensions': <LazyConfigValue value={'jupyterlab': True, 'jupyter_lsp': True, 'jupyter_server_terminals': True, 'notebook': True, 'notebook_shim': True}>}}
    [D 2025-09-26 14:37:32.027 ServerApp] Raising open file limit: soft 1024->4096; hard 131072->131072
    [W 2025-09-26 14:37:32.056 ServerApp] WARNING: The Jupyter server is listening on all IP addresses and not using encryption. This is not recommended.
    [I 2025-09-26 14:37:32.057 ServerApp] notebook_shim | extension was successfully loaded.
    [D 2025-09-26 14:37:32.058 ServerApp] [lsp] rootUri will be file:///home/<my username>
    [D 2025-09-26 14:37:32.059 ServerApp] [lsp] virtualDocumentsUri will be file:///home/<my username>/.virtual_documents
    [I 2025-09-26 14:37:32.059 ServerApp] jupyter_lsp | extension was successfully loaded.
    [I 2025-09-26 14:37:32.060 ServerApp] jupyter_server_terminals | extension was successfully loaded.
    [I 2025-09-26 14:37:32.062 LabApp] JupyterLab extension loaded from /usr/local/lib/python3.9/site-packages/jupyterlab
    [I 2025-09-26 14:37:32.062 LabApp] JupyterLab application directory is /usr/local/share/jupyter/lab
    [I 2025-09-26 14:37:32.062 LabApp] Extension Manager is 'pypi'.
    [I 2025-09-26 14:37:32.092 ServerApp] jupyterlab | extension was successfully loaded.
    [I 2025-09-26 14:37:32.095 ServerApp] notebook | extension was successfully loaded.
    [I 2025-09-26 14:37:32.096 ServerApp] Serving notebooks from local directory: /home/<my username>
    [I 2025-09-26 14:37:32.096 ServerApp] Jupyter Server 2.17.0 is running at:
    [I 2025-09-26 14:37:32.096 ServerApp] http://localhost:49754/node/ondemand-poc-node-00/49754/lab?token=...
    [I 2025-09-26 14:37:32.096 ServerApp]     http://127.0.0.1:49754/node/ondemand-poc-node-00/49754/lab?token=...
    [I 2025-09-26 14:37:32.096 ServerApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
    [D 2025-09-26 14:37:32.126 ServerApp] notebook_shim | extension was successfully started.
    [D 2025-09-26 14:37:32.126 ServerApp] jupyter_lsp | extension was successfully started.
    [D 2025-09-26 14:37:32.126 ServerApp] jupyter_server_terminals | extension was successfully started.
    [D 2025-09-26 14:37:32.126 ServerApp] jupyterlab | extension was successfully started.
    [D 2025-09-26 14:37:32.126 ServerApp] notebook | extension was successfully started.

                                        ...........
                               < Plenty of checks we omit>
                                        ...........

    [D 2025-09-26 14:37:32.185 ServerApp] [lsp] None of the installed servers require virtual documents disabling shadow filesystem.
    [D 2025-09-26 14:37:32.185 ServerApp] [lsp] The following Language Servers will be available: {}
    Discovered Jupyter Notebook server listening on port 49754!
    TIMING - Wait ended at: Fri Sep 26 02:37:32 PM CEST 2025
    Generating connection YAML file...
    [D 2025-09-26 14:37:48.458 ServerApp] Generating new user for token-authenticated request: 4c96c22248e64708888d3131d8eed403
    [I 2025-09-26 14:37:48.461 ServerApp] User 4c96c22248e64708888d3131d8eed403 logged in.
    [I 2025-09-26 14:37:48.463 ServerApp] 302 POST /node/ondemand-poc-node-00/49754/login (4c96c22248e64708888d3131d8eed403@192.168.1.200) 6.49ms
    [I 2025-09-26 14:37:48.475 ServerApp] 302 GET /node/ondemand-poc-node-00/49754/ (@192.168.1.200) 0.83ms
    [D 2025-09-26 14:37:48.532 ServerApp] Path main.d65d083251e934338191.js served from /usr/local/share/jupyter/lab/static/main.d65d083251e934338191.js
    [D 2025-09-26 14:37:48.533 ServerApp] 200 GET /node/ondemand-poc-node-00/49754/static/lab/main.d65d083251e934338191.js?v=d65d083251e934338191 (4c96c22248e64708888d3131d8eed403@192.168.1.200) 2.00ms
    [D 2025-09-26 14:37:48.572 ServerApp] Path jupyterlab_pygments/static/remoteEntry.5cbb9d2323598fbda535.js served from /usr/local/share/jupyter/labextensions/jupyterlab_pygments/static/remoteEntry.5cbb9d2323598fbda535.js
    [D 2025-09-26 14:37:48.573 ServerApp] 200 GET /node/ondemand-poc-node-00/49754/lab/extensions/jupyterlab_pygments/static/remoteEntry.5cbb9d2323598fbda535.js (4c96c22248e64708888d3131d8eed403@192.168.1.200) 2.58ms
    [D 2025-09-26 14:37:48.579 ServerApp] Path @jupyter-notebook/lab-extension/static/remoteEntry.7e3d7768845089ad5f5f.js served from /usr/local/share/jupyter/labextensions/@jupyter-notebook/lab-extension/static/remoteEntry.7e3d7768845089ad5f5f.js
    [D 2025-09-26 14:37:48.579 ServerApp] 200 GET /node/ondemand-poc-node-00/49754/lab/extensions/@jupyter-notebook/lab-extension/static/remoteEntry.7e3d7768845089ad5f5f.js (4c96c22248e64708888d3131d8eed403@192.168.1.200) 2.19ms
    [D 2025-09-26 14:37:48.593 ServerApp] Path favicons/favicon.ico served from /usr/local/lib/python3.9/site-packages/jupyter_server/static/favicons/favicon.ico
    [D 2025-09-26 14:37:48.594 ServerApp] 200 GET /node/ondemand-poc-node-00/49754/static/favicons/favicon.ico (4c96c22248e64708888d3131d8eed403@192.168.1.200) 2.07ms
    [D 2025-09-26 14:37:48.596 ServerApp] Path jlab_core.8aca991c1d97720da49a.js served from /usr/local/share/jupyter/lab/static/jlab_core.8aca991c1d97720da49a.js
    [D 2025-09-26 14:37:48.610 ServerApp] 200 GET /node/ondemand-poc-node-00/49754/static/lab/jlab_core.8aca991c1d97720da49a.js?v=8aca991c1d97720da49a (4c96c22248e64708888d3131d8eed403@192.168.1.200) 15.31ms

Our view.html.erb is the following:

<form action="/node/<%= host %>/<%= port %>/login" method="post" target="_blank">
  <input type="hidden" name="password" value="<%= password %>">
  <button class="btn btn-primary" type="submit">
    <i class="fa fa-eye"></i> Connect to Jupyter
  </button>
</form>

We found the following errors in the Apache logs:

[Fri Sep 26 14:37:48.907938 2025] [proxy:error] [pid 1914:tid 2087] (111)Connection refused: AH00957: http: attempt to connect to 192.168.1.50:80 (ondemand-poc-node-00:80) failed
[Fri Sep 26 14:37:48.907954 2025] [proxy_http:error] [pid 1914:tid 2087] [client <my ip>:40561] AH01114: HTTP: failed to make connection to backend: ondemand-poc-node-00, referer: https://<ood domain name>/node/ondemand-poc-node-00/49754/lab?

We wonder why there is an attempt to connect to the port 80 in the compute node.

Thanks a lot for your support.

2

Hello, welcome, and thank you for the detailed post!

I’m not sure looking at this why 80 is being used, but I have a few questions.

1. When you moved to HTTPS and after getting all of that setup, did you adjust your ood_portal.yml and restart httpd?

2. Can you share your regex for the host_regex being used to reverse proxy?

3. Could you share the script.sh.erb itself so I can see the jupyter lab command issued?

4. Lastly, can you confirm that the enable_reverse_proxy: true and node_uri: ‘/node’ are set?

My guess is something with the reverse proxy regex is off and this then fails somehow and defaults to port 80, but I’m not totally sure why that is happening without learning a bit more.

3

Hi Travis,

Thank you for your answer! I reply to your questions inline.

Yes, we did adjust our ood_portal.yml and restarted Apache service.

 host_regex: 'ondemand-poc-node-.*'

# script.sh.erb

#!/usr/bin/env bash

# Benchmark info
echo "TIMING - Starting main script at: $(date)"

# Set working directory to home directory
cd "${HOME}"

#
# Start Jupyter Notebook Server
#

<%- unless context.modules.blank? -%> 
# Purge the module environment to avoid conflicts
module purge

# Load the require modules
#module load <%= context.modules %>

# List loaded modules
module list
<%- end -%> 

# Benchmark info
echo "TIMING - Starting jupyter at: $(date)"

# Launch the Jupyter Notebook Server
set -x
jupyter lab --config="${CONFIG_FILE}"

We actually hadn’t included the variable enable_reverse_proxy in our initial portal configuration. Could you please tell us if it is a new feature? We have not found it in the documentation. Now, we have added it manually on ood_portal.yml and we have run update_ood_portal but it didn’t change the Apache configuration.

Regarding the node_uri, we confirm we are using the following:

node_uri: '/node'
rnode_uri: '/rnode'

Let us know if you need additional information.
Thanks once again!

4

Hi!

Did you have any chance to have a look at this issue?

Thanks!

5

Sorry for the delay.

I’ve been trying to recreate the issue on my end but have not had any luck so far.

The enable_reverse_proxy was a wrong setting, sorry about that.

I’ve been trying to play with some of the Jupyter configs itself to recreate the problem but nothing so far. The original referrer is using https so I’m trying to just unerstand why that then falls back to http on port 80 in the end and this seems like something is wrong in Apache and not Jupyter itself.

I’d recommend using the Reverse proxy page to confirm first that you can make this conenction to the compute cluster Jupyter is sitting on using this page: 3. Enable Reverse Proxy — Open OnDemand 4.0.0 documentation

I think verifying the regex for connecting back to the compute cluster can at least eliminate any apache default behavior causing this http request from some kind of failed rewrite due to a bad regex.

6

Hi Travis,

Thank you so much for your answer.

We have followed the instructions in the link you mentioned and have not found any further hint on what the issue could be.

We will try to inspect the network traffic more in detail and we will come back to you as soon as possible.

7

Hi Travis,

We resolved the issue by changing the host_regex expression from ondemand-poc-node-.* to ondemand-poc-node-.*?. Adding the “?” to the end of the regex turned out to be the key.

We got the idea from this issue by @link89 and the pull request therein.

The change did resolve issues with jupyter and RStudio that looked very different from each other.

We believe this might be worth adding to the documentation for other users who encounter the same problem.

1 Like