We have one user who is suddenly unable to access Jupyter Lab through OOD. They get a 500 internal server error after they click the “Connect to Jupyter” button. We are out of ideas as to what could be causing the issue. I had them clear their browser cache. The Jupyter “cache” files in their home directory have been deleted. We’ve removed and re-added their account to our LDAP server. We’re just out of ideas at this point. Below is their output.log file. The “Jupyter Server Terminals requires Jupyter Server 2.0+” is a red herring as my other system admin sees that same message when he launches Jupyter Lab and it works fine for him. Any assistance would be greatly appreciated. Thanks!
Script starting…
Waiting for Jupyter Notebook server to open port 6896…
TIMING - Starting wait at: Thu Nov 20 02:59:47 PM CST 2025
TIMING - Starting main script at: Thu Nov 20 02:59:47 PM CST 2025
Currently Loaded Modules:
- slurm/slurm/24.05.7 3) jupyter/15.5.1 5) shared
- gcc/13.1.0 4) python39
TIMING - Starting jupyter at: Thu Nov 20 02:59:47 PM CST 2025
Python 3.9.21
3.6.8
- jupyter-lab --config=/home//ondemand/data/sys/dashboard/batch_connect/sys/jupyter-lab/output/3d8654fb-3e39-4dcf-b751-b226ce6b5b00/config.py
[W 2025-11-20 14:59:50.304 ServerApp] jupyter_server_terminals | error adding extension (enabled: True): Jupyter Server Terminals requires Jupyter Server 2.0+
Traceback (most recent call last):
File “/cm/shared/apps/jupyter/15.5.1/lib/python3.9/site-packages/jupyter_server/extension/manager.py”, line 320, in add_extension
extpkg = ExtensionPackage(name=extension_name, enabled=enabled)
File “/cm/shared/apps/jupyter/15.5.1/lib/python3.9/site-packages/jupyter_server/extension/manager.py”, line 166, in init
super().init(*args, **kwargs)
File “/cm/shared/apps/jupyter/15.5.1/lib/python3.9/site-packages/traitlets/traitlets.py”, line 1369, in init
value = self._traits[key]._cross_validate(self, getattr(self, key))
File “/cm/shared/apps/jupyter/15.5.1/lib/python3.9/site-packages/traitlets/traitlets.py”, line 730, in _cross_validate
value = obj._trait_validators[self.name](obj, proposal)
File “/cm/shared/apps/jupyter/15.5.1/lib/python3.9/site-packages/traitlets/traitlets.py”, line 1241, in call
return self.func(*args, **kwargs)
File “/cm/shared/apps/jupyter/15.5.1/lib/python3.9/site-packages/jupyter_server/extension/manager.py”, line 175, in _validate_name
self._module, self._metadata = get_metadata(name)
File “/cm/shared/apps/jupyter/15.5.1/lib/python3.9/site-packages/jupyter_server/extension/utils.py”, line 60, in get_metadata
module = importlib.import_module(package_name)
File “/cm/local/apps/python39/lib/python3.9/importlib/init.py”, line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File “”, line 1030, in _gcd_import
File “”, line 1007, in _find_and_load
File “”, line 986, in _find_and_load_unlocked
File “”, line 680, in _load_unlocked
File “”, line 850, in exec_module
File “”, line 228, in _call_with_frames_removed
File “/home//.local/lib/python3.9/site-packages/jupyter_server_terminals/init.py”, line 13, in
raise RuntimeError(msg)
RuntimeError: Jupyter Server Terminals requires Jupyter Server 2.0+
[I 2025-11-20 14:59:50.370 ServerApp] jupyter_lsp | extension was successfully linked.
[I 2025-11-20 14:59:50.375 ServerApp] jupyter_server_fileid | extension was successfully linked.
[I 2025-11-20 14:59:50.379 ServerApp] jupyter_server_ydoc | extension was successfully linked.
[W 2025-11-20 14:59:50.380 LabApp] ‘ip’ has moved from NotebookApp to ServerApp. This config will be passed to ServerApp. Be sure to update your config before our next release.
[W 2025-11-20 14:59:50.380 LabApp] ‘port’ has moved from NotebookApp to ServerApp. This config will be passed to ServerApp. Be sure to update your config before our next release.
[W 2025-11-20 14:59:50.380 LabApp] ‘port_retries’ has moved from NotebookApp to ServerApp. This config will be passed to ServerApp. Be sure to update your config before our next release.
[W 2025-11-20 14:59:50.380 LabApp] ‘password’ has moved from NotebookApp to ServerApp. This config will be passed to ServerApp. Be sure to update your config before our next release.
[W 2025-11-20 14:59:50.380 LabApp] ‘base_url’ has moved from NotebookApp to ServerApp. This config will be passed to ServerApp. Be sure to update your config before our next release.
[W 2025-11-20 14:59:50.382 LabApp] ‘base_url’ has moved from NotebookApp to ServerApp. This config will be passed to ServerApp. Be sure to update your config before our next release.
[W 2025-11-20 14:59:50.382 LabApp] ‘allow_origin’ has moved from NotebookApp to ServerApp. This config will be passed to ServerApp. Be sure to update your config before our next release.
[W 2025-11-20 14:59:50.382 LabApp] ‘notebook_dir’ has moved from NotebookApp to ServerApp. This config will be passed to ServerApp. Be sure to update your config before our next release.
[W 2025-11-20 14:59:50.382 LabApp] ‘disable_check_xsrf’ has moved from NotebookApp to ServerApp. This config will be passed to ServerApp. Be sure to update your config before our next release.
[W 2025-11-20 14:59:50.385 ServerApp] notebook_dir is deprecated, use root_dir
[I 2025-11-20 14:59:50.386 ServerApp] jupyterlab | extension was successfully linked.
[I 2025-11-20 14:59:50.389 ServerApp] nbclassic | extension was successfully linked.
[I 2025-11-20 14:59:50.840 ServerApp] cm_jupyter_kernel_creator | extension was found and enabled by notebook_shim. Consider moving the extension to Jupyter Server’s extension paths.
[I 2025-11-20 14:59:50.840 ServerApp] cm_jupyter_kernel_creator | extension was successfully linked.
[I 2025-11-20 14:59:51.404 ServerApp] cm_jupyter_addons | extension was found and enabled by notebook_shim. Consider moving the extension to Jupyter Server’s extension paths.
[I 2025-11-20 14:59:51.405 ServerApp] cm_jupyter_addons | extension was successfully linked.
[I 2025-11-20 14:59:51.422 ServerApp] cm_jupyter_vnc | extension was found and enabled by notebook_shim. Consider moving the extension to Jupyter Server’s extension paths.
[I 2025-11-20 14:59:51.422 ServerApp] cm_jupyter_vnc | extension was successfully linked.
[I 2025-11-20 14:59:52.765 ServerApp] cm_jupyter_kubernetes_operators_manager | extension was found and enabled by notebook_shim. Consider moving the extension to Jupyter Server’s extension paths.
[I 2025-11-20 14:59:52.765 ServerApp] cm_jupyter_kubernetes_operators_manager | extension was successfully linked.
[I 2025-11-20 14:59:52.765 ServerApp] notebook_shim | extension was successfully linked.
[W 2025-11-20 14:59:52.827 ServerApp] WARNING: The Jupyter server is listening on all IP addresses and not using encryption. This is not recommended.
[I 2025-11-20 14:59:52.829 ServerApp] notebook_shim | extension was successfully loaded.
[I 2025-11-20 14:59:52.830 ServerApp] cm_jupyter_addons | extension was successfully loaded.
[I 2025-11-20 14:59:52.831 ServerApp] cm_jupyter_kernel_creator | extension was successfully loaded.
[I 2025-11-20 14:59:52.838 ServerApp] cm_jupyter_kubernetes_operators_manager | extension was successfully loaded.
[I 2025-11-20 14:59:52.839 ServerApp] cm_jupyter_vnc | extension was successfully loaded.
[I 2025-11-20 14:59:52.841 ServerApp] jupyter_lsp | extension was successfully loaded.
[I 2025-11-20 14:59:52.841 FileIdExtension] Configured File ID manager: ArbitraryFileIdManager
[I 2025-11-20 14:59:52.841 FileIdExtension] ArbitraryFileIdManager : Configured root dir: /home/
[I 2025-11-20 14:59:52.841 FileIdExtension] ArbitraryFileIdManager : Configured database path: /home//.local/share/jupyter/file_id_manager.db
[I 2025-11-20 14:59:52.855 FileIdExtension] ArbitraryFileIdManager : Successfully connected to database file.
[I 2025-11-20 14:59:52.855 FileIdExtension] ArbitraryFileIdManager : Creating File ID tables and indices with journal_mode = DELETE
[I 2025-11-20 14:59:52.856 ServerApp] jupyter_server_fileid | extension was successfully loaded.
[I 2025-11-20 14:59:52.856 ServerApp] jupyter_server_ydoc | extension was successfully loaded.
[I 2025-11-20 14:59:52.872 LabApp] JupyterLab extension loaded from /cm/shared/apps/jupyter/15.5.1/lib/python3.9/site-packages/jupyterlab
[I 2025-11-20 14:59:52.872 LabApp] JupyterLab application directory is /cm/shared/apps/jupyter/15.5.1/share/jupyter/lab
[I 2025-11-20 14:59:52.877 ServerApp] jupyterlab | extension was successfully loaded.
[I 2025-11-20 14:59:52.884 ServerApp] nbclassic | extension was successfully loaded.
[I 2025-11-20 14:59:52.885 ServerApp] Serving notebooks from local directory: /home/
[I 2025-11-20 14:59:52.885 ServerApp] Jupyter Server 1.24.0 is running at:
[I 2025-11-20 14:59:52.885 ServerApp] http://localhost:6896/node/talon06.ib.cluster/6896/lab
[I 2025-11-20 14:59:52.885 ServerApp] or http://127.0.0.1:6896/node/talon06.ib.cluster/6896/lab
[I 2025-11-20 14:59:52.885 ServerApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
[W 2025-11-20 14:59:52.906 ServerApp] No web browser found: could not locate runnable browser.
Discovered Jupyter Notebook server listening on port 6896!
TIMING - Wait ended at: Thu Nov 20 02:59:53 PM CST 2025
[I 2025-11-20 14:59:53.714 ServerApp] Skipped non-installed server(s): bash-language-server, dockerfile-language-server-nodejs, javascript-typescript-langserver, jedi-language-server, julia-language-server, pyright, python-language-server, python-lsp-server, r-languageserver, sql-language-server, texlab, typescript-language-server, unified-language-server, vscode-css-languageserver-bin, vscode-html-languageserver-bin, vscode-json-languageserver-bin, yaml-language-server
Generating connection YAML file…
[I 2025-11-20 14:59:56.685 ServerApp] 302 POST /node/talon06.ib.cluster/6896/login (172.17.4.241) 1.82ms
[I 2025-11-20 14:59:56.697 ServerApp] 302 GET /node/talon06.ib.cluster/6896/ (172.17.4.241) 0.49ms
[E 2025-11-20 14:59:56.719 ServerApp] Uncaught exception GET /node/talon06.ib.cluster/6896/lab? (172.17.4.241)
HTTPServerRequest(protocol=‘http’, host=‘apps.talon.und.edu’, method=‘GET’, uri=‘/node/talon06.ib.cluster/6896/lab?’, version=‘HTTP/1.1’, remote_ip=‘172.17.4.241’)
Traceback (most recent call last):
File “/cm/shared/apps/jupyter/15.5.1/lib/python3.9/site-packages/tornado/web.py”, line 1788, in _execute
result = method(*self.path_args, **self.path_kwargs)
File “/cm/shared/apps/jupyter/15.5.1/lib/python3.9/site-packages/tornado/web.py”, line 3301, in wrapper
return method(self, args, **kwargs)
File “/cm/shared/apps/jupyter/15.5.1/lib/python3.9/site-packages/tornado/web.py”, line 1967, in wrapper
return method(self, args, **kwargs)
File “/cm/shared/apps/jupyter/15.5.1/lib/python3.9/site-packages/jupyterlab_server/handlers.py”, line 162, in get
page_config = self.get_page_config()
File “/cm/shared/apps/jupyter/15.5.1/lib/python3.9/site-packages/jupyterlab_server/handlers.py”, line 143, in get_page_config
page_config, get_page_config(labextensions_path, settings_dir, logger=self.log)
File “/cm/shared/apps/jupyter/15.5.1/lib/python3.9/site-packages/jupyterlab_server/config.py”, line 148, in get_page_config
federated_exts = get_federated_extensions(labextensions_path)
File “/cm/shared/apps/jupyter/15.5.1/lib/python3.9/site-packages/jupyterlab_server/config.py”, line 50, in get_federated_extensions
pkgdata = json.load(fid)
File “/cm/local/apps/python39/lib/python3.9/json/init.py”, line 293, in load
return loads(fp.read(),
File “/cm/local/apps/python39/lib/python3.9/json/init.py”, line 346, in loads
return _default_decoder.decode(s)
File “/cm/local/apps/python39/lib/python3.9/json/decoder.py”, line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File “/cm/local/apps/python39/lib/python3.9/json/decoder.py”, line 355, in raw_decode
raise JSONDecodeError(“Expecting value”, s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
[E 2025-11-20 14:59:56.736 ServerApp] Could not open static file ‘’
[E 2025-11-20 14:59:56.736 LabApp] {
“Host”: “apps.talon.und.edu”,
“Accept”: "text/html,application/xhtml+xml,application/xml;q=0.9,/;q=0.8",
“Referer”: “https://apps.talon.und.edu/pun/sys/dashboard/batch_connect/sessions”,
“User-Agent”: “Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:145.0) Gecko/20100101 Firefox/145.0”
}
[E 2025-11-20 14:59:56.737 LabApp] 500 GET /node/talon06.ib.cluster/6896/lab? (172.17.4.241) 33.62ms referer=https://apps.talon.und.edu/pun/sys/dashboard/batch_connect/sessions
[W 2025-11-20 14:59:56.765 ServerApp] 404 GET /node/talon06.ib.cluster/6896/static/lab/style/bootstrap.min.css (172.17.4.241) 5.40ms referer=https://apps.talon.und.edu/node/talon06.ib.cluster/6896/lab?
[W 2025-11-20 14:59:56.769 ServerApp] 403 GET /node/talon06.ib.cluster/6896/static/lab/style/bootstrap-theme.min.css (172.17.4.241): style/bootstrap-theme.min.css is not in root static directory
[W 2025-11-20 14:59:56.770 ServerApp] 403 GET /node/talon06.ib.cluster/6896/static/lab/style/bootstrap-theme.min.css (172.17.4.241) 1.69ms referer=https://apps.talon.und.edu/node/talon06.ib.cluster/6896/lab?
[W 2025-11-20 14:59:56.770 ServerApp] 403 GET /node/talon06.ib.cluster/6896/static/lab/style/index.css (172.17.4.241): style/index.css is not in root static directory
[W 2025-11-20 14:59:56.771 ServerApp] 403 GET /node/talon06.ib.cluster/6896/static/lab/style/index.css (172.17.4.241) 2.20ms referer=https://apps.talon.und.edu/node/talon06.ib.cluster/6896/lab?
[W 2025-11-20 14:59:56.774 ServerApp] 403 GET /node/talon06.ib.cluster/6896/static/lab/logo/logo.png (172.17.4.241): logo/logo.png is not in root static directory
[W 2025-11-20 14:59:56.774 ServerApp] 403 GET /node/talon06.ib.cluster/6896/static/lab/logo/logo.png (172.17.4.241) 0.99ms referer=https://apps.talon.und.edu/node/talon06.ib.cluster/6896/lab?
[W 2025-11-20 14:59:56.781 ServerApp] 403 GET /node/talon06.ib.cluster/6896/static/lab/favicon.ico (172.17.4.241): favicon.ico is not in root static directory
[W 2025-11-20 14:59:56.782 ServerApp] 403 GET /node/talon06.ib.cluster/6896/static/lab/favicon.ico (172.17.4.241) 1.00ms referer=https://apps.talon.und.edu/node/talon06.ib.cluster/6896/lab?
[E 2025-11-20 15:00:52.835 ServerApp] No CMD cookie found
[E 2025-11-20 15:00:52.854 ServerApp] Error occured accessing CMDaemon:
[Errno 111] Connection refused
[E 2025-11-20 15:01:52.850 ServerApp] Error occured accessing CMDaemon:
[Errno 111] Connection refused
[E 2025-11-20 15:02:52.850 ServerApp] Error occured accessing CMDaemon:
[Errno 111] Connection refused
[E 2025-11-20 15:03:52.850 ServerApp] Error occured accessing CMDaemon:
[Errno 111] Connection refused
[E 2025-11-20 15:04:52.851 ServerApp] Error occured accessing CMDaemon:
[Errno 111] Connection refused
[E 2025-11-20 15:05:52.850 ServerApp] Error occured accessing CMDaemon:
[Errno 111] Connection refused
[E 2025-11-20 15:06:52.850 ServerApp] Error occured accessing CMDaemon:
[Errno 111] Connection refused
[E 2025-11-20 15:07:52.851 ServerApp] Error occured accessing CMDaemon:
[Errno 111] Connection refused
[E 2025-11-20 15:08:52.850 ServerApp] Error occured accessing CMDaemon:
[Errno 111] Connection refused