My prod ood instance seems to have lost its mind; anything thoughts that would help with debugging would be appreciated. When (after restarting my pun instance) I click on Jobs->Active Jobs I get a blank page. My dev instance, of course, does not show this problem.
The error log for my prod instance shows
App 173191 output: [2026-09-09 06:38:38 -0700 ] INFO "method=GET path=/pun/sys/dashboard/apps/show/activejobs format=html controller=AppsController action=show status=302 allocations=6033 duration=8.23 view=0.00 location=``https://ood.hpc.arizona.edu/pun/sys/dashboard/activejobs``"
App 173191 output: [2026-09-09 06:38:38 -0700 ] DEBUG "Rendering layout layouts/active_jobs.html.erb"
App 173191 output: [2026-09-09 06:38:38 -0700 ] DEBUG "Rendering active_jobs/index.html.erb within layouts/active_jobs"
App 173191 output: [2026-09-09 06:38:38 -0700 ] DEBUG "Rendering layouts/application.html.erb"
App 173191 output: [2026-09-09 06:38:38 -0700 ] INFO "method=GET path=/pun/sys/dashboard/activejobs format=html controller=ActiveJobsController action=index status=200 allocations=20335 duration=25.75 view=20.12"
App 173191 output: [2026-09-09 06:38:38 -0700 ] INFO "method=GET path=/pun/sys/dashboard/activejobs.json format=json controller=ActiveJobsController action=index status=200 allocations=6192 duration=6.74 view=0.00"
Dev’s log looks similar but includes the lines
App 205510 output: [2026-09-09 06:38:00 -0700 ] INFO "execve = [{"SLURM_CONF"=>"/etc/slurm/slurm.conf"}, "/usr/bin/squeue", "--all", "--states=all", "--noconvert", "-o", "\u001E%a\u001F%N\u001F%Y\u001F%j\u001F%u\u001F%P\u001F%M\u001F%A\u001F%t"]"
App 205510 output: [2026-09-09 06:38:01 -0700 ] DEBUG "wrote jobs to stream: 1576"
between the last 2 GETs. I suspect that’s the key, but I have no idea why squeue isn’t being called in prod.
Any ideas appreciated
Ric
--
Env: ood 4.2.4 on Rocky8 on both dev and prod.