Ood v4.1.1, Shell Access failed to launch

Get Help
1

OS: Rocky Linux release 10.1 (Red Quartz)
ood: 4.1.1

Click the menu: Clusters: Shell Access, got the error.
The web page says:
We’re sorry, but something went wrong.

Here is the related log file, the last 20 lines.

tail -n 20 /var/log/ondemand-nginx/tom/error.log

App 5955 output:  9: 0x14b9c642c063 v8::internal::PagedSpaceAllocatorPolicy::TryExpandAndAllocate(unsigned long, v8::internal::AllocationOrigin) [/lib64/libnode.so.127]
App 5955 output: 10: 0x14b9c642c2ce v8::internal::PagedSpaceAllocatorPolicy::RefillLab(int, v8::internal::AllocationOrigin) [/lib64/libnode.so.127]
App 5955 output: 11: 0x14b9c642a94d v8::internal::MainAllocator::EnsureAllocation(int, v8::internal::AllocationAlignment, v8::internal::AllocationOrigin) [/lib64/libnode.so.127]
App 5955 output: 12: 0x14b9c642abc2 v8::internal::MainAllocator::AllocateRawSlowUnaligned(int, v8::internal::AllocationOrigin) [/lib64/libnode.so.127]
App 5955 output: 13: 0x14b9c63ce2d5 v8::internal:
:CodeBuilder::AllocateUninitializedInstructionStream(bool) [/lib64/libnode.so.127]
App 5955 output: 14: 0x14b9c63e47ef v8::internal:
:CodeBuilder::BuildInternal(bool) [/lib64/libnode.so.127]
App 5955 output: 15: 0x14b9c6a664c3 v8::internal::baseline::BaselineCompiler::Build(v8::internal::LocalIsolate*) [/lib64/libnode.so.127]
App 5955 output: 16: 0x14b9c62131d6 v8::internal::GenerateBaselineCode(v8::internal::Isolate*, v8::internal::Handlev8::internal::SharedFunctionInfo) [/lib64/libnode.so.127]
App 5955 output: 17: 0x14b9c627de2b v8::internal::Compiler::CompileSharedWithBaseline(v8::internal::Isolate*, v8::internal::Handlev8::internal::SharedFunctionInfo, v8::internal::Compiler::ClearExceptionFlag, v8::internal::IsCompiledScope*) [/lib64/libnode.so.127]
App 5955 output: 18: 0x14b9c6281ba7 v8::internal::Compiler::CompileBaseline(v8::internal::Isolate*, v8::internal::Handlev8::internal::JSFunction, v8::internal::Compiler::ClearExceptionFlag, v8::internal::IsCompiledScope*) [/lib64/libnode.so.127]
App 5955 output: 19: 0x14b9c6a647d2 v8::internal::baseline::BaselineBatchCompiler::CompileBatch(v8::internal::Handlev8::internal::JSFunction) [/lib64/libnode.so.127]
App 5955 output: 20: 0x14b9c637130b v8::internal::TieringManager::OnInterruptTick(v8::internal::Handlev8::internal::JSFunction, v8::internal::CodeKind) [/lib64/libnode.so.127]
App 5955 output: 21: 0x14b9c68e503d  [/lib64/libnode.so.127]
App 5955 output: 22: 0x14b9c68e6fd5 v8::internal::Runtime_BytecodeBudgetInterruptWithStackCheck_Ignition(int, unsigned long*, v8::internal::Isolate*) [/lib64/libnode.so.127]
App 5955 output: 23: 0x14b96602c776
[ E 2026-02-22 15:12:40.0534 3964/T1i age/Cor/App/Implementation.cpp:218 ]: Could not spawn process for application /var/www/ood/apps/sys/shell: The application process exited prematurely.
Error ID: 15eb60ee
Error details saved to: /tmp/passenger-error-WSJHqm.html

[ E 2026-02-22 15:12:40.0569 3964/Td age/Cor/Con/CheckoutSession.cpp:368 ]: [Client 3-3] Cannot checkout session because a spawning error occurred. The identifier of the error is 15eb60ee. Please see earlier logs for details about the error.
2

It appears to be a text core dump or a stack trace with memory pointers.

I any case, I think I’d like to see the full output as I’m not really sure what’s going on here.

I guess I’d ask if you have selinux on and/or if there’s anything in /var/log/audit. I can’t really tell what’s going on here, but it appears to have some trouble allocating memory?

3

OS: Rocky Linux 10.1.
SELinux, disabled by running the cmd, grubby --update-kernel ALL --args selinux=0

Removed all the files under the folder, /var/log/ondemand-nginx/tom
Restart the OS.

Then login the ood with user tom, menu, Clusters, Shell Access, reports,

We’s sorry, but something went wrong.

The following is the full error.log

root@login:/var/log/ondemand-nginx/tom# cat error.log
[ N 2026-02-24 13:50:31.0445 132784/T1 age/Wat/WatchdogMain.cpp:1365 ]: Starting Passenger watchdog…
[ N 2026-02-24 13:50:31.0699 132788/T1 age/Cor/CoreMain.cpp:1284 ]: Starting Passenger core…
[ N 2026-02-24 13:50:31.0700 132788/T1 age/Cor/CoreMain.cpp:245 ]: Passenger core running in multi-application mode.
[ N 2026-02-24 13:50:31.1138 132788/T1 age/Cor/CoreMain.cpp:962 ]: Passenger core online, PID 132788
App 132822 output: [2026-02-24 13:50:33 +0800 ]  WARN “Error opening MOTD at \nException: bad URI (is not URI?): nil”
App 132822 output: [2026-02-24 13:50:33 +0800 ] DEBUG “Rendering layout layouts/application.html.erb”
App 132822 output: [2026-02-24 13:50:33 +0800 ] DEBUG “Rendering dashboard/index.html.erb within layouts/application”
App 132822 output: [2026-02-24 13:50:33 +0800 ]  INFO “method=GET path=/pun/sys/dashboard/ format=html controller=DashboardController action=index status=200 allocations=100398 duration=149.42 view=82.14”
App 133698 output:
App 133698 output:
App 133698 output: #
App 133698 output: # Fatal error in , line 0
App 133698 output: # Check failed: 12 == (__errno_location ()).
App 133698 output: #
App 133698 output: #
App 133698 output: #
App 133698 output: #FailureMessage Object: 0x7ffe21d686f0
App 133698 output:
App 133698 output: ----- Native stack trace -----
App 133698 output:
App 133698 output:  1: 0x152bc664e119  [/lib64/libnode.so.127]
App 133698 output:  2: 0x152bc6cf268f V8_Fatal(char const, …) [/lib64/libnode.so.127]
App 133698 output:  3: 0x152bc6cf9ee5  [/lib64/libnode.so.127]
App 133698 output:  4: 0x152bc8862bbe v8::internal::MemoryAllocator::SetPermissionsOnExecutableMemoryChunk(v8::internal::VirtualMemory*, unsigned long, unsigned long, unsigned long) [/lib64/libnode.so.127]
App 133698 output:  5: 0x152bc8862f48 v8::internal::MemoryAllocator::AllocateAlignedMemory(unsigned long, unsigned long, unsigned long, v8::internal::AllocationSpace, v8::internal::Executability, void*, v8::internal::VirtualMemory*) [/lib64/libnode.so.127]
App 133698 output:  6: 0x152bc8863026 v8::internal::MemoryAllocator::AllocateUninitializedChunkAt(v8::internal::BaseSpace*, unsigned long, v8::internal::Executability, unsigned long, v8::internal::PageSize) [/lib64/libnode.so.127]
App 133698 output:  7: 0x152bc88634a9 v8::internal::MemoryAllocator::AllocatePage(v8::internal::MemoryAllocator::AllocationMode, v8::internal::Space*, v8::internal::Executability) [/lib64/libnode.so.127]
App 133698 output:  8: 0x152bc88810b7 v8::internal::PagedSpaceBase::TryExpand(v8::internal::LocalHeap*, v8::internal::AllocationOrigin) [/lib64/libnode.so.127]
App 133698 output:  9: 0x152bc882c063 v8::internal::PagedSpaceAllocatorPolicy::TryExpandAndAllocate(unsigned long, v8::internal::AllocationOrigin) [/lib64/libnode.so.127]
App 133698 output: 10: 0x152bc882c2ce v8::internal::PagedSpaceAllocatorPolicy::RefillLab(int, v8::internal::AllocationOrigin) [/lib64/libnode.so.127]
App 133698 output: 11: 0x152bc882a94d v8::internal::MainAllocator::EnsureAllocation(int, v8::internal::AllocationAlignment, v8::internal::AllocationOrigin) [/lib64/libnode.so.127]
App 133698 output: 12: 0x152bc882abc2 v8::internal::MainAllocator::AllocateRawSlowUnaligned(int, v8::internal::AllocationOrigin) [/lib64/libnode.so.127]
App 133698 output: 13: 0x152bc87ce2d5 v8::internal:
:CodeBuilder::AllocateUninitializedInstructionStream(bool) [/lib64/libnode.so.127]
App 133698 output: 14: 0x152bc87e47ef v8::internal:
:CodeBuilder::BuildInternal(bool) [/lib64/libnode.so.127]
App 133698 output: 15: 0x152bc8e664c3 v8::internal::baseline::BaselineCompiler::Build(v8::internal::LocalIsolate*) [/lib64/libnode.so.127]
App 133698 output: 16: 0x152bc86131d6 v8::internal::GenerateBaselineCode(v8::internal::Isolate*, v8::internal::Handlev8::internal::SharedFunctionInfo) [/lib64/libnode.so.127]
App 133698 output: 17: 0x152bc867de2b v8::internal::Compiler::CompileSharedWithBaseline(v8::internal::Isolate*, v8::internal::Handlev8::internal::SharedFunctionInfo, v8::internal::Compiler::ClearExceptionFlag, v8::internal::IsCompiledScope*) [/lib64/libnode.so.127]
App 133698 output: 18: 0x152bc8681ba7 v8::internal::Compiler::CompileBaseline(v8::internal::Isolate*, v8::internal::Handle<v8::internal::JSFunction
App 133698 output: >, v8::internal::Compiler::ClearExceptionFlag, v8::internal::IsCompiledScope*) [/lib64/libnode.so.127]
App 133698 output: 19: 0x152bc8e647d2 v8::internal::baseline::BaselineBatchCompiler::CompileBatch(v8::internal::Handlev8::internal::JSFunction) [/lib64/libnode.so.127]
App 133698 output: 20: 0x152bc877130b v8::internal::TieringManager::OnInterruptTick(v8::internal::Handlev8::internal::JSFunction, v8::internal::CodeKind) [/lib64/libnode.so.127]
App 133698 output: 21: 0x152bc8ce503d  [/lib64/libnode.so.127]
App 133698 output: 22: 0x152bc8ce6fd5 v8::internal::Runtime_BytecodeBudgetInterruptWithStackCheck_Ignition(int, unsigned long*, v8::internal::Isolate*) [/lib64/libnode.so.127]
App 133698 output: 23: 0x152b6842c776
[ E 2026-02-24 13:51:57.6888 132788/Tx age/Cor/App/Implementation.cpp:218 ]: Could not spawn process for application /var/www/ood/apps/sys/shell: The application process exited prematurely.
Error ID: 990139a5
Error details saved to: /tmp/passenger-error-2fCjAj.html

[ E 2026-02-24 13:51:57.6937 132788/Tb age/Cor/Con/CheckoutSession.cpp:368 ]: [Client 2-1] Cannot checkout session because a spawning error occurred. The identifier of the error is 990139a5. Please see earlier logs for details about the error.
App 135108 output:
App 135108 output:
App 135108 output: #
App 135108 output: # Fatal error in , line 0
App 135108 output: # Check failed: 12 == (__errno_location ()).
App 135108 output: #
App 135108 output: #
App 135108 output: #
App 135108 output: #FailureMessage Object: 0x7ffc44f45050
App 135108 output: ----- Native stack trace -----
App 135108 output:
App 135108 output:  1: 0x14ce3524e119  [/lib64/libnode.so.127]
App 135108 output:  2: 0x14ce358f268f V8_Fatal(char const, …) [/lib64/libnode.so.127]
App 135108 output:  3: 0x14ce358f9ee5  [/lib64/libnode.so.127]
App 135108 output:  4: 0x14ce37462bbe v8::internal::MemoryAllocator::SetPermissionsOnExecutableMemoryChunk(v8::internal::VirtualMemory*, unsigned long, unsigned long, unsigned long) [/lib64/libnode.so.127]
App 135108 output:  5: 0x14ce37462f48 v8::internal::MemoryAllocator::AllocateAlignedMemory(unsigned long, unsigned long, unsigned long, v8::internal::AllocationSpace, v8::internal::Executability, void*, v8::internal::VirtualMemory*) [/lib64/libnode.so.127]
App 135108 output:  6: 0x14ce37463026 v8::internal::MemoryAllocator::AllocateUninitializedChunkAt(v8::internal::BaseSpace*, unsigned long, v8::internal::Executability, unsigned long, v8::internal::PageSize) [/lib64/libnode.so.127]
App 135108 output:  7: 0x14ce374634a9 v8::internal::MemoryAllocator::AllocatePage(v8::internal::MemoryAllocator::AllocationMode, v8::internal::Space*, v8::internal::Executability) [/lib64/libnode.so.127]
App 135108 output:  8: 0x14ce374810b7 v8::internal::PagedSpaceBase::TryExpand(v8::internal::LocalHeap*, v8::internal::AllocationOrigin) [/lib64/libnode.so.127]
App 135108 output:  9: 0x14ce3742c063 v8::internal::PagedSpaceAllocatorPolicy::TryExpandAndAllocate(unsigned long, v8::internal::AllocationOrigin) [/lib64/libnode.so.127]
App 135108 output: 10: 0x14ce3742c2ce v8::internal::PagedSpaceAllocatorPolicy::RefillLab(int, v8::internal::AllocationOrigin) [/lib64/libnode.so.127]
App 135108 output: 11: 0x14ce3742a94d v8::internal::MainAllocator::EnsureAllocation(int, v8::internal::AllocationAlignment, v8::internal::AllocationOrigin) [/lib64/libnode.so.127]
App 135108 output: 12: 0x14ce3742abc2 v8::internal::MainAllocator::AllocateRawSlowUnaligned(int, v8::internal::AllocationOrigin) [/lib64/libnode.so.127]
App 135108 output: 13: 0x14ce373ce2d5 v8::internal:
:CodeBuilder::AllocateUninitializedInstructionStream(bool) [/lib64/libnode.so.127]
App 135108 output: 14: 0x14ce373e47ef v8::internal:
:CodeBuilder::BuildInternal(bool) [/lib64/libnode.so.127]
App 135108 output: 15: 0x14ce37a664c3 v8::internal::baseline::BaselineCompiler::Build(v8::internal::LocalIsolate*) [/lib64/libnode.so.127]
App 135108 output: 16: 0x14ce372131d6 v8::internal::GenerateBaselineCode(v8::internal::Isolate*, v8::internal::Handlev8::internal::SharedFunctionInfo) [/lib64/libnode.so.127]
App 135108 output: 17: 0x14ce3727de2b v8::internal::Compiler::CompileSharedWithBaseline(v8::internal::Isolate*, v8::internal::Handlev8::internal::SharedFunctionInfo, v8::internal::Compiler::ClearExceptionFlag, v8::internal::IsCompiledScope*) [/lib64/libnode.so.127]
App 135108 output: 18: 0x14ce37281ba7 v8::internal::Compiler::CompileBaseline(v8::internal::Isolate*, v8::internal::Handlev8::internal::JSFunction, v8::internal::Compiler::ClearExceptionFlag, v8::internal::IsCompiledScope*) [/lib64/libnode.so.127]
App 135108 output: 19: 0x14ce37a647d2 v8::internal::baseline::BaselineBatchCompiler::CompileBatch(v8::internal::Handlev8::internal::JSFunction) [/lib64/libnode.so.127]
App 135108 output: 20: 0x14ce3737130b v8::internal::TieringManager::OnInterruptTick(v8::internal::Handlev8::internal::JSFunction, v8::internal::CodeKind) [/lib64/libnode.so.127]
App 135108 output: 21: 0x14ce378e503d  [/lib64/libnode.so.127]
App 135108 output: 22: 0x14ce378e6fd5 v8::internal::Runtime_BytecodeBudgetInterruptWithStackCheck_Ignition(int, unsigned long*, v8::internal::Isolate*) [/lib64/libnode.so.127]
App 135108 output: 23: 0x14cdd702c776
[ E 2026-02-24 13:54:25.3075 132788/T12 age/Cor/App/Implementation.cpp:218 ]: Could not spawn process for application /var/www/ood/apps/sys/shell: The application process exited prematurely.
Error ID: c3712949
Error details saved to: /tmp/passenger-error-WP6BLa.html

[ E 2026-02-24 13:54:25.3117 132788/Td age/Cor/Con/CheckoutSession.cpp:368 ]: [Client 3-1] Cannot checkout session because a spawning error occurred. The identifier of the error is c3712949. Please see earlier logs for details about the error.
root@login:/var/log/ondemand-nginx/tom

Screenshot 2026-02-24 at 13.59.51
Screenshot 2026-02-24 at 13.59.511676×1646 229 KB

In the meanwhile, the /var/log/audit reports,

type=USER_END msg=audit(1771912984.018:1034): pid=139949 uid=0 auid=0 ses=81 msg='op=PAM:session_close grantors=pam_loginuid,pam_keyinit,pam_limits,pam_systemd acct=“root” exe=“/usr/sbin/crond” hostname=? addr=? terminal=cron res=success’UID=“root” AUID=“root”
type=ANOM_ABEND msg=audit(1771913024.188:1035): auid=4294967295 uid=10000 gid=10000 ses=4294967295 pid=140411 comm=“node” exe=“/usr/bin/node-22” sig=5 res=1AUID=“unset” UID=“tom” GID=“demo_grp”
type=BPF msg=audit(1771913024.194:1036): prog-id=145 op=LOAD
type=BPF msg=audit(1771913024.194:1037): prog-id=146 op=LOAD
type=BPF msg=audit(1771913024.194:1038): prog-id=147 op=LOAD
type=SERVICE_START msg=audit(1771913024.195:1039): pid=1 uid=0 auid=4294967295 ses=4294967295 msg='unit=systemd-coredump@9-140443-0 comm=“systemd” exe=“/usr/lib/systemd/systemd” hostname=? addr=? terminal=? res=success’UID=“root” AUID=“unset”
type=SERVICE_STOP msg=audit(1771913024.252:1040): pid=1 uid=0 auid=4294967295 ses=4294967295 msg='unit=systemd-coredump@9-140443-0 comm=“systemd” exe=“/usr/lib/systemd/systemd” hostname=? addr=? terminal=? res=success’UID=“root” AUID=“unset”
type=BPF msg=audit(1771913024.276:1041): prog-id=147 op=UNLOAD
type=BPF msg=audit(1771913024.276:1042): prog-id=146 op=UNLOAD
type=BPF msg=audit(1771913024.276:1043): prog-id=145 op=UNLOAD
type=ANOM_ABEND msg=audit(1771913037.621:1044): auid=4294967295 uid=10000 gid=10000 ses=4294967295 pid=140587 comm=“node” exe=“/usr/bin/node-22” sig=5 res=1AUID=“unset” UID=“tom” GID=“demo_grp”
type=BPF msg=audit(1771913037.627:1045): prog-id=148 op=LOAD
type=BPF msg=audit(1771913037.627:1046): prog-id=149 op=LOAD
type=BPF msg=audit(1771913037.627:1047): prog-id=150 op=LOAD
type=SERVICE_START msg=audit(1771913037.627:1048): pid=1 uid=0 auid=4294967295 ses=4294967295 msg='unit=systemd-coredump@10-140619-0 comm=“systemd” exe=“/usr/lib/systemd/systemd” hostname=? addr=? terminal=? res=success’UID=“root” AUID=“unset”
type=SERVICE_STOP msg=audit(1771913037.675:1049): pid=1 uid=0 auid=4294967295 ses=4294967295 msg='unit=systemd-coredump@10-140619-0 comm=“systemd” exe=“/usr/lib/systemd/systemd” hostname=? addr=? terminal=? res=success’UID=“root” AUID=“unset”
type=BPF msg=audit(1771913037.705:1050): prog-id=150 op=UNLOAD
type=BPF msg=audit(1771913037.705:1051): prog-id=149 op=UNLOAD
type=BPF msg=audit(1771913037.705:1052): prog-id=148 op=UNLOAD
type=ANOM_ABEND msg=audit(1771913063.006:1053): auid=4294967295 uid=10000 gid=10000 ses=4294967295 pid=140836 comm=“node” exe=“/usr/bin/node-22” sig=5 res=1AUID=“unset” UID=“tom” GID=“demo_grp”
type=BPF msg=audit(1771913063.013:1054): prog-id=151 op=LOAD
type=BPF msg=audit(1771913063.013:1055): prog-id=152 op=LOAD
type=BPF msg=audit(1771913063.013:1056): prog-id=153 op=LOAD
type=SERVICE_START msg=audit(1771913063.014:1057): pid=1 uid=0 auid=4294967295 ses=4294967295 msg='unit=systemd-coredump@11-140876-0 comm=“systemd” exe=“/usr/lib/systemd/systemd” hostname=? addr=? terminal=? res=success’UID=“root” AUID=“unset”
type=SERVICE_STOP msg=audit(1771913063.067:1058): pid=1 uid=0 auid=4294967295 ses=4294967295 msg='unit=systemd-coredump@11-140876-0 comm=“systemd” exe=“/usr/lib/systemd/systemd” hostname=? addr=? terminal=? res=success’UID=“root” AUID=“unset”
type=BPF msg=audit(1771913063.089:1059): prog-id=153 op=UNLOAD
type=BPF msg=audit(1771913063.089:1060): prog-id=152 op=UNLOAD
type=BPF msg=audit(1771913063.089:1061): prog-id=151 op=UNLOAD

4

OK well I can tell from the stack trace it’s trying to set permissions on executable memory which is quite low level for me.

I don’t see selinux denying this - but I do see BPF running. I found this issue below which appears to be very similar to this and they’re talking about rsyslog - something I’m not familiar with.

Apparently their fix is disabling MemoryDenyWriteExecute on Rsyslog systemd service file. If that doesn’t seem to have any relevance to you, then I’d check what the BPF service/program is up to.

5

Thanks.
You are right.
Setting MemoryDenyWriteExecute=no fix the issue.