Ondemand passenger error

Get Help
1

Some users are experiencing this, error on opening dashboard,

image
image1920×1046 29.4 KB

On checking the nginx logs of the user, we find this,

App 22786 output: Error: The application encountered the following error: No such file or directory - connect(2) for /tmp/passenger.dJ28FMk/apps.s/ruby.4vAJzyAC9Gt60i3gpPp1Wkp2uvRVvx3uKiztUM5R88puLgufhtaSvluN8kYW96S (Errno::ENOENT)
App 22786 output:     /opt/rh/ondemand/root/usr/share/ruby/vendor_ruby/phusion_passenger/request_handler.rb:305:in `initialize'
App 22786 output:     /opt/rh/ondemand/root/usr/share/ruby/vendor_ruby/phusion_passenger/request_handler.rb:305:in `new'
App 22786 output:     /opt/rh/ondemand/root/usr/share/ruby/vendor_ruby/phusion_passenger/request_handler.rb:305:in `block in create_unix_socket_on_filesystem'

App 22786 output:     /opt/rh/ondemand/root/usr/share/ruby/vendor_ruby/phusion_passenger/utils.rb:65:in `block in retry_at_most'

App 22786 output:     /opt/rh/ondemand/root/usr/share/ruby/vendor_ruby/phusion_passenger/utils.rb:63:in `times'

App 22786 output:     /opt/rh/ondemand/root/usr/share/ruby/vendor_ruby/phusion_passenger/utils.rb:63:in `retry_at_most'

App 22786 output:     /opt/rh/ondemand/root/usr/share/ruby/vendor_ruby/phusion_passenger/request_handler.rb:302:in `create_unix_socket_on_filesystem'

App 22786 output:     /opt/rh/ondemand/root/usr/share/ruby/vendor_ruby/phusion_passenger/request_handler.rb:100:in `initialize'

App 22786 output:     /opt/rh/ondemand/root/usr/share/passenger/helper-scripts/rack-loader.rb:123:in `new'

App 22786 output:     /opt/rh/ondemand/root/usr/share/passenger/helper-scripts/rack-loader.rb:123:in `block in <module:App>'

App 22786 output:     /opt/rh/ondemand/root/usr/share/ruby/vendor_ruby/phusion_passenger/loader_shared_helpers.rb:397:in `run_block_and_record_step_progress'

App 22786 output:     /opt/rh/ondemand/root/usr/share/passenger/helper-scripts/rack-loader.rb:120:in `<module:App>'

App 22786 output:     /opt/rh/ondemand/root/usr/share/passenger/helper-scripts/rack-loader.rb:28:in `<module:PhusionPassenger>'

App 22786 output:     /opt/rh/ondemand/root/usr/share/passenger/helper-scripts/rack-loader.rb:27:in `<main>'

[ E 2023-04-10 08:34:26.6131 462/T5a age/Cor/App/Implementation.cpp:221 ]: Could not spawn process for application /var/www/ood/apps/sys/dashboard: The application encountered the following error: No such file or directory - connect(2) for /tmp/passenger.dJ28FMk/apps.s/ruby.4vAJzyAC9Gt60i3gpPp1Wkp2uvRVvx3uKiztUM5R88puLgufhtaSvluN8kYW96S (Errno::ENOENT)

  Error ID: c4e8a3f7

  Error details saved to: /tmp/passenger-error-YinRxQ.html

 

[ E 2023-04-10 08:34:26.6312 462/Ta age/Cor/Con/CheckoutSession.cpp:283 ]: [Client 2-169] Cannot checkout session because a spawning error occurred. The identifier of the error is c4e8a3f7. Please see earlier logs for details about the error.

Killing the nginx process for the particular user brings it back to normalcy, but any idea why this happens, it happened to few (not all) of our users at the same day.

2

This is the tempfile that passenger uses to boot the app and connect to it. I’m wondering 2 things:

One, if you’re OS is cleaning that tmp directory rather aggressively. I.e., removing those directories when the files are still being accessed.

Or if you’re running out of resources attempting to create the file. journalctl may say something about resource exhaustion, though I’m not sure if that’s the issue or it’s infact that the files are being removed too early.

3

I verified that the files are being removed early, once I created a directory called apps.s under tmp/passenger., it again started working, unsure how this is happened to 3 users at once, we use AL2

4

We have our ondemand inside an apptainer and /tmp directory was shared with the host by default, on moving /tmp exclusively to apptainer, host no longer cleaned up the files and so we didn’t have this issue after that change.

5

This topic was automatically closed 180 days after the last reply. New replies are no longer allowed.