Hi folks,
Due to the extreme popularity of OnDemand, we are now getting requests to setup multiple focused instances on various clusters. Currently, we are running into the following error, and I have a suspicion what is going on:
...
App 133531 output: config.ru:1:in `new'
App 133531 output: config.ru:1:in `<main>'
App 133531 output: /opt/rh/ondemand/root/usr/share/passenger/helper-scripts/rack-loader.rb:96:in `eval'
App 133531 output: /opt/rh/ondemand/root/usr/share/passenger/helper-scripts/rack-loader.rb:96:in `load_app'
App 133531 output: /opt/rh/ondemand/root/usr/share/passenger/helper-scripts/rack-loader.rb:116:in `block in <module:App>'
App 133531 output: /opt/rh/ondemand/root/usr/share/ruby/vendor_ruby/phusion_passenger/loader_shared_helpers.rb:399:in `run_block_and_record_step_progress'
App 133531 output: /opt/rh/ondemand/root/usr/share/passenger/helper-scripts/rack-loader.rb:115:in `<module:App>'
App 133531 output: /opt/rh/ondemand/root/usr/share/passenger/helper-scripts/rack-loader.rb:28:in `<module:PhusionPassenger>'
App 133531 output: /opt/rh/ondemand/root/usr/share/passenger/helper-scripts/rack-loader.rb:27:in `<main>'
[ E 2024-09-30 12:25:35.8075 133367/T1h age/Cor/App/Implementation.cpp:221 ]: Could not spawn process for application /var/www/ood/apps/sys/dashboard: The application encountered the following error: File exists @ dir_s_mkdir - /home (Errno::EEXIST)
Error ID: b328cb3a
Error details saved to: /tmp/passenger-error-bXKqbg.html
[ E 2024-09-30 12:25:35.8146 133367/Tc age/Cor/Con/CheckoutSession.cpp:283 ]: [Client 3-1] Cannot checkout session because a spawning error occurred. The identifier of the error is b328cb3a. Please see earlier logs for details about the error.
[ N 2024-09-30 14:00:04.6648 133367/T8 age/Cor/CoreMain.cpp:671 ]: Signal received. Gracefully shutting down... (send signal 2 more time(s) to force shutdown)
[ N 2024-09-30 14:00:04.6650 133367/T1 age/Cor/CoreMain.cpp:1246 ]: Received command to shutdown gracefully. Waiting until all clients have disconnected...
[ N 2024-09-30 14:00:04.6651 133367/T8 Ser/Server.h:902 ]: [ServerThr.1] Freed 0 spare client objects
[ N 2024-09-30 14:00:04.6651 133367/T8 Ser/Server.h:558 ]: [ServerThr.1] Shutdown finished
[ N 2024-09-30 14:00:04.6652 133367/Te Ser/Server.h:902 ]: [ServerThr.4] Freed 0 spare client objects
[ N 2024-09-30 14:00:04.6652 133367/Te Ser/Server.h:558 ]: [ServerThr.4] Shutdown finished
[ N 2024-09-30 14:00:04.6652 133367/Tj Ser/Server.h:902 ]: [ServerThr.6] Freed 0 spare client objects
[ N 2024-09-30 14:00:04.6652 133367/Tj Ser/Server.h:558 ]: [ServerThr.6] Shutdown finished
[ N 2024-09-30 14:00:04.6652 133367/Tl Ser/Server.h:902 ]: [ServerThr.7] Freed 0 spare client objects
[ N 2024-09-30 14:00:04.6653 133367/Ta Ser/Server.h:902 ]: [ServerThr.2] Freed 0 spare client objects
[ N 2024-09-30 14:00:04.6652 133367/Tg Ser/Server.h:902 ]: [ServerThr.5] Freed 0 spare client objects
[ N 2024-09-30 14:00:04.6653 133367/Tl Ser/Server.h:558 ]: [ServerThr.7] Shutdown finished
[ N 2024-09-30 14:00:04.6653 133367/Ta Ser/Server.h:558 ]: [ServerThr.2] Shutdown finished
...
From what I picked up from other posts, the CSRF token associated with the previous instance could throw this error. Sticky sessions are not the solution for our instance as some users need to be able to access the different instances, sometimes simultaneously. My account allowed me to use both instances, but most other users cannot.
Is there an environment variable that controls the path /home/user/ondemand
? Is there an environment variable I can set to change the name of this directory? I looked in the docs, but could not find anything.
Wondering how other folks solved this problem as well.
Thanks in advance!
Walid