I’ve set up a few quick launch apps on our OpenOndemand platform. While they function smoothly with regular apps, I encountered errors for the preset apps, and here is the output:
/var/spool/slurmd/job76753/slurm_script: line 3: module: command not found
Setting VNC password...
Error: no HOME environment variable
Starting VNC server...
vncserver: The HOME environment variable is not set.
vncserver: The HOME environment variable is not set.
vncserver: The HOME environment variable is not set.
vncserver: The HOME environment variable is not set.
vncserver: The HOME environment variable is not set.
vncserver: The HOME environment variable is not set.
vncserver: The HOME environment variable is not set.
vncserver: The HOME environment variable is not set.
vncserver: The HOME environment variable is not set.
vncserver: The HOME environment variable is not set.
vncserver: The HOME environment variable is not set.
vncserver: The HOME environment variable is not set.
vncserver: The HOME environment variable is not set.
vncserver: The HOME environment variable is not set.
vncserver: The HOME environment variable is not set.
vncserver: The HOME environment variable is not set.
vncserver: The HOME environment variable is not set.
vncserver: The HOME environment variable is not set.
vncserver: The HOME environment variable is not set.
vncserver: The HOME environment variable is not set.
Cleaning up...
vncserver: The HOME environment variable is not set.
You have a slurm cluster so that’s weird that that’s happening.
This doesn’t happen with regular desktop applications? I’m struggling to see how OOD can get you into that state because at that point it’s a Slurm thing - slurm doesn’t seem to have completely loaded your bash environment when the job’s running.
Yeah, it doesn’t happen with all regular apps, but it happened with all quick launch apps, also we got weird errors in Job composer:
A problem occurred while initializing your data for this app.
At your own risk you can still open the app or you can just go back to the dashboard
Share this with the developer of your app: The setup production script is supposed to be idempotent and is run each time the user opens the app through the dashboard.
Exception: OodApp::SetupScriptFailed
Per user setup failed for script at /var/www/ood/apps/sys/myjobs/./bin/setup-production for user kenny with output: /usr/share/gems/gems/bundler-2.2.24/lib/bundler/spec_set.rb:90:in `block in materialize': Could not find rdoc-6.3.3 in any of the sources (Bundler::GemNotFound)
from /usr/share/gems/gems/bundler-2.2.24/lib/bundler/spec_set.rb:85:in `map!'
from /usr/share/gems/gems/bundler-2.2.24/lib/bundler/spec_set.rb:85:in `materialize'
from /usr/share/gems/gems/bundler-2.2.24/lib/bundler/definition.rb:244:in `specs_for'
from /usr/share/gems/gems/bundler-2.2.24/lib/bundler/runtime.rb:18:in `setup'
from /usr/share/gems/gems/bundler-2.2.24/lib/bundler.rb:149:in `setup'
from /usr/share/gems/gems/bundler-2.2.24/lib/bundler/setup.rb:20:in `block in <top (required)>'
from /usr/share/gems/gems/bundler-2.2.24/lib/bundler/ui/shell.rb:136:in `with_level'
from /usr/share/gems/gems/bundler-2.2.24/lib/bundler/ui/shell.rb:88:in `silence'
from /usr/share/gems/gems/bundler-2.2.24/lib/bundler/setup.rb:20:in `<top (required)>'
from <internal:/usr/share/rubygems/rubygems/core_ext/kernel_require.rb>:160:in `require'
from <internal:/usr/share/rubygems/rubygems/core_ext/kernel_require.rb>:160:in `rescue in require'
from <internal:/usr/share/rubygems/rubygems/core_ext/kernel_require.rb>:149:in `require'
<internal:/usr/share/rubygems/rubygems/core_ext/kernel_require.rb>:85:in `require': cannot load such file -- bundler/setup (LoadError)
from <internal:/usr/share/rubygems/rubygems/core_ext/kernel_require.rb>:85:in `require'
Stack trace:
/var/www/ood/apps/sys/dashboard/app/apps/ood_app.rb:271:in `block (2 levels) in run_setup_production'
/var/www/ood/apps/sys/dashboard/app/apps/ood_app.rb:257:in `chdir'
/var/www/ood/apps/sys/dashboard/app/apps/ood_app.rb:257:in `block in run_setup_production'
/usr/share/gems/gems/bundler-2.2.24/lib/bundler.rb:397:in `block in with_unbundled_env'
/usr/share/gems/gems/bundler-2.2.24/lib/bundler.rb:701:in `with_env'
/usr/share/gems/gems/bundler-2.2.24/lib/bundler.rb:397:in `with_unbundled_env'
/var/www/ood/apps/sys/dashboard/app/apps/ood_app.rb:254:in `run_setup_production'
/var/www/ood/apps/sys/dashboard/app/controllers/apps_controller.rb:35:in `show'
/opt/ood/ondemand/root/usr/share/gems/3.0/ondemand/3.0.1-1/gems/actionpack-6.1.7.3/lib/action_controller/metal/basic_implicit_render.rb:6:in `send_action'
/opt/ood/ondemand/root/usr/share/gems/3.0/ondemand/3.0.1-1/gems/actionpack-6.1.7.3/lib/abstract_controller/base.rb:228:in `process_action'
/opt/ood/ondemand/root/usr/share/gems/3.0/ondemand/3.0.1-1/gems/actionpack-6.1.7.3/lib/action_controller/metal/rendering.rb:30:in `process_action'
/opt/ood/ondemand/root/usr/share/gems/3.0/ondemand/3.0.1-1/gems/actionpack-6.1.7.3/lib/abstract_controller/callbacks.rb:42:in `block in process_action'
/opt/ood/ondemand/root/usr/share/gems/3.0/ondemand/3.0.1-1/gems/activesupport-6.1.7.3/lib/active_support/callbacks.rb:106:in `run_callbacks'
/opt/ood/ondemand/root/usr/share/gems/3.0/ondemand/3.0.1-1/gems/actionpack-6.1.7.3/lib/abstract_controller/callbacks.rb:41:in `process_action'
/opt/ood/ondemand/root/usr/share/gems/3.0/ondemand/3.0.1-1/gems/actionpack-6.1.7.3/lib/action_controller/metal/rescue.rb:22:in `process_action'
/opt/ood/ondemand/root/usr/share/gems/3.0/ondemand/3.0.1-1/gems/actionpack-6.1.7.3/lib/action_controller/metal/instrumentation.rb:34:in `block in process_action'
/opt/ood/ondemand/root/usr/share/gems/3.0/ondemand/3.0.1-1/gems/activesupport-6.1.7.3/lib/active_support/notifications.rb:203:in `block in instrument'
/opt/ood/ondemand/root/usr/share/gems/3.0/ondemand/3.0.1-1/gems/activesupport-6.1.7.3/lib/active_support/notifications/instrumenter.rb:24:in `instrument'
/opt/ood/ondemand/root/usr/share/gems/3.0/ondemand/3.0.1-1/gems/activesupport-6.1.7.3/lib/active_support/notifications.rb:203:in `instrument'
/opt/ood/ondemand/root/usr/share/gems/3.0/ondemand/3.0.1-1/gems/actionpack-6.1.7.3/lib/action_controller/metal/instrumentation.rb:33:in `process_action'
/opt/ood/ondemand/root/usr/share/gems/3.0/ondemand/3.0.1-1/gems/actionpack-6.1.7.3/lib/action_controller/metal/params_wrapper.rb:249:in `process_action'
/opt/ood/ondemand/root/usr/share/gems/3.0/ondemand/3.0.1-1/gems/actionpack-6.1.7.3/lib/abstract_controller/base.rb:165:in `process'
/opt/ood/ondemand/root/usr/share/gems/3.0/ondemand/3.0.1-1/gems/actionview-6.1.7.3/lib/action_view/rendering.rb:39:in `process'
/opt/ood/ondemand/root/usr/share/gems/3.0/ondemand/3.0.1-1/gems/actionpack-6.1.7.3/lib/action_controller/metal.rb:190:in `dispatch'
/opt/ood/ondemand/root/usr/share/gems/3.0/ondemand/3.0.1-1/gems/actionpack-6.1.7.3/lib/action_controller/metal.rb:254:in `dispatch'
/opt/ood/ondemand/root/usr/share/gems/3.0/ondemand/3.0.1-1/gems/actionpack-6.1.7.3/lib/action_dispatch/routing/route_set.rb:50:in `dispatch'
/opt/ood/ondemand/root/usr/share/gems/3.0/ondemand/3.0.1-1/gems/actionpack-6.1.7.3/lib/action_dispatch/routing/route_set.rb:33:in `serve'
/opt/ood/ondemand/root/usr/share/gems/3.0/ondemand/3.0.1-1/gems/actionpack-6.1.7.3/lib/action_dispatch/journey/router.rb:50:in `block in serve'
/opt/ood/ondemand/root/usr/share/gems/3.0/ondemand/3.0.1-1/gems/actionpack-6.1.7.3/lib/action_dispatch/journey/router.rb:32:in `each'
/opt/ood/ondemand/root/usr/share/gems/3.0/ondemand/3.0.1-1/gems/actionpack-6.1.7.3/lib/action_dispatch/journey/router.rb:32:in `serve'
/opt/ood/ondemand/root/usr/share/gems/3.0/ondemand/3.0.1-1/gems/actionpack-6.1.7.3/lib/action_dispatch/routing/route_set.rb:842:in `call'
/opt/ood/ondemand/root/usr/share/gems/3.0/ondemand/3.0.1-1/gems/rack-2.2.6.4/lib/rack/tempfile_reaper.rb:15:in `call'
/opt/ood/ondemand/root/usr/share/gems/3.0/ondemand/3.0.1-1/gems/rack-2.2.6.4/lib/rack/etag.rb:27:in `call'
/opt/ood/ondemand/root/usr/share/gems/3.0/ondemand/3.0.1-1/gems/rack-2.2.6.4/lib/rack/conditional_get.rb:27:in `call'
/opt/ood/ondemand/root/usr/share/gems/3.0/ondemand/3.0.1-1/gems/rack-2.2.6.4/lib/rack/head.rb:12:in `call'
/opt/ood/ondemand/root/usr/share/gems/3.0/ondemand/3.0.1-1/gems/actionpack-6.1.7.3/lib/action_dispatch/http/permissions_policy.rb:22:in `call'
/opt/ood/ondemand/root/usr/share/gems/3.0/ondemand/3.0.1-1/gems/actionpack-6.1.7.3/lib/action_dispatch/http/content_security_policy.rb:19:in `call'
/opt/ood/ondemand/root/usr/share/gems/3.0/ondemand/3.0.1-1/gems/rack-2.2.6.4/lib/rack/session/abstract/id.rb:266:in `context'
/opt/ood/ondemand/root/usr/share/gems/3.0/ondemand/3.0.1-1/gems/rack-2.2.6.4/lib/rack/session/abstract/id.rb:260:in `call'
/opt/ood/ondemand/root/usr/share/gems/3.0/ondemand/3.0.1-1/gems/actionpack-6.1.7.3/lib/action_dispatch/middleware/cookies.rb:697:in `call'
/opt/ood/ondemand/root/usr/share/gems/3.0/ondemand/3.0.1-1/gems/actionpack-6.1.7.3/lib/action_dispatch/middleware/callbacks.rb:27:in `block in call'
/opt/ood/ondemand/root/usr/share/gems/3.0/ondemand/3.0.1-1/gems/activesupport-6.1.7.3/lib/active_support/callbacks.rb:98:in `run_callbacks'
/opt/ood/ondemand/root/usr/share/gems/3.0/ondemand/3.0.1-1/gems/actionpack-6.1.7.3/lib/action_dispatch/middleware/callbacks.rb:26:in `call'
/opt/ood/ondemand/root/usr/share/gems/3.0/ondemand/3.0.1-1/gems/actionpack-6.1.7.3/lib/action_dispatch/middleware/actionable_exceptions.rb:18:in `call'
/opt/ood/ondemand/root/usr/share/gems/3.0/ondemand/3.0.1-1/gems/actionpack-6.1.7.3/lib/action_dispatch/middleware/debug_exceptions.rb:29:in `call'
/opt/ood/ondemand/root/usr/share/gems/3.0/ondemand/3.0.1-1/gems/actionpack-6.1.7.3/lib/action_dispatch/middleware/show_exceptions.rb:33:in `call'
/opt/ood/ondemand/root/usr/share/gems/3.0/ondemand/3.0.1-1/gems/lograge-0.12.0/lib/lograge/rails_ext/rack/logger.rb:18:in `call_app'
/opt/ood/ondemand/root/usr/share/gems/3.0/ondemand/3.0.1-1/gems/railties-6.1.7.3/lib/rails/rack/logger.rb:26:in `block in call'
/opt/ood/ondemand/root/usr/share/gems/3.0/ondemand/3.0.1-1/gems/activesupport-6.1.7.3/lib/active_support/tagged_logging.rb:99:in `block in tagged'
/opt/ood/ondemand/root/usr/share/gems/3.0/ondemand/3.0.1-1/gems/activesupport-6.1.7.3/lib/active_support/tagged_logging.rb:37:in `tagged'
/opt/ood/ondemand/root/usr/share/gems/3.0/ondemand/3.0.1-1/gems/activesupport-6.1.7.3/lib/active_support/tagged_logging.rb:99:in `tagged'
/opt/ood/ondemand/root/usr/share/gems/3.0/ondemand/3.0.1-1/gems/railties-6.1.7.3/lib/rails/rack/logger.rb:26:in `call'
/opt/ood/ondemand/root/usr/share/gems/3.0/ondemand/3.0.1-1/gems/actionpack-6.1.7.3/lib/action_dispatch/middleware/remote_ip.rb:81:in `call'
/opt/ood/ondemand/root/usr/share/gems/3.0/ondemand/3.0.1-1/gems/request_store-1.5.1/lib/request_store/middleware.rb:19:in `call'
/opt/ood/ondemand/root/usr/share/gems/3.0/ondemand/3.0.1-1/gems/actionpack-6.1.7.3/lib/action_dispatch/middleware/request_id.rb:26:in `call'
/opt/ood/ondemand/root/usr/share/gems/3.0/ondemand/3.0.1-1/gems/rack-2.2.6.4/lib/rack/method_override.rb:24:in `call'
/opt/ood/ondemand/root/usr/share/gems/3.0/ondemand/3.0.1-1/gems/rack-2.2.6.4/lib/rack/runtime.rb:22:in `call'
/opt/ood/ondemand/root/usr/share/gems/3.0/ondemand/3.0.1-1/gems/actionpack-6.1.7.3/lib/action_dispatch/middleware/executor.rb:14:in `call'
/opt/ood/ondemand/root/usr/share/gems/3.0/ondemand/3.0.1-1/gems/rack-2.2.6.4/lib/rack/sendfile.rb:110:in `call'
/opt/ood/ondemand/root/usr/share/gems/3.0/ondemand/3.0.1-1/gems/actionpack-6.1.7.3/lib/action_dispatch/middleware/host_authorization.rb:148:in `call'
/opt/ood/ondemand/root/usr/share/gems/3.0/ondemand/3.0.1-1/gems/railties-6.1.7.3/lib/rails/engine.rb:539:in `call'
/opt/rh/ondemand/root/usr/share/ruby/vendor_ruby/phusion_passenger/rack/thread_handler_extension.rb:107:in `process_request'
/opt/rh/ondemand/root/usr/share/ruby/vendor_ruby/phusion_passenger/request_handler/thread_handler.rb:149:in `accept_and_process_next_request'
/opt/rh/ondemand/root/usr/share/ruby/vendor_ruby/phusion_passenger/request_handler/thread_handler.rb:110:in `main_loop'
/opt/rh/ondemand/root/usr/share/ruby/vendor_ruby/phusion_passenger/request_handler.rb:419:in `block (3 levels) in start_threads'
/opt/rh/ondemand/root/usr/share/ruby/vendor_ruby/phusion_passenger/utils.rb:113:in `block in create_thread_and_abort_on_exception'
Thank you for your very quick response, it did fix that issue. Regarding the previous error, I am curious, the preset app and regular app should share the same environment, is that right? What could cause the slurm didn’t load the bash environment?
This is also my question. What’s the behavior of the regular app that provides choices? If all you’ve provided is the form.yml.erb - that’s the only thing that changes - the HTML form that is shown to the user. The sub-app will use the same submit.yml.erb and the same script.sh.
You can check you ondemand-nginx logs for execve commands to see that we issue basically the same sbatch command either way.
Certainly, I’ll investigate that, thank you. One important point to note is that the user_defined_context.json file is empty for the preset app, whereas in the regular app, it contains the following information:
I think this is something weird with the