New Install, SSH is working, Job Composer is throwing a Ton of ruby errors, and won't open

Get Help
1

Brief environment set-up.
3 compute nodes, and a Slurm client. (also running OPen On Demand)
It is a login node.
SSH connects to iteself fine, but won’t allow the Jobs Composer link.
What does that need in terms of dependencies, and configuration to get working?
(Home Directories mapped via NFS to a NFS share on a Windows Server)

Error snippet:

A problem occurred while initializing your data for this app.
At your own risk you can still open the app or you can just go back to the dashboard

Share this with the developer of your app: The setup production script is supposed to be idempotent and is run each time the user opens the app through the dashboard.

Exception: OodApp::SetupScriptFailed
Per user setup failed for script at /var/www/ood/apps/sys/myjobs/./bin/setup-production for user roybal with output: rake aborted!
ActiveRecord::StatementInvalid: SQLite3::IOException: disk I/O error
/opt/ood/gems/gems/sqlite3-1.4.2/lib/sqlite3/resultset.rb:170:in step'* */opt/ood/gems/gems/sqlite3-1.4.2/lib/sqlite3/resultset.rb:170:in next_hash’
/opt/ood/gems/gems/sqlite3-1.4.2/lib/sqlite3/resultset.rb:105:in next'* */opt/ood/gems/gems/sqlite3-1.4.2/lib/sqlite3/resultset.rb:133:in each’
/opt/ood/gems/gems/sqlite3-1.4.2/lib/sqlite3/database.rb:202:in to_a'* */opt/ood/gems/gems/sqlite3-1.4.2/lib/sqlite3/database.rb:202:in block in execute’
/opt/ood/gems/gems/sqlite3-1.4.2/lib/sqlite3/database.rb:151:in prepare'* */opt/ood/gems/gems/sqlite3-1.4.2/lib/sqlite3/database.rb:193:in execute’

2

This is your error. I’d think of one thing you need on the NFS is to disable locking through the nolock configuration. I just checked our mounts and we use the local_lock=none option.

Additionally, the error, while kind of generic seems to indicate that the sqlite3 file is writable, but the log file (a secondary hidden file sqlite3 uses to make tmp updates) isn’t.

I’d suggest trying to replicate interactively in a shell with this command:

sqlite3 ~/ondemand/data/sys/myjobs/production.sqlite3

If you get it to work interactively in a shell - then it should work in OnDemand. My guess is the CLI sqlite3 command won’t work.

3

I did not have sqlite3 installed on the login node. Are there any other dependencies, for the login node, seperate from the Open OnDemand Webserver?

4

No, everything that’s required to run OnDemand was installed through the package (an .rpm or .deb). The sqlite3 command isn’t a direct requirement - I was only using that to illustrate another way to try to debug this while removing Open OnDemand from the equation.

In any case - this issue is directly related to how those HOME directories are mounted. Do you have the option to disable local locks enabled?

5

Yes, I added the switch, rebooted, same issue. Are any others using a Windows hosted NFS share?

6

Sorry, I’m not aware of any.