OOD 1.7.0 Job Composer issue - corrupted SQLite3 database?

Get Help
1

Hello,

One of our users has reported a problem whenever she attempts to use the Job Composer in Open OnDemand 1.7.0. When she does so, she is met with an error dialog box stating “We’re sorry but something went wrong. If you are the application owner check the logs for more information.” In the error.log under her /var/log/ondemand-nginx/ subdirectory, I find these lines reporting a fatal error:

App 1426 output: [2020-09-09 10:24:56 -0500 ]  INFO "method=GET path=/pun/sys/myjobs/ format=html controller=WorkflowsController action=index status=500 error='ActionView::Template::Error: SQLite3::CorruptException: database disk image is malformed: SELECT \"jobs\".* FROM \"jobs\" WHERE \"jobs\".\"workflow_id\" IN (?, ?, ?, ?, ?, ?, ?)' duration=121.83 view=0.00 db=21.25"
App 1426 output: [2020-09-09 10:24:56 -0500 ] FATAL ""
App 1426 output: [2020-09-09 10:24:56 -0500 ] FATAL "ActionView::Template::Error (SQLite3::CorruptException: database disk image is malformed: SELECT \"jobs\".* FROM \"jobs\" WHERE \"jobs\".\"workflow_id\" IN (?, ?, ?, ?, ?, ?, ?)):"
App 1426 output: [2020-09-09 10:24:56 -0500 ] FATAL "78:           </thead>\n    79: \n    80:           <tbody>\n    81:           <% @workflows.each do |workflow| %>\n    82:               <tr class=\"job-row<%= \" missing-cluster\" unless workflow.batch_host_exists? %><%= \" missing-dir\" unless workflow.staged_dir_exists? %><%= \" missing-script\" unless workflow.script_name && workflow.staged_script_exists? %>\" id=\"<%= workflow.id %>\">\n    83:                 <td data-sort=\"<%= workflow.id %>\"><small><%= local_time(workflow.created_at) %></small></td>\n    84:                 <td><span title=\"<%= workflow.name %>\"><strong><%= workflow.name %></strong></span></td>"
App 1426 output: [2020-09-09 10:24:56 -0500 ] FATAL ""
App 1426 output: [2020-09-09 10:24:56 -0500 ] FATAL "app/views/workflows/index.html.erb:81:in `_app_views_workflows_index_html_erb__46927622005262942_23456431227300'"

What is the best way for me to remediate this issue? I know that there is a production.sqlite3 database file in her ondemand/data/sys/myjobs subdirectory. Is that the database that is corrupted? Would removing or renaming that file be the correct course of action? If so, does the user lose anything?

Thank you,

Richard

2

Removing or renaming that file and then asking the user to reload would solve the problem. However, we have not yet added anything to re-populate that database from the directory containing job data. Seems like this would be useful, even if it were just a rake task the user could run from the command line.

The database contains the list of jobs that were submitted through the Job Composer. Each record includes path to the job directory, form inputs that were modified in “Job Options” to produce the command line args to qsub/sbatch. The job directories are still there though, so job results would not been lost, only the convenience of access through the Job Composer.

3

That’s helpful. Thank you, sir.

The database repopulation you mentioned - I’d like to request that as a new feature. Is there already a topic that should fall under, or should that be a new one?

Thanks again.

4

So sorry for the late reply. An issue has been opened on the GitHub page to address this request here: Rake task for Job Composer db re-population · Issue #1289 · OSC/ondemand · GitHub