Ondemand 1.7.7 Job Composer Bug

Hi All,

We recently had a user report a bug with the job composer. They used job composer to submit a job. After a little bit, they were then unable to get back to the job composer. They would get the error “If you are the application owner check the logs for more information” (/pun/sys/myjobs). "

This was interesting because we had a few people test it and they could submit a job and not get the same error.

Here are the user’s logs:

App 114940 output: [2020-10-05 11:16:10 -0600 ] INFO "method=GET path=/pun/sys/dashboard/apps/show/myjobs format=html controller=AppsController action=show status=302 duration=264.70 view=0.00 location=[https://[hostname_1]/pun/sys/myjobs"](https://[hostname_1]/pun/sys/myjobs%22) 
App 115138 output: Rails Error: Unable to access log file. Please ensure that /opt/<hostname>/var/apps/sys/myjobs/log/production.log exists and is writable (ie, make it writable for user and group: chmod 0664 /opt/<hostname>/var/apps/sys/myjobs/log/production.log). The log level has been raised to WARN and the output directed to STDERR until the problem is fixed. 
App 115138 output: [2020-10-05 11:16:12 -0600 ] DEBUG "\e[1m\e[36mWorkflow Load (2.0ms)\e[0m \e[1m\e[34mSELECT \"workflows\".* FROM \"workflows\" INNER JOIN \"jobs\" ON \"jobs\".\"workflow_id\" = \"workflows\".\"id\" WHERE \"jobs\".\"status\" IN (?, ?, ?, ?)\e[0m [[\"status\", \"H\"], [\"status\", \"Q\"], [\"status\", \"R\"], [\"status\", \"S\"]]" 
App 115138 output: [2020-10-05 11:16:12 -0600 ] DEBUG "\e[1m\e[36mJob Load (2.2ms)\e[0m \e[1m\e[34mSELECT \"jobs\".* FROM \"jobs\" WHERE \"jobs\".\"workflow_id\" IN (?, ?, ?, ?)\e[0m [[\"workflow_id\", 5], [\"workflow_id\", 6], [\"workflow_id\", 7], [\"workflow_id\", 8]]" 
App 115138 output: [2020-10-05 11:16:12 -0600 ] INFO "execve = [{\"PBS_DEFAULT\"=>\"[hostname_2]\", \"PBS_EXEC\"=>\"/opt/pbs\"}, \"/opt/pbs/bin/qstat\", \"-f\", \"-t\", \"2859773.sawtoothpbs\"]" 
App 115138 output: [2020-10-05 11:16:12 -0600 ] INFO "method=GET path=/pun/sys/myjobs/ format=html controller=WorkflowsController action=index status=500 error='ArgumentError: wrong number of arguments (given 1, expected 0)' duration=198.89 view=0.00 db=25.27" 
App 115138 output: [2020-10-05 11:16:12 -0600 ] FATAL "" 
App 115138 output: [2020-10-05 11:16:12 -0600 ] FATAL "ArgumentError (wrong number of arguments (given 1, expected 0)):" 
App 115138 output: [2020-10-05 11:16:12 -0600 ] FATAL "" 
App 115138 output: [2020-10-05 11:16:12 -0600 ] FATAL "app/models/workflow.rb:19:in `each'\napp/models/workflow.rb:19:in `update_status!'\napp/controllers/workflows_controller.rb:224:in `each'\napp/controllers/workflows_controller.rb:224:in `update_jobs'"

Here’s mine that does work:

App 196586 output: [2020-10-06 07:40:04 -0600 ] INFO "method=GET path=/pun/sys/dashboard/apps/show/myjobs format=html controller=AppsController action=show status=302 duration=695.65 view=0.00 location=[https://[hostname_1]/pun/sys/myjobs"](https://[hostname_1]/pun/sys/myjobs%22) 
App 196785 output: Rails Error: Unable to access log file. Please ensure that /opt/<hostname>/var/apps/sys/myjobs/log/production.log exists and is writable (ie, make it writable for user and group: chmod 0664 /opt/<hostname>/var/apps/sys/myjobs/log/production.log). The log level has been raised to WARN and the output directed to STDERR until the problem is fixed. 
App 196785 output: [2020-10-06 07:40:10 -0600 ] DEBUG "\e[1m\e[36mWorkflow Load (3.5ms)\e[0m \e[1mSELECT \"workflows\".* FROM \"workflows\" INNER JOIN \"jobs\" ON \"jobs\".\"workflow_id\" = \"workflows\".\"id\" WHERE \"jobs\".\"status\" IN ('H', 'Q', 'R', 'S')\e[0m" 
App 196785 output: [2020-10-06 07:40:10 -0600 ] DEBUG "\e[1m\e[35mWorkflow Load (4.0ms)\e[0m SELECT \"workflows\".* FROM \"workflows\"" 
App 196785 output: [2020-10-06 07:40:10 -0600 ] INFO "method=GET path=/pun/sys/myjobs/ format=html controller=WorkflowsController action=index status=200 duration=216.85 view=18.74 db=27.93"

It seems like this is the issue:

".\"status\" IN (?, ?, ?, ?)\" - bad
".\"status\" IN ('H', 'Q', 'R', 'S')\" - good

I don’t think the presence of the question marks is the problem. That is just the ActiveRecord “array conditions” Active Record Query Interface — Ruby on Rails Guides. It is odd that the logging differs though. But I’m concerned about ArgumentError: wrong number of arguments (given 1, expected 0) and then

App 115138 output: [2020-10-05 11:16:12 -0600 ] FATAL "app/models/workflow.rb:19:in `each'\napp/models/workflow.rb:19:in `update_status!'\napp/controllers/workflows_controller.rb:224:in `each'\napp/controllers/workflows_controller.rb:224:in `update_jobs'"

I opened an issue Job Composer results in exception after job submission · Issue #887 · OSC/ondemand · GitHub to track this.

This topic was automatically closed 180 days after the last reply. New replies are no longer allowed.