MATLAB sbatch: error: invalid partition specified

Pardon the newbie question but I don’t see in the MATLAB App instructions where to put options for Slurm’s -A/–account. Is that what “Project” is for?

#### Failed to submit session with the following error:

sbatch: error: invalid partition specified: 1 sbatch: error: Batch job submission failed: Invalid partition name specified

Also when would this be useful, I’m guessing there are invalid characters for partition/account names?

* If this job failed to submit because of an invalid job name please ask your administrator to configure OnDemand to set the environment variable OOD_JOB_NAME_ILLEGAL_CHARS.

I would check your /var/log/ondemand-nginx/$USER/error.log for execve commands to see the exact sbatch command you’re issuing.

Yes, bc_account is for the account that you submit with, but the error seems to indicate the issue is with the partition you’re trying to use. Looking at the sbatch command will give us some insight into which partition you’re trying to use.

App 296280 output: [2024-03-27 16:00:31 -0400 ]  INFO "execve = [\"git\", \"describe\", \"--always\", \"--tags\"]"
App 296280 output: [2024-03-27 16:00:31 -0400 ]  INFO "method=GET path=/pun/sys/dashboard/batch_connect/dev/bc_my_center_matlab/session_contexts/new format=html controller=BatchConnect::SessionContextsController action=new status=200 duration=182.17 view=138.62"
App 296280 output: [2024-03-27 16:00:31 -0400 ]  INFO "method=GET path=/pun/sys/dashboard/apps/icon/bc_my_center_matlab/dev/me format=html controller=AppsController action=icon status=200 duration=8.41 view=0.00"
App 296280 output: [2024-03-27 16:00:34 -0400 ]  INFO "execve = [{\"SLURM_CONF\"=>\"/cm/shared/apps/slurm/var/etc/slurm.conf\"}, \"/cm/shared/apps/slurm/current/bin/sbatch\", \"-D\", \"/path/to/me/ondemand/data/sys/dashboard/batch_connect/dev/bc_my_center_matlab/output/d3eb14bc-31eb-48db-9947-e37cb422cfef\", \"-J\", \"sys/dashboard/dev/bc_my_center_matlab\", \"-o\", \"/path/to/me/ondemand/data/sys/dashboard/batch_connect/dev/bc_my_center_matlab/output/d3eb14bc-31eb-48db-9947-e37cb422cfef/output.log\", \"-A\", \"\", \"-t\", \"01:00:00\", \"--export\", \"NONE\", \"--nodes\", \"1\", \"--ntasks-per-node\", \"1\", \"--parsable\", \"-M\", \"terremoto\"]"
App 296280 output: [2024-03-27 16:00:34 -0400 ] ERROR "ERROR: OodCore::JobAdapterError - sbatch: error: invalid partition specified: 1\nsbatch: error: Batch job submission failed: Invalid partition name specified"
App 296280 output: [2024-03-27 16:00:34 -0400 ]  INFO "execve = [\"git\", \"describe\", \"--always\", \"--tags\"]"
App 296280 output: [2024-03-27 16:00:34 -0400 ]  INFO "method=POST path=/pun/sys/dashboard/batch_connect/dev/bc_my_center_matlab/session_contexts format=html controller=BatchConnect::SessionContextsController action=create status=200 duration=217.88 view=13.08"
[ N 2024-03-27 16:05:34.1025 296216/T4 age/Cor/CoreMain.cpp:1147 ]: Checking whether to disconnect long-running connections for process 296280, application /var/www/ood/apps/sys/dashboard (production)
App 300976 output: [2024-03-27 16:17:51 -0400 ] ERROR "ERROR: NoMethodError - undefined method `[]' for nil:NilClass"
App 300976 output: [2024-03-27 16:17:51 -0400 ]  INFO "execve = [\"git\", \"describe\", \"--always\", \"--tags\"]"
App 300976 output: [2024-03-27 16:17:51 -0400 ]  INFO "method=POST path=/pun/sys/dashboard/batch_connect/dev/bc_my_center_matlab/session_contexts format=html controller=BatchConnect::SessionContextsController action=create status=200 duration=250.55 view=158.44"
App 300976 output: [2024-03-27 16:17:54 -0400 ] ERROR "ERROR: NoMethodError - undefined method `[]' for nil:NilClass"

Also seeing:
This app does not supply a sub app form file under the directory '/var/www/ood/apps/dev/me/gateway/bc_my_center_matlab/local'

\"-A\", \"\", – the account you submit with appears to be an empty string. It seems like you should fill this out when you submit the app or remove it if you don’t need to specify the account.

Yes but that Project drop down is empty how does one populate that?

Should be a text field - can you share your form.yml?

Sure I’ve kept the defaults so I still have all the Owens/Pitzer options.

form.yml.txt (4.9 KB)

I see - yes you’re using our accounts - or what we’d expect our accounts to look like. you see here the groups array is all of the groups you belong to that match the regular expression /^P./ (which is what OSC accounts look like).

<%-\
  groups = OodSupport::User.new.groups.sort_by(&:id).tap \{ |groups|\
    groups.unshift(groups.delete(OodSupport::Process.group))\
  \}.map(&:name).grep(/^P./)\
-%>\

  account:\
    label: "Project"\
    widget: select\
    options:\
      <%- groups.each do |group| %>\
      - "<%= group %>"\
      <%- end %>\

You have a couple options - remove all of this and use bc_account for a text field OR you can use auto_accounts if you’re on Slurm OR you can modify the grep expression to search for other groups you may be a part of.

Note that if you change account in the form - you’ll need to remove it from the submit.yml.erb file. bc_account and auto_accounts know how to submit themselves, so there’s no need for them in submit.yml.erb and you’ll keep getting empty strings if you keep account set there.

Yes this would be great! Are there instructions for this?

It’s here. Just replace account with auto_accounts in the form.yml.erb and get rid of all the references to account (specifically in the form and submit files).

https://osc.github.io/ood-documentation/latest/how-tos/app-development/interactive/form.html?highlight=auto_accounts#automatic-predefined-attributes

In submit.yml.erb I only see:

script:
  accounting_id: "<%= account %>"
  native:

So just delete ‘account’?

And in form.yml.erb I only see the one instance of account here:

  account:
    label: "Project"

It also mentions to enable Dynamic Form Widgets, but I don’t see a form.js file, since we upgraded from 1.7 up to 3.0, does it need to be manually created?

Yes delete all instances of account if you’re replacing it with auto_accounts.

The docs say

Prior to version 2.0, sites would have to make their own custom form.js to add dynamic behavior for batch connect forms. While sites can still have their own custom javascript, 2.0 added out of the box support for common use cases based on configuration.

so no you don’t need your own form.js if if you can configure the dynamic behavior you want in YAML. It seems like you’re just starting - so don’t add it until you find that you need to do something that we don’t support out of the box.

Also - thinking about this a little more - if you’re upgrading from 1.7, then your 1.7 application for Matlab should still work. All of these additions we’re talking about here are just that - additions. Additional things you could use like auto_accounts instead of supplying a list of valid accounts by hand. But even the hand built list in your 1.7 app will still work.

No difference after making the 2 changes,

Failed to submit session with the following error:
sbatch: error: invalid partition specified: 1
sbatch: error: Batch job submission failed: Invalid partition name specified

This looks like the value ‘1’ is being passed to the -A option.

Oh, forgot to mention that we never had any apps configured in the older version. Just the ability to see your files and submit jobs. So we’d like to add some apps like Matlab, Jupyter Notebooks, RStudio, etc.

Would need to know how to do that. Also I was referencing this instruction:

auto_accounts will generate cluster aware lists if you have dynamic options enabled. This means it will only show a list of accounts available for the cluster that’s currently selected. If this setting is not enabled, it will generate a list of all accounts available on all clusters and will not hide any of them.

That implies that dynamic options needs to be enabled.

It’s a configuration if you follow that link. At OSC all our accounts work on all our clusters, but if you have some accounts that work for some clusters and so on, if you add that configuration then it’ll only show the accounts for the cluster that you’ve selected. so it’ll automatically toggle accounts that don’tt exist on the cluster that you’ve selected.

Again, you can find the exact sbatch command being issued in your log file. I would ask if your account selection list is actually populated and if you did in fact remove the account reference in the submit.yml.erb.

App 264580 output: [2024-03-29 12:53:57 -0400 ]  INFO "execve = [{\"SLURM_CONF\"=>\"/cm/shared/apps/slurm/var/etc/slurm.conf\"}, \"/cm/shared/apps/slurm/current/bin/sbatch\", \"-D\", \"/moto/home/rk3199/ondemand/data/sys/dashboard/batch_connect/dev/bc_my_center_matlab/output/51d023a9-2f45-4b46-ac03-d419df0559c4\", \"-J\", \"sys/dashboard/dev/bc_my_center_matlab\", \"-o\", \"/moto/home/rk3199/ondemand/data/sys/dashboard/batch_connect/dev/bc_my_center_matlab/output/51d023a9-2f45-4b46-ac03-d419df0559c4/output.log\", \"-A\", \"\", \"-t\", \"01:00:00\", \"--export\", \"NONE\", \"--nodes\", \"1\", \"--ntasks-per-node\", \"1\", \"--parsable\", \"-M\", \"terremoto\"]"
App 264580 output: [2024-03-29 12:53:57 -0400 ] ERROR "ERROR: OodCore::JobAdapterError - sbatch: error: invalid partition specified: 1\nsbatch: error: Batch job submission failed: Invalid partition name specified"
App 264580 output: [2024-03-29 12:53:57 -0400 ]  INFO "execve = [\"git\", \"describe\", \"--always\", \"--tags\"]"

No it’s not. I type in my account name as the field now allows entering a value.

Indeed I did:

script:
  accounting_id: "<%=  %>"
  native:
  <%- slurm_args.each do |arg| %>
    - "<%= arg %>"
  <%- end %>

This is why it’s blank. auto_accounts and bc_account already know how to submit the account, so if you’re using one of those you don’t need this line at all.

If you’re using a user defined name like my_new_account_field, then you need to set it here.

accounting_id: "<%= my_new_account_field %>"

That’s why we provide auto_accounts and bc_account fields, so that you don’t have to provide them in the submit.yml - they do this automatically. On the other hand - if you define a completely new field - then you need to use it in submit.yml.