Launcher: form.yml - shared cpu, memory, etc. snippet

Get Help
1

As most of my form.yml have the same snippet in form.yml for picking cpu cores, memory, time, etc. I would like to create this in a single place and simply include in the form.yml file.

Where would be the best place in the OOD file system hierarchy to put this?

2

Is the snippet yaml only or does it include erb tags?

3

Most of my launchers are form.yml and yaml only.

4

Maybe under /etc/ood/config/apps/dashboard/batch_connect/partials/ would make sense.

There are two problems with the current rendering of these erb.

  1. We aren’t using Rails views to render, but straight erb so we lose some of the magic for easily including “partials” into the main erb, with auto-indenting
  2. The resulting YAML is being loaded with “safe” load and without enabling anchor support, so we can’t do a trick like including the YAML anchor at the top of the YAML file and then including the snippet in the desired location.

So you can do something like this but it is clunky:

In form.yml.erb:

  - num_cores
  - bc_email_on_started
attributes:
<%= File.read("/etc/ood/config/apps/dashboard/batch_connect/partials/num_cores.yml").indent(2) %>
  bc_account:
    label: "Project"
    help: "You can leave this blank if **not** in multiple projects."
  node_type:

where /etc/ood/config/apps/dashboard/batch_connect/partials/num_cores.yml is

num_cores:
  widget: "number_field"
  label: "Number of cores"
  value: 1
  help: |
    Number of cores on node type (4 GB per core unless requesting whole
    node). Leave blank if requesting full node.
  min: 0
  max: 28
  step: 1
  id: 'num_cores'

or have form.yml.erb be:

  - num_cores
  - bc_email_on_started
attributes:
  num_cores:
<%= File.read("/etc/ood/config/apps/dashboard/batch_connect/partials/num_cores.yml").indent(4) %>
  bc_account:
    label: "Project"
    help: "You can leave this blank if **not** in multiple projects."
  node_type:
  • notice the indent(4)

then have /etc/ood/config/apps/dashboard/batch_connect/partials/num_cores.yml be

widget: "number_field"
label: "Number of cores"
value: 1
help: |
  Number of cores on node type (4 GB per core unless requesting whole
  node). Leave blank if requesting full node.
min: 0
max: 28
step: 1
id: 'num_cores'

So it isn’t pretty erb but it is perhaps functional. The errors are not helpful if the resulting indentation errors are problematic…

5

Thanks for asking this question Shawn. Our clusters are mixed with different specs for different classes of machine, and so this is something that I had been starting to wonder about: providing a centralized hardware configuration that could be re-used by our various Batch Connect applications. Using form.yml as an ERB template seemed like an obvious way to start, but I hadn’t gotten around to trying it yet and so wasn’t aware of the pain points of doing so.

6

I don’t know if I should add on here, or create a new thread. But is there any more information for setting up the backend for cpu cores, memory, … I am trying to add an option for selecting memory to the jupyter app and am getting nowhere fast. Using the above info I have made the forms for it, but I’m not sure how to transform the output into something that will result in a submitted job. I’m using pbspro if anyone wants to be extra specific. :slight_smile:
Thanks,
Aragorn.

7

@shawn.doughty I have a new idea about this that may work better. See my comment to Where to put boilerplate code used in templates

The idea is to just define things in a Ruby hash, and call to_json on the hash, which will return an inline string form of that hash. As long as it is not a nested hash, it appears you can add it as a value to the key value pair in a YAML map. Extra benefits include easy app specific customization using Hash#merge method.