Sharing common header and common attributes

Get Help
1

Hi, we have finally gotten to the point that we have a common header script to be included in all forms for all applications.

<%-
  #Get list of groups user belongs to
  g_cmd = "/usr/local/sbin/user-groups.sh $USER"
  begin
    output, status = Open3.capture2e(g_cmd)
    if status.success?
      accounts = output.split("\n").map(&:strip).reject(&:blank?).sort
      priaccts = accounts.grep(/priority/)
      grpaccts = accounts.grep(/group/)
    else
      raise output
    end
  rescue => e
    queues = []
    error = e.message.strip
  end
-%>

We now also have common attributes for a smart form that allows only certain groups to be selected when choosing a certain partition to send a job to, and sets constraints for the amount of memory and cores based on partition selected as well. The common attributes use arrays produced by the header script. It also hides fields based on selections that aren’t relevant.

Here is our base common.yml.erb file:

# # ~/ondemand/dev/jupyter/common.yml.erb

  # Get the number of CPU's
  num_cores:
    widget: "number_field"
    label: "Number of CPU's"
    help: "<small>Number of CPU's for Jupyter session.</small>"
    value: 4
    min: 2
    max: 256
    step: 1

  # Get the number of GPU's
  num_gpus:
    label: "Number of GPU's"
    widget: "number_field"
    help: "<small>Number of GPU's to allocate for job</small>"
    min: 0
    max: 4
    value: 0
    step: 1

  # Get the number of Memory in GB
  mem_task:
    widget: "number_field"
    label: "Size of RAM in GB"
    help: "<small>The amount of memory to allocate for job</small>"
    value: 4
    min: 4
    max: 1000
    step: 1

  # Get the accounts
  custom_paccount:
    label: "Priority Accounts"
    help: "<small>Select priority account to submit job under</small>"
    widget: select
    options:
    <%- priaccts.each do |a| -%>
      - [ "<%= a %>", "<%= a %>" ]
    <%- end -%>

  custom_gaccount:
    label: "Group Accounts"
    help: "<small>Select group account to submit job under</small>"
    widget: select
    options:
    <%- grpaccts.each do |a| -%>
      - [ "<%= a %>", "<%= a %>" ]
    <%- end -%>

  # Get the GPU type
  # A40 nodes have up to 64 CPUs and 500G
  # A100 nodes have up to 96 CPUs and 500G
  gpu_type:
    label: "NVidia GPU Type"
    help: "<small>VRAM A40 > A100, Compute A100 > A40</small>"
    widget: select
    options:
    - [
      'NVidia A40', 'a40',
       data-max-num-gpus: 2,
       data-max-num-cores : 64,
       data-max-mem-task : 500,
      ]
    - [
      'NVidia A100', 'a100',
       data-max-num-gpus: 4,
       data-max-num-cores : 96,
       data-max-mem-task : 500,
      ]

  enable_pri_queue:
    widget: "check_box"

  custom_queue:
    label: "Partition"
    help: "<small>Select the partition to allocate Jupyter job to.</small>"
    widget: select
    options:
    - [
      'Priority', 'priority',

I’m wondering how accessing arrays generated by the header script from the attributes that are read in. Can somebody please point me in the right direction? I’ll keep digging through the forums for examples.

Kenny

2

Hi, I believe I found my solution in Where to put boilerplate code used in templates - #2 by efranz.

Using classes in a .rb file in initializers should do it.

3

Thank you for finding that. I was looking for that or another similar topic, but yes that’s exactly what you want.

1 Like
4

Hey Jeff,
Have you been able to come across an example for encoding attributes with arrays for options?

  gpu_type:
    label: "NVidia GPU Type"
    help: "<small>VRAM A40 > A100, Compute A100 > A40</small>"
    widget: select
    options:
    - [
      'NVidia A40', 'a40',
       data-max-num-gpus: 2,
       data-max-num-cores : 64,
       data-max-mem-task : 500,
      ]
    - [
      'NVidia A100', 'a100',
       data-max-num-gpus: 4,
       data-max-num-cores : 96,
       data-max-mem-task : 500,
      ]

When I try to set this up in a .rb file using the examples from the link above I get a ton of parsing errors.

  def gpu_type
    {
      label: "NVidia GPU Type",
      help: "<small>VRAM A40 > A100, Compute A100 > A40</small>",
      widget: "select",
      options:,
      - [ 'NVidia A40', 'a40', data-max-num-gpus: 2, data-max-num-cores : 64, data-max-mem-task : 500, ],
      - [ 'NVidia A100', 'a100', data-max-num-gpus: 4, data-max-num-cores : 96, data-max-mem-task : 500, ],
    }
  end
5

This is what you’re looking for.

  def gpu_type
    {
      label: "NVidia GPU Type",
      help: "<small>VRAM A40 > A100, Compute A100 > A40</small>",
      widget: "select",
      options: [
        ['NVidia A40', 'a40', { 'data-max-num-gpus': 2, 'data-max-num-cores': 64, 'data-max-mem-task': 500 }],
        ['NVidia A100', 'a100', { 'data-max-num-gpus': 4, 'data-max-num-cores': 96, 'data-max-mem-task': 500 }]
      ]
    }
  end
1 Like
6

Giving that a try, thanks Jeff! I knew it had something to do with handling an array.

Kenny

7

Hi, I wanted to share our results with rci-tempest.rb staged in /etc/ood/config/apps/dashboard/initializers/rci-tempest.rb. This was a little tricky as we first tried a minimalist approach, but found that no including data-max-num-gpus: 0 and data-set-num-gpus: 0 in non-GPU partitions confused the form and would have error messages show up on hidden fields, effectively disabling the Launch button.

We ran into a weird caching issue during testing each partition. Yesterday the partition I selected was for GPU Priority and set num_cores to 32 and num_gpus to 1 and gpu_type to A100. (the data-max-num-gpus is set to 96 for A100). This job launches as it is within constraints. Today I set to Interactive partition with cpus 32 and mem 16G, which runs fine also. The next time return to the form (by selecting it in my sandbox) the num_cpus gets set to 96 (the data-max-num-cores for A100 GPU), obviously grabbing it from the cache.

We will set data-set-num-cores and data-set-mem-task constraints in each to try and override the cache value being grabbed and post again.

require “open3”
include Open3

module RciTempest
class Accounts
def initialize(user)
output, status = Open3.capture2e(“/usr/local/sbin/user-groups.sh #{user}”)
if status.success?
@accts = output.split(“\n”).map(&:strip).sort
else
raise output
end
end

def get_priaccts()
  @accts.grep(/priority/)
end

def get_grpaccts
  @accts.grep(/group/)
end

end

class Attribs
def num_cores
{
widget: “number_field”,
label: “Number of CPU’s”,
help: “Number of CPU’s for Jupyter session.”,
value: 4,
min: 2,
max: 256,
step: 1,
}
end

def num_gpus
  {
    label: "Number of GPU's",
    widget: "number_field",
    help: "<small>Number of GPU's to allocate for job</small>",
    min: 0,
    max: 4,
    value: 0,
    step: 1,
  }
end

def mem_task
  {
    widget: "number_field",
    label: "Size of RAM in GB",
    help: "<small>The amount of memory to allocate for job</small>",
    value: 4,
    min: 4,
    max: 1000,
    step: 1,
  }
end

def gpu_type
  {
    label: "NVidia GPU Type",
    help: "<small>VRAM A40 > A100, Compute A100 > A40</small>",
    widget: "select",
    options: [
      ['NVidia A40', 'a40', { 'data-max-num-gpus': 2, 'data-max-num-cores': 64, 'data-max-mem-task': 500 }],
      ['NVidia A100', 'a100', { 'data-max-num-gpus': 4, 'data-max-num-cores': 96, 'data-max-mem-task': 500 }]
    ]
  }
end

def custom_queue
  {
    label: "Partition",
    help: "Partition selection.",
    widget: "select",
    options: [
      [ 'Priority', 'priority', { 'data-max-num-cores': 256, 'data-max-mem-task': 1000, 'data-max-bc-num-hours': 336, 'data-min-num-gpus': 0, 'data-set-num-gpus': 0, 'data-hide-num-gpus': true, 'data-hide-gpu-type': true, 'data-hide-custom-gaccount': true, 'data-set-enable-pri-queue': 1 }],
      [ 'GPU Priority', 'gpupriority', { 'data-max-bc-num-hours': 336, 'data-min-num-gpus': 1, 'data-set-num-gpus': 1, 'data-hide-custom-gaccount': true, 'data-set-enable-pri-queue': 1 }],
      [ 'Interactive', 'interactive', { 'data-max-num-cores': 32, 'data-max-mem-task': 250, 'data-max-bc-num-hours': 336, 'data-min-num-gpus': 0, 'data-set-num-gpus': 0, 'data-hide-num-gpus': true, 'data-hide-gpu-type': true, 'data-hide-custom-gaccount': true, 'data-set-enable-pri-queue': 1 }],
      [ 'Unsafe', 'unsafe', { 'data-max-num-cores': 256, 'data-max-mem-task': 1000, 'data-max-bc-num-hours': 168, 'data-min-num-gpus': 0, 'data-set-num-gpus': 0, 'data-hide-num-gpus': true, 'data-hide-gpu-type': true, 'data-hide-custom-paccount': true, 'data-set-enable-pri-queue': 0 }],
      [ 'GPU Unsafe', 'gpuunsafe', { 'data-max-bc-num-hours': 168, 'data-min-num-gpus': 1, 'data-hide-custom-paccount': true, 'data-set-enable-pri-queue': 0 }],
    ]
  }
end

end
end

8

This topic was automatically closed 180 days after the last reply. New replies are no longer allowed.