Sharing common code among multiple forms

Hi,

I’m fairly new to OOD and am working with an established system. I’m not very familiar with ruby and could use some help.

We have a block of code which we need to include in the form.yml.erb file for several different apps. I would like to keep that common code in one file which would then be referenced by each form.yml.erb that needs it.

I gather that ERB.new can be used for this, but I have tried a number of variations and have not been able to get it working. Could you please provide an example to follow?

Suppose, for example, the following block works as expected when included in the form.yml.erb file for some app.

<%
    if not ENV['MY_ENV_VAR'].nil?
      my_value = ENV['MY_ENV_VAR'].split(':').reject(&:blank?).sort
    end
%>

The my_value variable is then referenced elsewhere in the form.

I put those 3 lines of code into a separate file in the same directory. Then, I replaced the block with the following:

<% ERB.new(File.read('/full/path/to/the/right/spot/common.yml.erb'), nil, nil).result(binding) %>

That results in an error like this:
NameError: undefined local variable or method my_value’ for #ERB:0x000055bfa3aeb2b8`

Naturally, I tried defining my_value explicitly in form.yml.erb just before the ERB.new statement. That results in this error:

NoMethodError: undefined method to_h’ for #String:0x000055e7c7b64450 Did you mean? to_d to_r to_f to_i to_s to_c`

What’s the right way to do this?

Thanks for your help!

Chris

The way I was able to get this to work was to have the code in the common.yml.erb like you have, but then in the form.yml.erb at the top I did this:

<%-
  ...
  my_value = nil
  ERB.new(File.read('/path/to/common.yml.erb')).result(instance_eval{binding})
%>
...

This made my variable accessible.

The second error you ran into I would need to see the rest of the code to understand. Either what was returned is an incorrect type due to not having the instance_eval and therefore a String is having to_h called on it which won’t work as that method is not defined for that type, or there’s just some ruby code doing something incorrect on the result.

If you could post the rest of the form.yml.erb I may be able to see what is wrong.

Travis,

Thanks for that, it definitely solves the first problem.

Below, I’ve included a simplified version of my form.yml.erb which gives the same NoMethod error. I’m also including a simpler common.yml.erb.

My common.yml.erb file now contains just this one line:
my_value = [ 'thing1', 'thing2' ]

The specific error I see using this file is:
NoMethodError: undefined method to_h’ for “my_value = [ ‘thing1’, ‘thing2’ ]”:String Did you mean? to_d to_r to_f to_i to_s to_c`

And, form.yml.erb:

<%-
my_value = nil
ERB.new(File.read('/path/to/common.yml.erb')).result(instance_eval{binding})
%>

---

cluster: "my-cluster"

form:
  - ood_home_var
<%- if ! my_value.nil? -%>
  - ood_my_value
<%- end -%>

attributes:
  ood_home_var:
    label: "Home directory for Jupyter"
    value: "${HOME}/home_jupyter"
    help: "You can leave the default value if you are not sure."
    id: "ood_home_var_id"

<%- if ! my_value.blank? -%>
  # this bit based on the example at https://osc.github.io/ood-documentation/release-1.3/app-development/tutorials-interactive-apps/add-custom-queue/local-dynamic-list.html?highlight=form%20yml%20erb
  ood_my_value:
    label: My Value
    help: |
      No help.
    widget: select
    options:
    <%- my_value.each do |q| -%>
      - [ "<%= q %>", "<%= q %>" ]
    <%- end -%>
<%- end -%>

Ok, I think I see part of the issue. When ERB.new is invoked, it is returning a string initially. So, the array is not an array at that point, so the to_h isn’t available. This might start to get a bit hacky and unwieldy the more I look at what you’ll have to do here.

I think this can get you the array you want, but we may need to tweak this some:

<%-
my_value = nil
erb_result = ERB.new(File.read('/path/to/common.yml.erb')).result(instance_eval{binding})
my_value = erb_result.split("=")   # remove the '=' and return an array with remaining elements
                      .last               # remove the head, 'my_value' here, 
                      .strip[1..-2]       # strip the brackets, first and second to last positions
                      .split(",")         # Remove the comma
                      .map(&:strip)       # map will return an array, and each element of array has strip applied
                      .map { |s| s[1..-2] } # now take that array, and map it to a new one with the brackets
%>

This should all result in the array you expect of ['thing1', 'thing2'] to be used later in the code and should respond correctly, though we may still need some tweaking.

Hopefully this code helps explain some rubyism at least and gives you some more clues, but let me know if the error changes and what to if so.

Yes, that explains very nicely what’s going on. Thanks for sharing your ruby expertise!

Unfortunately, the situation is a bit more complex and that probably makes the result parsing solution more trouble than it’s worth.

The actual common code I’m using defines multiple list variables and may become more complex over time. That’s part of the reason why I wanted to investigate some sort of inclusion mechanism.

It’s surprising that there isn’t some simple, standard way of reusing code in this context. I was half expecting you to come back with an answer saying "don’t use ERB.new, do this instead’. I can live with it, though.

In fact, now I’m considering whether we might keep the common code in a single file in the repo, and then use ansible to inject that code into destination file templates on deployment.

Thanks again for your help!

Chris

Let me make sure I also understand the idea correctly, and please correct me where I go off track.

An example would be these 2 apps below, where each app has a form.yml.erb that uses a groups list to populate some fields.

And you are looking for a way to template out that groups code to each file and call it somehow to populate the code. Or is this off the mark?

I don’t know how to really do that using ERB off the top of my head without playing with it more, and I’m not aware of a standard way in OOD to do this.

It may just be some of the OOD Gems will provide what you are after? Ood Support has docs that you may find useful:

https://rubygems.org/gems/ood_support

Is there a pattern to what it is you were hoping to factor out across these forms?

Travis,

Yes, that’s exactly what I’m trying to do. I’m not particular about whether it happens via the ERB class or not, it’s just the most likely thing I found when I went looking for a solution.

I’ll look into the gems doc, that’s a good suggestion. Though, I suspect it’s not something that’s specific to OOD. The same use case should be relevant for any application using erb templating.

Anyways, I learned something here and that’s a good outcome in any case!

Chris

Maybe defining all this complex stuff in an initializer suites your needs? You can define entire classes with functions then call those classes and methods in your apps.

That said - in 3.0 we supply auto_accounts and auto_groups both. (along with auto_queues and auto_modules`).

Hi Jeff,

Thank you, that’s the solution I need!

I had earlier tried defining a bare variable in the initializer but found I couldn’t reference it in the form. So, I went looking for other solutions.

Now, I see that wrapping the same code in a class definition works just fine. And, it certainly improves the organization of the code overall.

Thanks again - I’m looking forward to having time to jump into 3.0!

Chris

I thought I should circle back and share my results. I leveraged the knowledge shared by both Jeff and Travis to condense the form changes down to just two additional lines. All the rest of the functionality is encapsulated in the initializer.

I should back up a bit and explain the objective. We wanted to add some dynamic elements to some of our app forms to show a list of SLA and queue choices based on the user’s entitlements. Further, the default SLA selection should be blank and the queue selection widget should remain hidden until an SLA is specified. If the user has no entitlements, these elements should not appear at all in the rendered form.

In addition, the form.yml.erb files are shared among multiple teams and not all of them use SLA’s. We wanted to minimize the impact of this feature on the form files to ease maintenance, etc. We are looking forward to the new features in 3.0 which should help with this config sharing, but it will likely take some time before we see that in production.

With the initializer code we have now, we just need to add one line to the form section and one line to the attributes section of the form.yml files.

Like this:

form:
...
 <%= UserOptions.sla_form_elements %>
...
attributes:
...
<%= UserOptions.sla_widgets %>
...

The initializer code is below. I have a software background, but I’m not really familiar with ruby. I’ll be happy to hear any suggestions for improving the code.

Thanks again for your help with this!

Chris





# this initializer defines the UserOptions class, which can be referenced by app forms to populate
# UI elements for sla and queue selection (for those users who have access to group-specific slas).

# for details about how dynamic form elements work, see the example at https://osc.github.io/ood-documentation/release-1.3/app-development/tutorials-interactive-apps/add-custom-queue/local-dynamic-list.html?highlight=form%20yml%20erb

class UserOptions
    # define some paths
    @@bsla_path = "/path/to/bin/bsla"
    @@group_config_dir = "/path/to/groups"
    @@lsf_envdir = '/path/to/lsf/conf'

    # hard-coded list of queue choices
    @@queues = [ 'short', 'long' ]

    # text added to the 'form' section of the form yml file
    @@form_elements = <<-EOF
  - ood_user_sla
  - ood_user_queue
EOF

    # first part of sla selection widget definition
    @@sla_widget_preamble = <<-EOF
  ood_user_sla:
    label: SLA
    help: |
      Select an SLA from the drop-down.
    widget: select
    options:
EOF

    # when a user has access to group-specific slas, the default sla selection widget appears in the form and the default selection
    # is empty, i.e. no sla. while there is no sla selected, the queue selection widget is hidden.
    @@sla_widget_empty_option = <<-EOF
      - [ "", "none", data-hide-ood-user-queue: true ]
EOF

    # template for populating selection widget options
    @@generic_widget_option = <<-'EOF'
      - [ "%{option_text}", "%{option_value}" ]
EOF

    # first part of queue selection widget definition
    @@queue_widget_preamble = <<-EOF
  ood_user_queue:
    label: Queue
    help: |
      Select a queue from the drop-down.
    widget: select
    value: "short"
    options:
EOF

    # use bsla to fetch the list of all sla's defined by the system config
    def self.system_slas
        @system_slas ||= begin
            slas = %x{LSF_ENVDIR=#{@@lsf_envdir} #{@@bsla_path} | /usr/bin/awk 'match(\$0, /SERVICE CLASS NAME:\\s*(\\S+_\\S+)$/, m) {print m[1]}'}
            if $?.exitstatus != 0
                puts('Non-zero exit code from sla fetch command pipeline: #{$?.exitstatus}')
                return
            end
            @system_slas = slas.split("\n")
        end
        @system_slas
    end

    # search group config files to fetch the list of sla's available to the current user
    def self.user_slas
        @user_slas ||= begin
            uslas = []

            user = ENV['USER']
            for sla in system_slas
                group_file_name = "user_#{sla}"
                group_file = File.join(@@group_config_dir, group_file_name)
                %x{/bin/grep #{user} #{group_file}}
                if $?.exitstatus == 0
                    uslas.push(sla)
                end
            end

            @user_slas = uslas
        end
        @user_slas
    end

    # return the yml 'form' section definitions appropriate to the current user's sla access
    def self.sla_form_elements
        @sla_form_elements ||= begin
            if ! user_slas.to_a.empty?
                template = @@form_elements

                ERB.new(template, trim_mode: nil, eoutvar: "@sla_form_elements").result binding
            end
            @sla_form_elements
        end
        @sla_form_elements
    end

    # return the yml 'attibutes' section definitions appropriate to the current user's sla access
    def self.sla_widgets
        @sla_widgets ||= begin
            if ! user_slas.to_a.empty?
                template = @@sla_widget_preamble

                # populate the selection options, including a leading blank selection
                template += @@sla_widget_empty_option
                user_slas.each do |item|
                    value_hash = {
                        option_text: item,
                        option_value: item
                    }
                    template += sprintf(@@generic_widget_option, value_hash)
                end

                template += "\n"

                template += @@queue_widget_preamble

                @@queues.each do |item|
                    value_hash = {
                        option_text: item,
                        option_value: item
                    }
                    template += sprintf(@@generic_widget_option, value_hash)
                end

                ERB.new(template, trim_mode: nil, eoutvar: "@sla_widgets").result binding
            end
        end
        @sla_widgets
    end
end

# calling these here initializes all the instance variables on load
UserOptions.sla_form_elements
UserOptions.sla_widgets
1 Like

@codecat555 Thanks for posting the working config you ended up with!

The solution we landed at, for NMSU, was a bit different but we also have a simpler use case. For us we wanted to share common code for form.yml.erb, submit.yml.erb, form.js, and some json files we use to define partition attributes (max values for form fields and such).

We combined all of our OOD apps into a single monorepo (NMSU_HPC / ood-apps-nmsu · GitLab) and used a templates directory to contain shared code. For each app we create 2 symlinks, a folder called shared (relative symlink to ../templates) and for apps that need it form.js (form.js.erb is ignored by OOD so we can’t use ruby here for sharing). Then inside our application erb files we use a combination of ERB.new, YAML.load, and YAML.dump to import and process our shared erb files.

One other optimization we made was removing the per-app Slurm partition queries to a OOD Dashboard initializer. Specifically as a global ruby variable and not a class or function. We wanted to ensure we only queried slurm once occasionally instead of 8+ times (number of OOD apps) every second. Not sure if it’s a bug but we have noticed that navigating between pages in OOD causes all interactive apps to recompile the form.yml.erb of every application. There is another bug that causes this recompile behavior to happen every few seconds if there is an running app the user is connected to. Moving our Slurm queries out of our apps and into a dashboard initializer reduced a lot of load on our Slurm head controller.