Need some help with custom submission and custom script files together with bc_desktop app

[novosirj (Ryan Novosielski)]

Hi there. New to maintaining an Open OnDemand instance (we’ve currently got 3).

The person who did our initial install did something similar to what the poster here did. modifying files in /var/www/ood/apps to make customizations:

OOD update from 1.5 to 1.6 experience

Hi Ric, I have modified the form.yml and submit.yml.erb in the bc_desktop to add core count, so, I am not sure if I can modify it in the /etc/ood/config/apps/bc_desktop/*.yml cluster files. I’ll ask the OOD guys at PEARC about it. MC

…which bit me on upgrade is it did the poster in the other thread. I’ve since seen that what should have been done is for /etc/ood/config/apps to be where customizations should be made. To that end, I’ve added a “submit: submit/submit.yml.erb” to the various /etc/ood/config/apps/bc_desktop/*.yml. That works fine. Where I’m running into trouble is then also attempting to customize/replace the operations in /var/www/ood/apps/sys/bc_desktop/template/script.sh.erb using files in /etc. My most recent attempt looked like this:

---
batch_connect:
  template: basic
  script_file: ./script.sh.erb

…which I’ve also tried using an absolute PATH to /etc/ood/config/apps/bc_desktop/submit or a couple of other permutations. I also initially tried “template: vnc” – which is what I really want – but looking at the documentation for the template files made it look possible that script_file wasn’t supported by the vnc template. In any case, the end result is that the “output” directory for the session in my home directory has script.sh.erb.original and script.sh, but no sign of my customizations. What does work is modifying /var/www/ood/apps/sys/bc_desktop_oarc/template/script.sh.erb, so I’m looking for whatever the equivalent would be.

Can someone:

1. Help me with an example of how to have both a custom submit.yml.erb and a custom script.sh.erb?
2. Point me what log files to look in for a failure at this step?

Currently what happens is that it starts to spawn a job and the job disappears, likely since the custom script.sh.erb Is not being run, which is required to start NoVNC in our environment.

Thanks in advance for any tips.

[jeff.ohrstrom]

@novosirj

What does work is modifying /var/www/ood/apps/sys/bc_desktop_oarc/template/script.sh.erb, so I’m looking for whatever the equivalent would be.

I think copying it is the only way to go. We setup and ship bc_desktop under the assumption it’s OK. You could try script_wrapper if it’s only one or two things but if it’s a very large override, then copying is the only option.

It looks like you’ve figured out desktops but here’s our configuration for ondemand.osc.edu. We drop these folders into /etc/ood/config to make an /etc/ood/config/apps directory. We literally drop this folder there directly without any post-processing, so I think that’s a good reference point. Even if you bc_desktop_oarc you’ll probably want to deploy files like these to /etc/ood/config/apps/bc_desktop_oarc.

I can’t tell if you want to boot a desktop (or different types of desktops like mate or xfce) or you want to boot an application. The bc_desktop app is a convenience application quickly deploying desktops with different configurations (resoure allocations or into different clusters and so on). Where we’ve setup the submit.sh.erb scripts because they’re fairly generic and all we want is to start an xfce session for example.

@novosirj

Help me with an example of how to have both a custom submit.yml.erb and a custom script.sh.erb?

If you want to boot an application - over VNC - then you can reference how we deploy matlab. But when we deploy it, it’s a seperate app. That is, we deploy that folder to /var/www/ood/apps/sys/bc_osc_matlab and it’s seperate (folder and application) from bc_desktop. And even though you boot a full desktop environment, it’s really only meant to look like you’ve booted matlab.

Is this what you’re looking for? Can you help me understand what you’re end goal is? What addtional functionality you’re trying to get out of bc_desktop?

@novosirj

Point me what log files to look in for a failure at this step?

Logs will appear in a directory like this: /users/PZS0714/johrstrom/ondemand/data/sys/dashboard/batch_connect/sys/bc_desktop/vdi-owens/output/f7dd10e8-0d27-4ceb-96df-782cc5248443

If I make some of those bits variable, it would be this. UUID is generated at runtime and completely dynamic for every submission, and APP_NAME is what you call your app.
$HOME/ondemand/data/sys/dashboard/batch_connect/sys/bc_desktop/$APP_NAME/output/$UUID. But that’s for the actual job submission, I’m guessing you want logs for the actual job creation which could be in /var/log/ondemand-nginx/$USER, though there may or may not be anything helpful here.

[efranz]

What modifications to the script itself do you want to make?

[novosirj (Ryan Novosielski)]

Basically all we need to do is to change the following line in the bc_desktop/template/script.sh.erb:

source "<%= session.staged_root.join("desktops", "#{context.desktop}.sh") %>"

…and instead do something like this:

singularity exec ... image.img "<%= session.staged_root.join("desktops", "#{context.desktop}.sh") %>"

script_wrapper seems maybe to be a little too far upstream, but there may be some solution I’m not seeing that one of you might.

@jeff.ohrstrom

I can’t tell if you want to boot a desktop (or different types of desktops like mate or xfce) or you want to boot an application. The bc_desktop app is a convenience application quickly deploying desktops with different configurations (resoure allocations or into different clusters and so on). Where we’ve setup the submit.sh.erb scripts because they’re fairly generic and all we want is to start an xfce session for example.

We want to run a desktop, and by and large the existing options are fine, but need some changes because we don’t have some of the dependencies on all of our nodes, and therefore need to point to alternate locations for some of the dependencies (like VNC and websockify) and use Singularity to run an image that contains the desktop group, etc. The alternate location stuff should all be doable via environment variables set in the various YAML files, but the one command I’m not sure how to handle.

@jeff.ohrstrom

If I make some of those bits variable, it would be this. UUID is generated at runtime and completely dynamic for every submission, and APP_NAME is what you call your app.
$HOME/ondemand/data/sys/dashboard/batch_connect/sys/bc_desktop/$APP_NAME/output/$UUID . But that’s for the actual job submission, I’m guessing you want logs for the actual job creation which could be in /var/log/ondemand-nginx/$USER , though there may or may not be anything helpful here.

Had discovered both of those locations already; unfortunately nothing related to the job dying during creation in either one of them, but it sounds like what I was trying to do is not achievable via the method I was trying, so that might be moot.

[efranz]

I think @jeff.ohrstrom’s recommendation is correct. Recursively copy /var/www/ood/apps/sys/bc_desktop to /var/www/ood/apps/sys/bc_desktop_oarc, then modify your version. The “overrides” in /etc/ will go under /etc/ood/config/apps/bc_desktop_oarc since thats the convention - the overrides directory name matches the app directory name. We will definitely consider this case in a future update to the plugin architecture.

[novosirj (Ryan Novosielski)]

But just to be clear, it’s not possible in any way to just override script.sh.erb by putting something in /etc/ood/config/apps/bc_desktop someplace?

[jeff.ohrstrom]

That’s correct, though I think we’ll probably want to add some sort of feature to do this.

[novosirj (Ryan Novosielski)]

It wasn’t posted to this question, but @jeff.ohrstrom since created a GitHub issue for this enhancement:

BC Desktop support for singularity

[jeff.ohrstrom]

@novosirj - Sorry it took so long to figure out that we could indeed use script_wrapper for your use case .

In any case, you’re on that topic and the github issue as well, but I’ve able to demonstrate how this works in this can works in the topic below in case anyone sees this in the future.

Mate Desktop in a Singularity container?

OK, I think I’ve solved this (finally ). The first issue you’ve had, turns out the easiest thing to do is just to remove mate-power-manager. Users shouldn’t be able to restart the node anyhow which I think is that widget does. The second is something we’d already seen apparently and I just forgot about it. So, if you’ll notice in the definition file, there’s a different RPM url. I was not able to solve the issue with turbovnc 2.1.1. It says you need the openssl or gnutls librarie…

[novosirj (Ryan Novosielski)]

Hey, listen, I’m happy to receive good news on any timetable, and thanks a lot for providing this software in the first place.

[novosirj (Ryan Novosielski)]

@jeff.ohrstrom I’ve realized that this does some of what I asked, but not all of it.

Correct me if I’m misunderstanding, but I think this still means that I have to edit some files in /var/www/ood/apps/sys/bc_desktop, or make a copy and modify it, right?

Ideally, I’d be able to make the changes somehow in /etc/ood/config/apps/bc_desktop, so they wouldn’t be overwritten each time/I wouldn’t have to merge files each time to override portions of the behavior.

Also since we’re already talking here, despite my bc_desktop_container app being a copy of the current bc_desktop app, which is 0.2.2, when I go to launch the app, it says 0.2.0, which I assume it was at some point – is there a cache I need to clear? Thanks!

[jeff.ohrstrom]

@novosirj

Ideally, I’d be able to make the changes somehow in /etc/ood/config/apps/bc_desktop

This is absolutely what you should do. I’ve update my answer in the other topic. Your intuition is right, you shouldn’t be modifying the files in /var/www/ood, but I’m not advocating that.

Here are the files you need.

# this is similar to any form you currently have. 
# Only you have to specify `submit: submit/container.yml.erb` (the file below)
/etc/ood/config/apps/bc_desktop/mate_container.yml

# I've provided this in the other topic
/etc/ood/config/apps/bc_desktop/submit/contianer.yml.erb

To the versions, I wouldn’t worry about it, it’s likely a misprint or some issue in not copying the .git files too.