Hi! I’m just exploring whether or not OOD could be used to implement something.
Basically, we have some high energy physics codes that take a while to run - they need a model file, and produce output files that can then be visualised. The academics want masters students to be able to run them without learning much HPC, and so would like a nice web interface for it.
I was wondering how the OOD App paradigm would handle that. You could create a job template that takes a few hours to run… but then visualisation would need to be done separately. You could create an interactive app… but then the examples make that look like it’s a continuous connection kind of thing. Would that be a combo of a Job Template for submitting the analyses, and a Passenger App for tracking and visualising completed jobs? Or is the Interactive App framework flexible enough to have e.g. a “HEP App” that lists previous runs associated with your account and also allows you to submit new ones?
The alternative is a Job Template that pushes the output to a new website built to visualise it, but it’d be much nicer to stay within the OOD ecosystem - I’m just not quite sure where its bounds lie.
This is likely the path forward. Using a template you could have the jobs run, land the data, then use an app for visualization that loads the output files into it’s environment. I don’t quite see a way around that workflow as you are needing a batch job to run, then you need to use the results in some interactive way.
True that it is a continuous connection to the compute nodes, but how else would the data render? I’m not sure why you wouldn’t want some time on a compute to render the image. What is it about running an interactive app in this way that is problematic? Passenger would not be what I’d recommend here as that would run on the web-node itself where OOD sits, not on the compute nodes.
Are you able to use something like Jupyter or Rstudio to pull that output data in and render it in one of those, without having to build a whole new interactive app?
I don’t quite see a way around that workflow as you are needing a batch job to run, then you need to use the results in some interactive way.
Ah, right - so the interactive app framework isn’t suitable for submitted job workflows. That’s something I wasn’t sure about.
True that it is a continuous connection to the compute nodes, but how else would the data render? I’m not sure why you wouldn’t want some time on a compute to render the image.
The outputs don’t actually need to render on HPC - their previous solution had it running on a standard PHP webserver. My impression is it might even be doable entirely user side, e.g. with a Plotly.js figure and the data included in the HTML.
Good point about Jupyter - realistically, making a job template that on completion copies a notebook template and customises it to point at the outputs is probably the easiest way.
Just wanted to add in that fundamentally what you are asking about is a multi-faceted workflow. There are a lot of approaches to these we’ve piloted / seen over the years.
For example, one of our first OOD apps was a welding simulation workflow, where the client could interactively setup the welding parameters (and geometry) all via HTML forms, which then would generate a mesh, then pass that mesh to a multi-physics solver, then compile the outputs into a nicely formatted PDF report.
Another app about a decade ago involved multiple step that the client would trigger / launch via icons on a virtual desktop (i.e. they were first prompted to open the mesh generation app, then initiate the solver, then view the results in a 3D visualization app, all launched from icons on a single desktop)
Finally, one of our largest clients at OSC has numerous active CFD related apps, where the client uploads a geometry, fires off at batch CFD solver, then comes back later and is able to interactively visualize the output in a variety of ways.
Just wanted to summarize some of these to illustrate a bit of the ‘art of the possible here’ (beyond what Travis said about Jupyter and RStudio, which are also great approaches)
Thanks! I think it’d be really useful to have somewhere in the app development documentation a set of typical use cases and the approaches used for them.