Run shell script from passenger app

Get Help
1

We have a script that queries squeue and shows the slurm associations and info about the jobs running under their slurm group account.

I’ve followed the passenger tutorial, nodejs version, that generates a ‘Hello World!’ string. I’m aware that node provides an exec() that can run shell commands/scripts. not so sure how to ensure that the shell created to run the command inherits the environment to know the account/user so as to leverage squeue.

The ‘activejobs’ script I thought might be similar, but appears to just expose something intrinsic to ood.

I’m hoping for some guidance on the appropriate ‘framework’ in which to create an app that runs a script in the cluster shell and presents information back on a standalone page, or in the dashboard window. I seem to recall an old tutorial featuring a ‘quota’ app. Perhaps that is no longer applicable. Thanks.

2

The Per User Nginx (PUN) architecture means that this nodejs stack is you, the current user.

You can verify this through os.userInfo().

Though I’m not 100% sure exec will create a shell environment. It’s likely more like a facade for exec linux system call: exec(3) - Linux manual page

3

Which is to say using exec is fine I’m sure, just be sure to capture the output.

4

Thanks, Jeff – I’m beginning to learn about node, and am becoming somewhat optimistic.

Take care
~ Em

5

And followed the tutorial ‘Hello World’ example. When running the app from the sandbox, Hello World appears in the browser window. yay.

i have a script app.js that applies a script /usr/local/bin/si, that queries squeue. this script runs via ‘node app.js’ and generates appropriate output in the shell. It runs in the sandbox, but without terminating.

Thoughts?

1 // app.js
2
3 const express = require(‘express’);
4 const app = express();
5 const port = 3000;
6
7 const { exec } = require(‘child_process’);
8
9 exec(‘sh /usr/local/bin/si’, (error, stdout, stderr) => {
10 if (error) {
11 console.error(Error: ${error.message});
12 return;
13 }
14 if (stderr) {
15 console.error(Stderr: ${stderr});
16 return;
17 }
18 console.log(Output: ${stdout});
19 });
20
21 console.log(Output: We are done here.);

6

NodeJs or Javascript is a nice language, but you have to remember that it’s asynchronous. That’s why they have APIs like execSync for synchronous execution.

I don’t know about “without terminating” my guess is more likely it’s responding to your request before it issues the command.

I’m guessing you see the log message Output: we are done here. before you see the message regarding standard out or standard error. And that’s because the order of execution isn’t synchronous and that’s just how the javascript language works.

7

Thanks, Jeff – I guess my next step is to learn how to get the shell script output into an html format for display? Much to learn still. Sorry for leaning on you as i get off the ground.

But as far as seeing output – i only see output when running on the command line. When running through the sandbox, i see nothing at all. With ‘new_window: false’ in the manifest, the sandbox page does not update. But the browser indicates active, until I select it to stop via ‘x’.

8

Yea if I use that, I get a 502 proxy error eventually. You don’t appear to be responding with anything.

Apparently this line from the starter app is important: It’s likely instructing express to boot up.

app.listen(port, () => {
  console.log(`Example app listening on port ${port}`);
})

Beyond that you’ll get error pages because the app, while it’s up and running, doesn’t know how to respond to any request.

9

Thanks, Jeff – That helps a lot. I’ll take more of a ‘trial and error’ approach for the rest of today ( :

Be well!

10

Some progress! Sorry for the code dump, and yet…
NB: unfortunately, the chat isnt preserving indentation.
// app.js

const express = require(‘express’);
const app = express();
const port = 3000;

const router = express.Router();
app.use(process.env.PASSENGER_BASE_URI || ‘/’, router);
router.get(‘/’, (req, res) => {

var { exec } = require('child_process');
    exec('sh /usr/local/bin/si | head -n 3', (error, stdout, stderr) => {

    if (error) {
        console.error(`Error: ${error.message}`);
        return;
     }
    if (stderr) {
        console.error(`Stderr: ${stderr}`);
        return;
    }
    const siout = stdout.toString();
    console.log(siout);
    res.send(siout);
})

})

app.listen(port, () => {
console.log(Example app listening on port ${port});
})

The output is unformatted:
PARTITION AVAIL_FEATURES AVA TIMELIMIT NODE STATE CPUS(A/I/O/T) NODELIST gpu gpup100 up 13-08:00:0 1 drain* 0/0/20/20 gput038 gpu gpup100 up 13-08:00:0 3 drain 0/0/60/60 gput[031-033]

Placing the ‘exec’ within the router.get seemed to be the key. In the ‘Hello World’ example, the ‘router.send’ is relied on to deliver the message into the /pun/dev/nodejs_info uri.

I’m happy to take hints on formatting the script output. I assume that is something to look into from nodejs itself. Time will tell.

11

At OSC we run a summer institute for high school students. Here’s the program that my team runs developing a web application to use Blender to create animations.

Hopefully it’s enough of a tutorial with documentation to give you a sense of how to build out a web application.

12

Hi, Jeff – Thanks, after reviewing, however, I find this use case does not address some fundamentals. Or it will eventually. I need to focus on the narrow use case of running a command.

There are a variety of methods to run a command/script in the shell. I’ve worked through and have a formulation that can output within a shell when run by ‘ruby app.rb’. The same structure that successfully generated the ‘Hello World’ output does not return output from a command.

cat hw.rb 
require 'sinatra/base'

class App < Sinatra::Base
  get '/' do
    'Hello World'
  end
end

I like what I’ve read about fork – which does return output as indicated above, but only in the ‘ruby app.rb’ environment of a shell – not through Sinatra.

cat app.rb
require 'sinatra'

class App < Sinatra::Base
  get '/' do
    fork { exec("ls") }
    puts "output collected, can we send it back?"
  end 
end

fork { exec("ls") }
puts "output collected, can we send it back?"

The final statement in app.rb would, as i understand, ‘fork’ and ‘exec’ the ‘ls’ when ‘ruby app.rb’; however, the ‘get’ won’t report back, as far as I can tell.

My next thought is to actually study again that bug report for the system status app, that (in part) takes output from Slurm, and formats it to display in the dashboard. That’s exactly my goal. The ruby file for system status in the ood_core might make just a little more sense to me – though I can also imagine that the functionality of the system status app is distributed amongst quite a few files.

Community members – please chime in! Certainly someone has implemented an app that runs a command and reports back to the dashboard ( :

13

Here’s what I’ve settled on. Formatting that appears in the shell is preserved and passed through to the web page. Next step, is to learn how to get this app output to appear in a ‘panel’ inset on the dashboard.

require 'sinatra'
require 'open3'

class App < Sinatra::Base
  get '/' do
    # Path to your bash script
    script_path = '<my_path>'

    # Run the bash script and capture the output
    stdout, stderr, status = Open3.capture3(script_path)

    if status.success?
      # Format the output in HTML
      output_html = "<pre>#{stdout}</pre>"
      erb :output, locals: { output: output_html }
    else
      # Handle errors
      "Error running script: #{stderr}"
    end
    output_html
  end
end

__END__

@@output
<!DOCTYPE html>
<html>
<head>
  <title>Bash Script Output</title>
  <style>
    body { font-family: Arial, sans-serif; }
    pre { background-color: #f4f4f4; padding: 10px; border: 1px solid #ccc; }
  </style>
</head>
<body>
  <h1>Output of the Bash Script</h1>
  <%= output %>
</body>
</html>
14

Sorry, let me rephrase as a question. In addition to including the app in the ‘Jobs’ pulldown menu, how to embed this app, or it’s output, in the dashboard? I’ve reviewed customizations a few times, but don’t see a path forward, except to somehow convert the app to a widget.

Thanks! Although, perhaps I should break this out into another question…?

15

To include another application in the jobs menu, simply use the Jobs category in the manifest.yml when it’s in the production location (development applications do not appear on the navigation bar).

You cannot embed this application’s output in the dashboard (which is itself another application). That is, the dashboard itself is a Passenger app, i.e., an entirely different boot stack with it’s own process tree once it’s booted. You’d have to convert it to a widget.

16

Thanks for the feedback, Jeff. Would you please clarify what you mean by converting a passenger app to a widget for use in OOD? I recognize that in the ondemand.yml, there are the:

- xdmod_widget_job_efficiency

- xdmod_widget_jobs

I would have no idea how to go about this.
Thanks for any pointers – happy to create a new question topic to support this if that’s helpful.

17

The documentation talks about how to add widgets:

You may want to refer to this topic which is similar to this request:

18

Thanks Jeff! – I’ll focus on the custom layouts. And thanks for the pointer to the other topic.
Let’s wrap this one and i’ll initiate further conversation through another topic when the time is right.