XDMoD integration feedback

We have connected our XDMoD to our test OOD instance that we moved to 1.8, and I have some questions and feedback.

First of all, things work as expected, that is we see the summary at the main dashboard and correct links to the XDMoD job stats from the finished Job Composer jobs. So that it great.

Also, authentication ended up being simpler than we thought, at least for now. We use Keycloak tied to campus AD in OOD, and simplesamlphp tied to campus AD in XDMoD. We do have to separately authenticate to both but once we do the XDMoD stuff does get displayed in OOD. We’ll be thinking on how to integrate the sign on.

Now a few questions.

  1. The OOD main dashboard efficiency reports data look off to me, can you please share how are they requested from XDMoD? So I can get the same data displayed on the XDMoD server to check if what OOD shows is correct.

  2. Is it possible to customize the XDMoD panels on the dashboard? For now it’d be enough to adjust the time period and be able to limit the reports to a single user or a single group.

  3. On a related note, I suppose there’s not a way to change location of the panels on the main dashboard now, right? Our layout is a bit awkward now with the quota warnings on the top and no MOTDs so there’s a lot of empty space on the left side of the dashboard.

  4. XDMoD links are not available in the Active Jobs. Is that an intent for now? If it is, any plans to make it available. I think that would be very beneficial, and was one of the things that I was looking forward to with the XDMoD integration.

Other than that it’s looking great, thanks for getting this in.

MC

1 Like

Both of the widgets are https://github.com/OSC/ondemand/blob/801ec0d4fe000c3e4ec962d3f253b06ae50d1948/apps/dashboard/app/views/dashboard/_xdmod_widget_jobs.html.erb and https://github.com/OSC/ondemand/blob/801ec0d4fe000c3e4ec962d3f253b06ae50d1948/apps/dashboard/app/views/dashboard/_xdmod_widget_job_efficiency.html.erb. They are crafted to produce the same requests (or very similar) to what is sent from the XDMoD user dashboard:

For locations of the panels - that is a cosmetic defect I have written up as an issue to address in a 1.8 patch release.

XDMoD links are not available in Active Jobs because:

  1. XDMoD is for completed jobs, not running or queued jobs
  2. If your site, like ours, runs the XDMoD ingest and aggregatation script once a day, links from Active Jobs to newly created jobs would be broken until this aggregation was run. After this aggregation is run, completed jobs likely would not be reported in Active Jobs.

Do you have a different experience or have XDMoD setup to run the aggregation more often? One of the improvements we had discussed the possibility of was running the ingest script (without aggregation) every 5 minutes or so in order that newly completed jobs would be included in the XDMoD API responses. Then we could present all the completed jobs for the past week (or last 200 or 1000 or whatever threshold we wanted to present" of completed jobs in OnDemand. That would then go in either a separate completed jobs app, or be merged with Active Jobs into a renamed “My Jobs” app.

This would not include adding completed jobs listing for all users in OnDemand, however, like we currently optionally show in Active Jobs for all running or queued jobs.

As for adjusting the panels are you imagining setting an env var to specify the time period that all users see? As for “be able to limit the reports to a single user or a single group”, are you seeing multiple users jobs in the reports - that might be due to your admin status. I thought I had fixed that (but maybe not) but there is a way to ensure that for admins you only see your jobs. Regular XDMoD users will see reports only pertaining to their own jobs.

As an XDMoD admin, if you want to vary whose jobs that you see - instead of just your own (since you could actually see everyone) that would be interesting. I’d need to look into that to see what it would take.

Thanks Eric,

our XDMoD admin just noted that I am an admin there, which is why OOD showed more jobs than just mine. He made me an user and only my jobs show, so, that’s good, and good that you confirmed that. Some kind of option for that would be useful, though. I do look at other people’s XDMoD data during troubleshooting so I need an admin there, but, I’d prefer to see only my jobs in OOD (and make sure OOD looks as it should for our users). But, as you I am not sure how that could be done, perhaps it may need some help from the XDMoD developers.

I’ll also ask our XDMoD admin on the ingest frequency here. I was not thinking straight when talking about the active jobs - obviously those will not be in XDMoD yet.

One more XDMoD specific question - I noticed that our job efficiencies are <50% when a colleague mentioned that XDMoD displays Performance CoPilot’s info, which is taking into consideration hyperthreading (2x logical CPU cores as physical cores), so, in effect it’s unlikely to get better than 50% efficiency. How do you deal with that at OSC?

Thanks,
MC