# Grafana in OOD - Ability to embed other panels

**URL:** <https://discourse.openondemand.org/t/grafana-in-ood-ability-to-embed-other-panels/3575>\
**Category:** Get Help\
**Tags:** feature-request, question\
**Created:** [June 17, 2024, 5:59pm UTC](https://discourse.openondemand.org/t/grafana-in-ood-ability-to-embed-other-panels/3575 "2024-06-17T17:59:19Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![ssivy](https://sea1.discourse-cdn.com/flex015/user_avatar/discourse.openondemand.org/ssivy/32/106_2.png) [@ssivy](https://discourse.openondemand.org/u/ssivy)\
**Post date:** [June 17, 2024, 5:59pm UTC](https://discourse.openondemand.org/t/grafana-in-ood-ability-to-embed-other-panels/3575/1 "2024-06-17T17:59:19Z")

</div>

Hello,  
I was able to integrated Grafana usage graphs into our Open OnDemand using the instructions provided in the documentation. In the documentation it lists the ‘cpu’ and ‘memory’ panels. Is there a way to get other panels to display. We also capture gpu and gpu memory usage which we would like to also display (obviously blank/empty graphs if no gpu was requested … that’s fine). I could find if there were ‘cpu’ and ‘memory’ are the only panel YAML keys available.

---

<div class="post-metadata">

**Author:** ![jeff.ohrstrom](https://sea1.discourse-cdn.com/flex015/user_avatar/discourse.openondemand.org/jeff.ohrstrom/32/136_2.png) [@jeff.ohrstrom](https://discourse.openondemand.org/u/jeff.ohrstrom)\
**Post date:** [June 21, 2024, 2:24pm UTC](https://discourse.openondemand.org/t/grafana-in-ood-ability-to-embed-other-panels/3575/2 "2024-06-21T14:24:05Z")

</div>

Without looking at the code, I suspect there’s no support for it. However, the panel should be providing a link to the more complete dashboard that users can navigate to.

 ![image](https://us1.discourse-cdn.com/flex015/uploads/osc/original/2X/3/3b5ec9695875a1d6f786b0c9db18b3e1256dc645.png)

That said - I did create this ticket to potentially support this:

> <https://github.com/OSC/ondemand/issues/3622>
>
> Someone on discourse is asking for \`gpu\` panel support in Activejobs grafana int…egration. 
> https://discourse.openondemand.org/t/grafana-in-ood-ability-to-embed-other-panels/3575
> 
> Given the rise of AI and GPU demand and so on, we should likely support this case, even if many or most jobs don't use GPUs.

---

<div class="post-metadata">

**Author:** ![jeff.ohrstrom](https://sea1.discourse-cdn.com/flex015/user_avatar/discourse.openondemand.org/jeff.ohrstrom/32/136_2.png) [@jeff.ohrstrom](https://discourse.openondemand.org/u/jeff.ohrstrom)\
**Post date:** [June 21, 2024, 2:51pm UTC](https://discourse.openondemand.org/t/grafana-in-ood-ability-to-embed-other-panels/3575/3 "2024-06-21T14:51:11Z")

</div>

> [@ssivy](#):
>
> We also capture gpu and gpu memory usage

An OSC colleague commented on the same github ticket that tracking GPU usage from Slurm and getting that into Prometheus is a bit of a challenge. Can you detail how you do the same?

---

<div class="post-metadata">

**Author:** ![ssivy](https://sea1.discourse-cdn.com/flex015/user_avatar/discourse.openondemand.org/ssivy/32/106_2.png) [@ssivy](https://discourse.openondemand.org/u/ssivy)\
**Post date:** [June 21, 2024, 4:10pm UTC](https://discourse.openondemand.org/t/grafana-in-ood-ability-to-embed-other-panels/3575/4 "2024-06-21T16:10:05Z")

</div>

I’m using this Prometheus exporter: [https://github.com/plazonic/nvidia\_gpu\_prometheus\_exporter](https://github.com/plazonic/nvidia_gpu_prometheus_exporter)

In addition to running the exporter, I needed these scripts in our SLURM prolog.d and epilog.d directories, respectively. Discussion and examples of these can be found in the GitHub for the jobstats project from our neighbors down the road at Princeton University. [https://github.com/PrincetonUniversity/jobstats](https://github.com/PrincetonUniversity/jobstats)

[root@gpu-node001 prolog.d]# cat /etc/slurm/prolog.d/gpustats\_helper\_prolog.sh

#!/bin/bash

[-z $CUDA\_VISIBLE\_DEVICES] && exit 0

DEST=/run/gpustat

[-e $DEST] || mkdir -m 755 $DEST

for i in ${GPU\_DEVICE\_ORDINAL//,/ } ${CUDA\_VISIBLE\_DEVICES//,/ }; do

echo $SLURM\_JOB\_ID $SLURM\_JOB\_UID \> $DEST/$i

done

exit 0

---

<div class="post-metadata">

**Author:** ![ssivy](https://sea1.discourse-cdn.com/flex015/user_avatar/discourse.openondemand.org/ssivy/32/106_2.png) [@ssivy](https://discourse.openondemand.org/u/ssivy)\
**Post date:** [June 21, 2024, 4:20pm UTC](https://discourse.openondemand.org/t/grafana-in-ood-ability-to-embed-other-panels/3575/5 "2024-06-21T16:20:00Z")

</div>

Here’s what the end-users see about their job usage if they click the link above the thumbnails of the images of the cpu and memory usage in OOD.

 ![jobinfo.png](https://us1.discourse-cdn.com/flex015/uploads/osc/original/2X/c/c9daa7b92cda3dcd8aee0a94640613f199063306.png)

---

<div class="post-metadata">

**Author:** ![jeff.ohrstrom](https://sea1.discourse-cdn.com/flex015/user_avatar/discourse.openondemand.org/jeff.ohrstrom/32/136_2.png) [@jeff.ohrstrom](https://discourse.openondemand.org/u/jeff.ohrstrom)\
**Post date:** [June 21, 2024, 4:21pm UTC](https://discourse.openondemand.org/t/grafana-in-ood-ability-to-embed-other-panels/3575/6 "2024-06-21T16:21:51Z")

</div>

Thanks for the additional information! I’ve updated the github ticket for the same. Yes I’m guessing you want the `GPU Utilization` panel to show up in OnDemand.

---

<div class="post-metadata">

**Author:** ![ssivy](https://sea1.discourse-cdn.com/flex015/user_avatar/discourse.openondemand.org/ssivy/32/106_2.png) [@ssivy](https://discourse.openondemand.org/u/ssivy)\
**Post date:** [June 21, 2024, 5:46pm UTC](https://discourse.openondemand.org/t/grafana-in-ood-ability-to-embed-other-panels/3575/7 "2024-06-21T17:46:37Z")

</div>

> I’m guessing you want the `GPU Utilization` panel to show up in OnDemand.  
> Yes. GPU Memory Utilization too would be nice … basically mirroring what we can display now for CPU and memory.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/flex015/uploads/osc/original/2X/b/bae70bd0ed39a3ae769c2108155f4cb3e9da8385.png) [@system](https://discourse.openondemand.org/u/system)\
**Post date:** [December 18, 2024, 5:46pm UTC](https://discourse.openondemand.org/t/grafana-in-ood-ability-to-embed-other-panels/3575/8 "2024-12-18T17:46:43Z")

</div>

This topic was automatically closed 180 days after the last reply. New replies are no longer allowed.