# VirtualGL on Shared Node with Multiple GPUs

**URL:** <https://discourse.openondemand.org/t/virtualgl-on-shared-node-with-multiple-gpus/1979>\
**Category:** Get Help\
**Tags:** question\
**Created:** [March 24, 2022, 10:11pm UTC](https://discourse.openondemand.org/t/virtualgl-on-shared-node-with-multiple-gpus/1979 "2022-03-24T22:11:49Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![ping.luo](https://avatars.discourse-cdn.com/v4/letter/p/b5e925/32.png) [@ping.luo](https://discourse.openondemand.org/u/ping.luo)\
**Post date:** [March 24, 2022, 10:11pm UTC](https://discourse.openondemand.org/t/virtualgl-on-shared-node-with-multiple-gpus/1979/1 "2022-03-24T22:11:49Z")

</div>

First thank @mcuma for raising the question in [this post](https://discourse.openondemand.org/t/virtualgl-in-ood-apps/750) and everyone else for providing additional information.

We were able to test direct GPU access through the EGL backend using @Micket’s method. However, for one of the applications we tested, older versions didn’t work properly: only the latest version could render smooth images, and all older versions showed corrupted images. The VirtualGL 3.0 [user guide](https://rawcdn.githack.com/VirtualGL/virtualgl/main/doc/index.html#hd002001) specifically mentioned that “As of this writing, the EGL back end does not yet support all of the GLX extensions and esoteric OpenGL features that the GLX back end supports.” Maybe that’s the reason the older versions of the software failed.

We then tested the GLX backend. With this approach, all versions of the software worked fine. However, we encountered a problem when testing two jobs shared the same node with multiple GPUs. The X server were started on the right GPU device. However, the execution of `vglrun` that happened later would always make the first` vglrun` terminated with the following error:

```auto
[VGL] ERROR: OpenGL error 0x0502
[VGL] ERROR: in readPixels--
[VGL] 435: Could not read pixels

```

Any ideas are appreciated.

Ping

PS, I want to mention that instead of changing TaskProlog in Slurm, I wrote a wrapper script for `vglrun`. It sets up the correct environment variables to use either the EGL backend or the GLX backend.

---

<div class="post-metadata">

**Author:** ![jeff.ohrstrom](https://sea1.discourse-cdn.com/flex015/user_avatar/discourse.openondemand.org/jeff.ohrstrom/32/136_2.png) [@jeff.ohrstrom](https://discourse.openondemand.org/u/jeff.ohrstrom)\
**Post date:** [March 25, 2022, 1:58pm UTC](https://discourse.openondemand.org/t/virtualgl-on-shared-node-with-multiple-gpus/1979/2 "2022-03-25T13:58:02Z")

</div>

I’m not sure, but I did find this where you may be able to find some debugging tips.

> <https://github.com/VirtualGL/virtualgl/issues/129>
>
> Hello, I'm on Ubuntu 20.04, nvidia driver 440. Virtualgl not working, e.g.:
> 
> \`…\`\`
> vglrun +v glxgears
> \[VGL\] Shared memory segment ID for vglconfig: 32802
> \[VGL\] VirtualGL v2.6.4 64-bit (Build 20200626)
> \[VGL\] Opening connection to 3D X server :0
> \[VGL\] Using Pbuffers for rendering
> \[VGL\] Using pixel buffer objects for readback (BGR --\> BGRA)
> \[VGL\] ERROR: OpenGL error 0x0502
> \[VGL\] ERROR: in readPixels--
> \[VGL\] 463: Could not Read Pixels
> \`\`\`
> This error started on Ubuntu 18.04 actually. I used to be able to connect to this machine via command line and vglconnect.
> 
> Thanks for any help!

These may be naive questions that you’ve already sorted through - but they popped into my mind and may help you debug.

- Sounds like you know it’s an issue that only arises when you use multiple GPUs. Is that right?
- Are you sure you’re attaching to the right `$DISPLAY` in each session? Lot’s of chatter about $DISPLAY on the github issue.

---

<div class="post-metadata">

**Author:** ![mcuma](https://sea1.discourse-cdn.com/flex015/user_avatar/discourse.openondemand.org/mcuma/32/171_2.png) [@mcuma](https://discourse.openondemand.org/u/mcuma)\
**Post date:** [March 25, 2022, 5:04pm UTC](https://discourse.openondemand.org/t/virtualgl-on-shared-node-with-multiple-gpus/1979/3 "2022-03-25T17:04:35Z")

</div>

Hi Ping,

what’s the application that does not run right with EGL?

We are setting up EGL with our new Rocky 8 setup and it’s working well, except for one application - IDV - which uses Java 3D and we suspect it’s hitting the EGL’s incomplete GLX implementation.

---

<div class="post-metadata">

**Author:** ![ping.luo](https://avatars.discourse-cdn.com/v4/letter/p/b5e925/32.png) [@ping.luo](https://discourse.openondemand.org/u/ping.luo)\
**Post date:** [March 25, 2022, 5:27pm UTC](https://discourse.openondemand.org/t/virtualgl-on-shared-node-with-multiple-gpus/1979/4 "2022-03-25T17:27:38Z")

</div>

Thanks Jeff. I’ll take a look at the discussion from the link you posted and see if anything will be useful.

To answer your question:

1. yes, it only popped up when multiple GPUs are used, and each GPU runs a different X server.
2. 
  1. I am sure I have attached the right $DISPLAY since the second vglrun always works while it causes the first running virtual crashes. In other words, the first vglrun works fine until the second vglrun starts running.

---

<div class="post-metadata">

**Author:** ![ping.luo](https://avatars.discourse-cdn.com/v4/letter/p/b5e925/32.png) [@ping.luo](https://discourse.openondemand.org/u/ping.luo)\
**Post date:** [March 25, 2022, 6:29pm UTC](https://discourse.openondemand.org/t/virtualgl-on-shared-node-with-multiple-gpus/1979/5 "2022-03-25T18:29:25Z")

</div>

Hi Martin,

The application having problems with the EGL backend is synopsis (version 5.4 and older). Version 5.5 works fine.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/flex015/uploads/osc/original/2X/b/bae70bd0ed39a3ae769c2108155f4cb3e9da8385.png) [@system](https://discourse.openondemand.org/u/system)\
**Post date:** [September 21, 2022, 6:29pm UTC](https://discourse.openondemand.org/t/virtualgl-on-shared-node-with-multiple-gpus/1979/6 "2022-09-21T18:29:42Z")

</div>

This topic was automatically closed 180 days after the last reply. New replies are no longer allowed.