Hello Everyone;
I got an issue with gpu allocation on one of my interactive session:
Description
When a job requesting a GPU is submitted through OOD, a GPU is allocated. If then the user connect to the compute node using ssh to the compute node, anther GPU id is shown.
Steps to reproduce
Submit a job using a GPU on OOD (lmstudio for example)
Connect to the compute node as the same user and type nvidia-smi.
Here the Lmstudio ran with desktop background (like Matlab)
using terminal xfce inside the lmstudio session, we can see the nvidia-smi has no load on gpu:
and on my machine as root I can see all gpus with lmstudio which has load the model gema 3-4b:
(bamboo)-[root@gpu002 ~]$ nvidia-smi
Tue Feb 17 15:12:49 2026
±----------------------------------------------------------------------------------------+
| NVIDIA-SMI 590.48.01 Driver Version: 590.48.01 CUDA Version: 13.1 |
±----------------------------------------±-----------------------±---------------------+
| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|=========================================+========================+======================|
| 0 NVIDIA GeForce RTX 3090 On | 00000000:01:00.0 Off | N/A |
| 0% 28C P8 27W / 370W | 4936MiB / 24576MiB | 0% Default |
| | | N/A |
±----------------------------------------±-----------------------±---------------------+
| 1 NVIDIA GeForce RTX 3090 On | 00000000:21:00.0 Off | N/A |
| 0% 28C P8 30W / 370W | 1459MiB / 24576MiB | 0% Default |
| | | N/A |
±----------------------------------------±-----------------------±---------------------+
| 2 NVIDIA GeForce RTX 3090 On | 00000000:22:00.0 Off | N/A |
| 0% 27C P8 37W / 370W | 1MiB / 24576MiB | 0% Default |
| | | N/A |
±----------------------------------------±-----------------------±---------------------+
| 3 NVIDIA GeForce RTX 3090 On | 00000000:41:00.0 Off | N/A |
| 0% 45C P2 135W / 370W | 9463MiB / 24576MiB | 0% Default |
| | | N/A |
±----------------------------------------±-----------------------±---------------------+
| 4 NVIDIA GeForce RTX 3090 On | 00000000:81:00.0 Off | N/A |
| 0% 27C P8 29W / 370W | 1MiB / 24576MiB | 0% Default |
| | | N/A |
±----------------------------------------±-----------------------±---------------------+
| 5 NVIDIA GeForce RTX 3090 On | 00000000:A1:00.0 Off | N/A |
| 0% 28C P8 32W / 370W | 1MiB / 24576MiB | 0% Default |
| | | N/A |
±----------------------------------------±-----------------------±---------------------+
| 6 NVIDIA GeForce RTX 3090 On | 00000000:C1:00.0 Off | N/A |
| 0% 28C P8 37W / 370W | 1MiB / 24576MiB | 0% Default |
| | | N/A |
±----------------------------------------±-----------------------±---------------------+
| 7 NVIDIA GeForce RTX 3090 On | 00000000:C2:00.0 Off | N/A |
| 0% 27C P8 29W / 370W | 1MiB / 24576MiB | 0% Default |
| | | N/A |
±----------------------------------------±-----------------------±---------------------+
±----------------------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=========================================================================================|
| 0 N/A N/A 1252958 C …lmstudio/.internal/utils/node 256MiB |
| 0 N/A N/A 1253567 C …lmstudio/.internal/utils/node 4670MiB |
| 1 N/A N/A 988618 C …o/.conda/envs/scvi/bin/python 1452MiB |
| 3 N/A N/A 1040687 C …/z/zarbi/torchenv/bin/python 9456MiB |
±----------------------------------------------------------------------------------------+
here the script.sh executed by OOD to start the session:
$ cat script.sh.erb
#!/bin/bash
module spider lmstudio
Clean the environment
module purge
Work around spam message about dconf write permissions
#export XDG_RUNTIME_DIR=“/tmp/${UID}”
unsetenv XDG_RUNTIME_DIR
Set working directory to home directory
cd “${HOME}”
Launch Xfce Window Manager and Panel
(
Enable 256 colors for remote terminals
export SEND_256_COLORS_TO_REMOTE=1
XDG base directories isolated in the OOD staged root
export XDG_CONFIG_HOME=“<%= session.staged_root.join(“config”) %>”
export XDG_DATA_HOME=“<%= session.staged_root.join(“share”) %>”
export XDG_CACHE_HOME=“$(mktemp -d)”
------------------------------------------------------------------
Electron apps (LM Studio) need xdg-desktop-portal to open file dialogs.
Start the generic portal and the GTK backend for XFCE.
------------------------------------------------------------------
/usr/libexec/xdg-desktop-portal-gtk & # Backend for XFCE
/usr/libexec/xdg-desktop-portal & # Portal service
Help some Electron apps detect the desktop/session type (harmless if unused)
export XDG_CURRENT_DESKTOP=XFCE
export XDG_SESSION_TYPE=x11
Give the portal time to initialize before the DE starts
sleep 2
module restore
set -x
xfwm4 --compositor=off --sm-client-disable &
xsetroot -solid “#D3D3D3”
xfsettingsd --sm-client-disable &
xfce4-panel --sm-client-disable
) &
module load <%= context.auto_modules_lmstudio %>
module spider lmstudio
module list
lm-studio
Is anyone has an idea why xfce allocate one more gpu ?