RStudio app: availableCores() not reporting Slurm-allocated cores

Get Help
1

Hi all,

Can anyone else verify that, when using the RStudio app, the ‘future’ library’s availableCores() function doesn’t report the slurm-cognizant number that one could see through srun/sbatch?

The following code would be inside an interactive RStudio app session:

> library(future)
> availableCores()
system
    28
> availableCores(methods = 'slurm')
current
      1

But via an sbatch script, results would look like:

> availableCores()
Slurm
    3

…and this is true even if you submit a script through the job composer. It just doesn’t work interactively through the app. I presume, because Singularity is not revealing … something?

I figured before I tinkered too much, I’d ask if anyone had a quick answer?

Thanks!
–jason

2

We’ve found that system('nproc') is a reliable way to determine how many cores are actually available to that cgroup.

Those other libraries don’t seem to recognize cgroups that limit the parallelism or they rely on environment variables to best effort guess? Anyhow good ol’ nproc is pretty reliable.

1 Like
3

This is a known issue and the futures package dev is working to fix this.

4

Should have been more explicit. This is a known issue in the parallel/snow package. The fix is the futures R package. The dev is working to add SLURM support.

1 Like
5

Ah thank you for the info!
Indeed system(‘nproc’) fit the bill nicely for our purposes.