I believe this has come up before and I think I understand some of the difficulties with implementing this, but we provide small (50GB) home directories, generous (10TB) scratch directories, and groups can also lease “project” storage. More sophisticated users, who are able to use a command line proficiently, usually run jobs from /scratch or /project. Is there any workaround that allows a user to set this up in Job Composer?
If this just isn’t going to work out, I may discuss with the rest of the team whether we might want to set the OOD root directory to /scratch. And tell users with /project to learn some command line.
I think the best practice is to hold data files in scratch or project and only hold shell scripts in your $HOME. So even though your scripts are in $HOME, the reference data located in other locations.
Indeed we tell our users the same because even though our HOME directories are large - they’re slower than project or scratch.
That said, you can set the OOD_DATAROOT to your scratch directory (as you’ve indicated). Though, this will be a breaking change for your users as they have everything in $HOME, they’ll have to copy everything to the appropriate location which could be kind of burdensome.
Thanks. We have been emphasizing getting them to command line for SLURM batch jobs and I was trying to de-emphasize that for beginners, who are often quite overwhelmed with CLI at first, so have been writing a tutorial that is more oriented around OOD. But it seems that if they are going to have to add a cd /scratch/someplace to their job scripts, maybe redirect output, or modify scripts or codes (and most of our users don’t have the ability to modify the programs they use), it seems like they might as well learn to cd /scratch/someplace and then type sbatch from a terminal. The File Explorer does relieve the need to learn a lot of file commands right off, so that helps even in this case.