File app buffer space

I’m deploying OOD on a VM on each cluster’s services VM host. Unfortunately these have pretty limited disk space so I’m offloading the file upload buffer space to an NFS share intended for these types of things (this also lets me set a much larger max file size than I could otherwise), but I’m not quite sure where it buffers to. The github page says /var/tmp but the documentation site says /var/lib/nginx/tmp/<user>/client_body. Unfortunately, I couldn’t experimentally determine it - I see df filling up /var partition, but I think it doesn’t flush the file out so I can’t actually find it. Which is it?

Also, the file app doesn’t seem to gracefully handle it’s buffer space filling up. It just sits there at 100%. At point I managed to get a partial file to show up without any indication it wasn’t complete, but I can’t seem to replicate that. I’m not sure if this is improved in the latest (I’m on [1.4.1] - 2018-01-03) but wanted to mention it.

Dan,

It should be /var/lib/nginx/tmp/<user>/client_body. That is controlled by the variable pun_tmp_root set in /etc/ood/config/nginx_stage.yml. If you are still in an experimental mood: try setting that to a directory that your OOD user does not have write permissions to, and attempt to upload a non-trivial file (like ~100mb in size). The file explorer GUI will hang at 100%, and the file will never actually appear in the destination directory.

We do not have a fix for failed uploads at this time.

Would you mind letting us know on which Github page you saw /var/tmp so that it can be fixed?

Opened https://github.com/OSC/ood-fileexplorer/issues/187

Thanks for clarifying the location. I saw /var/tmp on the README.md file in the fileexplorer repo:

Uploads are processed in /var/tmp by the Passenger process, so uploads will be practically limited by the available space in this location.

That was exactly the same behavior I saw with filling partition - just stuck at 100%. I wish I could replicate the partial file again (I was trying a 10GB file, and got a 200 something MB file), that seems far worse than just not uploading at all.

Thanks again!

Hi, everybody @osc

i have the same topic on OOD 1.7.6 about reaching the limit size of upload, sized by default to 10GB.
i have followed recommendation described in Is it possible to limit file upload size and created the /etc/ood/config/apps/files/env and set FILE_UPLOAD_MAX to 100GB limit
i have moved too the /var/tmp/ondemand-nginx under /tmp to have enough space and create a link to avoid changing the conf in nginx_stage.yml.

nevertheless, the upload of a file of 23GB fails :anguished:

[root@masterc77 tmp]# grep download.tar /var/log/ondemand-nginx/mjl/error.log
2020/03/13 22:37:39 [error] 18587#0: *545 client intended to send too large body: 24696064512 bytes, client: unix:, server: localhost, request: “PUT /pun/sys/files/api/v1/fs/data/COMPUTING/mjl/retour/download.tar HTTP/1.1”, host: “localhost”, referrer: "http…

what i have missed ?

thanks for your answer

jean-marie

@jms27000 Looks like we hard code 10G as a limit in the nginx PUN configuration. I’ve opened this ticket to either fix it or document it.