Hello,
We have a site where a user’s directory contains ~200k files and the file manager page crashes when trying to access (shows up to user as 502 proxy). Would it be possible to increase some timeout or other memory setting in nginx to work around this?
Logs from issue:
App 1151075 output: [2024-12-04 14:29:19 -0500 ] INFO “method=GET path=/pun/sys/dashboard/files/fs/PATH/TO/DIRECTORY format=html controller=FilesController action=fs status=200 duration=118696.55 view=67211.35”
[ N 2024-12-04 14:32:20.7901 1138756/T4 age/Cor/CoreMain.cpp:1147 ]: Checking whether to disconnect long-running connections for process 1151075, application /var/www/ood/apps/sys/dashboard (production)
…
…
App 1151075 output: [2024-12-04 14:29:19 -0500 ] INFO “method=GET path=/pun/sys/dashboard/files/fs/PATH/TO/DIRECTORY format=html controller=FilesController action=fs status=200 duration=118696.55 view=67211.35” [ N 2024-12-04 14:32:20.7901 1138756/T4 age/Cor/CoreMain.cpp:1147 ]: Checking whether to disconnect long-running connections for process 1151075, applicati
on /var/www/ood/apps/sys/dashboard (production)
[ N 2024-12-04 16:00:06.7309 1138756/T8 age/Cor/CoreMain.cpp:671 ]: Signal received. Gracefully shutting down… (send signal 2 more time(s) to force shutdown)
[ N 2024-12-04 16:00:06.7309 1138756/T1 age/Cor/CoreMain.cpp:1246 ]: Received command to shutdown gracefully. Waiting until all clients have disconnected…
Can’t speak for upping the timeouts but at the file system level a directory with 200k files/dirs in it is not great. You would have a hard time even ls’ing that dir. In my opinion that is egregious misuse of a filesystem.
I agree on that note. Users going to do user things though. Would be nice to limit number of files showing up on file manager, or some other behavior besides crash, if no other workaround.
Ah that is a good idea. @jeff.ohrstrom maybe a feature request? I suppose you might be limited by how directory contents is enumerated in the first place.
There’s no easy way to do this. getdents - the syscall that Ruby eventually makes, isn’t paginated. It’s all or nothing. Same with ls as it is with Ruby/OOD.
There is a ticket on OOD somewhere on this, it’s just super complicated.
In the very worst case the network file systems can freeze and freeze the PUN too because the file system calls never complete. and the PUN just hangs there indefinitely until you have to force stop it.
The only really reliable way to do this is to have a whole separate process that’s querying the file system. This way you could timeout in Rails, or just send back empty bodies while we wait, and have the ability to kill and respawn the process in the NFS freezes altogether. It’s just complicated because now we’d have to be in the business of managing processes when we’re not right now.