Sometimes at random i get a Permission denied fatal error while accessing some of my files.
In the example below I show you one of such errors occurred when i was trying to launch an interactive application (ParaView).
Normally there are no permission issues while interacting with my files. After this happens something wrong persist with the session and I’m unable to create/read/delete files anymore. A restart of the webserver is needed to bring things back to normal.
Hi Nicola, could you inspect the /home/user/ondemand/data/sys/dashboard/batch_connect/sys/Paraview/output/cf3de1a7-b3f6-45c0-a185-fcc1c587f7da/connection.yml file and verify that the mode is 600? AFAIK ondemand should be running as the user when these files are created, so the case that you create a file you cannot later view/edit shouldn’t happen.
After this happens something wrong persist with the session and I’m unable to create/read/delete files anymore
Does this mean you can’t perform any file operations across the system? It seems like this could be an error with your file system, especially if you are using a network file system (NFS).
The permissions are correct, so I shouldn’t have any issues with reading, writing, or executing. Also, since it happens randomly, I’m not able to properly reproduce the environment to test the error thoroughly.
Does this mean you can’t perform any file operations across the system? It seems like this could be an error with your file system, especially if you are using a network file system (NFS).
I start having issues when creating jobs or for example, when I need to upload a file, rename it, or upload a new one. So the whole environment seems compromised. The only thing that still works is connecting via SSH with the app.js terminal (probably because it switches to a different machine).
I’m using NFS with Access Control List (ACL).
I know this isn’t very repeatable, but do you notice anything in your journalctl or dmesg kernel logs when this happens? The fact that it works fine after a server restart, and that it doesn’t impact the shell app seem to suggest that it is some sort of momentary glitch when the file permissions are read. I don’t have much more insight other than that, but hopefully there are some clues in the kernel logs that can point towards what is actually happening.
First of all, thank you very much for your reply. Unfortunately, I don’t see any relevant information in the kernel logs, and there are no errors in the httpd logs either.
Initially, I thought it might be a conflict between the OnDemand nginx and Apache httpd, but even in that case there are no significant errors.
Have you ever encountered this kind of issue with Open OnDemand? Could it be an HTTPD service request error?
In any case, I’ve noticed that during the issue, the httpd service repeatedly sends “requests” trying to get a response, as if it’s unable to reply properly.
In this context, it often eventually returns a proxy error after a certain amount of time.
It doesn’t happen every time the issue occurs, but it’s a recurring error.