Note that the home directories are on NFS, and I’ve not changed any of the ondemand selinux booleans.
The hmd_page_size access from rclone is a result of enabling remote_files_enabled: true
Feeding this through audit2allow produces the following output, but I would stress that I don’t know if allowing any of these is a good idea; some of them might be better served with dontaudit rules.
module ood 1.0;
require {
type ood_pun_t;
type sysfs_t;
type nfs_t;
type systemd_userdbd_runtime_t;
class dir read;
class file read;
class sock_file unlink;
class netlink_route_socket create;
}
#============= ood_pun_t ==============
allow ood_pun_t nfs_t:sock_file unlink;
#!!!! This avc can be allowed using one of the these booleans:
# ondemand_use_torque, ondemand_use_slurm, ondemand_use_kubernetes
allow ood_pun_t self:netlink_route_socket create;
allow ood_pun_t sysfs_t:file read;
allow ood_pun_t systemd_userdbd_runtime_t:dir read;
So, are you trying to understand those logs and what is needed? The log entries look to all be related to how OOD operating for things like routing, network sockets, and working with systemd’s user database. There’s nothing odd there to me but I’m not sure if I understand the question.
@tdockendorf would you be able to speak more to this? I know all this is related to OOD’s functioning, but I’m not sure about moving things to dontaudit or allow to decrease the log verbosity and our recommendations for that.
The denial for “session-server-rpc.socket” and "sock_file " almost looks like a socket unit file name that maybe the systemd user stuff is doing when someone logs into OnDemand. This would only affect RHEL8 and RHEL9 and we’ve not seen this on at least RHEL8 host where we do run SELinux.
I think the rclone thing related to “sysfs_t” might be something we have to account for. I don’t think we’ve tested how rclone interacts with SELinux.
I can’t tell what the “systemd_userdbd_runtime_t” things are about. We’ve not seen those on RHEL8 test host so might be we need a RHEL9 host to replicate. A google search of “systemd_userdbd_runtime_t” brings up information a bout systemd-userdbd.service which looks like systemd trying to replace SSSD or something, so if you don’t need that you could probably turn off or mask that service or if it’s part of like /etc/nsswitch.conf, remove those entries.
At the very least if you’re using Slurm or Kubernetes you’ll need to enable those booleans to reduce some of the noise. If the noise is a concern you might have to create a custom policy to allow more things by “ood_pun_t” as it might be interacting with things differently on RHEL9 than we’ve tested. If the systemd user stuff is the issue you might have to find ways to turn some of that off as we’ve found the systemd user things to be quite annoying and not needed for most of what we do.
I would not recommend moving things to dontaudit cause debugging those failures is incredibly hard since the system has to be essentially idle to get anything meaningful out of the logs when dontaudit is turned off to debug failures.
And I think nss-systemd wants to use userdb (which I don’t have installed or enabled). I’ll try removing systemd from nsswitch.conf and see if that takes care of those.
I’m not running the per-user systemd stuff (I’ve removed the systemd line from system-auth which is referred to by /etc/pam.d/sudo, and there’s no systemd processes running other than the system-wide one).
Our scheduler is accessed via ssh, so I don’t need slurm/torque/kubernetes enabled.
The rclone I’m using is a local build of a 1.64 prerelease (we use ssh hostkey authentication, which isn’t handled by earlier versions of rclone, and only works in 1.64+ because you can tell it to use an external ssh command rather than an internal implementation).
I think the ssh ones also happen on the rhel8 webservers I’ve just replaced with the rhel9 ones, and probably have to do with hostkey authentication.
The socket one is strange as it looks like it’s hosted on NFS share, maybe the NFS $HOME as it appears to have the context associated with NFS mounts. Not sure if maybe rclone is using sockets in the user’s $HOME directory?
The SSH host based keys should be permitted by default via ondemand_use_ssh boolean. OSC uses SSH host based keys to permit SSH login from OnDemand into login nodes.
The sysfs_t appears to be related to the rclone config you’re using so that and possibly others could be good candidates for new boolean to support rclone.
I’ve figured out what the socket ones are - we have an rstudio app which constructs various files and directories under the job directory and then maps those into appropriate places in the singularity container. One of these directories is bound to /run, which rstudio goes and creates a socket in. The denial comes when it tries to clean up the old job directories (we have bc_clean_old_dirs set to true).