We have experienced a regression in the login authentication time of our Ondemand environment.
Our cluster is running RHEL8.7 and we are using Ondemand 2.0.31, and for authentication we rely on PAM with sssd. After a recent update (last december) of sssd from 2.6.2 to 2.7.3 our users experienced timeouts when attempting to login to Ondemand.
These timeouts usually last 90 seconds and the user is prompted again for its credentials. The welcome page does seem to be working after the second credential prompt, but opening a shell in the environment triggers another prompt and subsequent timeout.
We have tried to clear the PUN directories of individual users to refresh the secret key but this does not seem to solve the timeouts.
Reverting the sssd update solved our timeout issues for now, but we would like to get this to work with the current sssd versions, since the sssd update is marked as a security fix.
Would someone be able to tell us if they have experienced the same behaviour? And if so, is there a fix or workaround?
Our auth section in ood_portal.yml reads:
auth:
âAuthType Basicâ
âAuthName âNetIDââ
âAuthBasicProvider PAMâ
âAuthPAMService oodâ
âRequire valid-userâ
The good sssd version:
sssd-2.6.2-4.el8_6.1.src.rpm
The bad sssd versions:
sssd-2.7.3-4.el8_7.1.src.rpm and sssd-2.7.3-4.el8_7.3.src.rpm
Thank you for the suggestion. We have set the timeout value to 60 and tried other values, but the timeouts persisted.
Fortunately, one of my colleagues has found a RedHat bug report on sssd problems (for RedHat subscribers, it was: https://access.redhat.com/solutions/6994882), with the exact error messages from PAM and matching good and bad versions of sssd RPMâs.
It turns out that newer versions of sssd parallelize requests across different threads, and that sometimes file descriptors could inadvertedly be closed by other threads, causing PAM authorizations to fail.
For now we keep the sssd version to 2.6.2 and await the release of the sssd RPM with PR 6560 included.
Very recently an update to sssd has been issued by RedHat which we have tested against the aforementioned OpenOndemand version. The updated sssd version is sssd-2.8.2-2.el8.src.rpm and this has resolved the issue: logins are snappy and opening a shell works without issues.