Login timeouts after updating sssd from 2.6.2 to 2.7.3

Get Help
1

Dear OpenOndemand community,

We have experienced a regression in the login authentication time of our Ondemand environment.

Our cluster is running RHEL8.7 and we are using Ondemand 2.0.31, and for authentication we rely on PAM with sssd. After a recent update (last december) of sssd from 2.6.2 to 2.7.3 our users experienced timeouts when attempting to login to Ondemand.

These timeouts usually last 90 seconds and the user is prompted again for its credentials. The welcome page does seem to be working after the second credential prompt, but opening a shell in the environment triggers another prompt and subsequent timeout.

We have tried to clear the PUN directories of individual users to refresh the secret key but this does not seem to solve the timeouts.

Reverting the sssd update solved our timeout issues for now, but we would like to get this to work with the current sssd versions, since the sssd update is marked as a security fix.

Would someone be able to tell us if they have experienced the same behaviour? And if so, is there a fix or workaround?

Our auth section in ood_portal.yml reads:
auth:

  • ‘AuthType Basic’
  • ‘AuthName “NetID”’
  • ‘AuthBasicProvider PAM’
  • ‘AuthPAMService ood’
  • ‘Require valid-user’

The good sssd version:
sssd-2.6.2-4.el8_6.1.src.rpm

The bad sssd versions:
sssd-2.7.3-4.el8_7.1.src.rpm and sssd-2.7.3-4.el8_7.3.src.rpm

Thank You

2

This sounds similar to what we were experiencing a couple of months ago,
although our timeouts were 30-40 seconds.

I think the workaround was adding:

timeout = 60

in the “[pam]” section of our /etc/sssd/sssd.conf files.

The default is 10 seconds.

-Dj

3

Thank you for the suggestion. We have set the timeout value to 60 and tried other values, but the timeouts persisted.

Fortunately, one of my colleagues has found a RedHat bug report on sssd problems (for RedHat subscribers, it was: https://access.redhat.com/solutions/6994882), with the exact error messages from PAM and matching good and bad versions of sssd RPM’s.

We believe we have been bitten by this bug: 2149091 – Update to sssd-2.7.3-4.el8_7.1.x86_64 resulted in "Request to sssd failed. Device or resource busy"

It turns out that newer versions of sssd parallelize requests across different threads, and that sometimes file descriptors could inadvertedly be closed by other threads, causing PAM authorizations to fail.

For now we keep the sssd version to 2.6.2 and await the release of the sssd RPM with PR 6560 included.

4

Very recently an update to sssd has been issued by RedHat which we have tested against the aforementioned OpenOndemand version. The updated sssd version is sssd-2.8.2-2.el8.src.rpm and this has resolved the issue: logins are snappy and opening a shell works without issues.

5

I’m so sorry we didn’t get to this topic before. A thousand apologies.

Glad to hear an sssd update worked!

1 Like