Dex with LDAP giving internal server error (round 2)

This is a re-do of a March issue where that solution is not working this time. I have re-installed OOD, and have the dex section of /etc/ood/config/ood_portal.yml like this:

dex:
# # Default based on if ssl key for ood-portal-generator is defined
  ssl: true
# # Only used if SSL is disabled
#  http_port: "5556"
# # Only used if SSL is enabled
  https_port: "5554"
# # tls_cert and tls_key take OnDemand configured values for ssl and copy keys to /etc/ood/dex maintaining file names
  tls_cert: null
  tls_key: null
  storage_file: /etc/ood/dex/dex.db
  grpc: null
  expiry: null
# # Client ID, defaults to servername or FQDN
#  client_id: null
  client_name: OnDemand
# # Client secret, value auto generated
# # A value that is a filesystem path can be used to store secret in a file
  client_secret: /etc/ood/dex/ondemand.secret
# # The OnDemand redirectURI is auto-generated, this option allows adding additional URIs
#  client_redirect_uris: []
# # Additional Dex OIDC clients to configure
#  static_clients: []
# # The following example is to configure OpenLDAP
# # Docs: https://github.com/dexidp/dex/blob/master/Documentation/connectors/ldap.md
  connectors:
     - type: ldap
       id: ldap
       name: LDAP
       config:
         host: it-hs-dc03.cshl.edu:636
         rootCA: /etc/ssl/certs/it-hs-dc03.pem
         insecureSkipVerify: false
         bindDN: "CN=ldap authentication,OU=Service Accounts,OU=ITStaff,OU=Departments,OU=CSHLusers,DC=cshl,DC=edu"
         bindPW: *****
         userSearch:
           baseDN: OU=CSHLusers,DC=cshl,DC=edu
           filter: "(objectClass=user)"
           username: uid
           idAttr: uid
           emailAttr: uid
           nameAttr: uid
           preferredUsernameAttr: uid
        groupSearch:
           baseDN: OU=CSHLgroups,DC=cshl,DC=edu
           filter: "(objectClass=group)"
           userMatchers:
             - userAttr: DN
               groupAttr: member
           nameAttr: cn

I am getting an internal server error with the following in the log:

[Mon May 09 16:38:23.577353 2022] [auth_openidc:error] [pid 2066773:tid 140388216600320] [client 143.48.8.161:52349] oidc_check_userid_openidc: configuration error: the authentication type is set to "openid-connect" but OIDCRedirectURI has not been set

Ideas and suggestions welcome!

Thanks

Hi @patoddh.

Thanks for posting. I do not see the exact issue. I’m going to paste my personal, functional LDAP config for OOD. It works properly with the exception that I have not setup ssl since i’m on my private network. Maybe you can compare the mine against yours, or maybe someone else will be able to chime in to help.

Thanks,
-gerald


issuer: http://rocky8.gbsw.net:5556
storage:
** type: sqlite3**
** config:**
** file: “/etc/ood/dex/dex.db”**
web:
** http: 0.0.0.0:5556**
telemetry:
** http: 0.0.0.0:5558**
staticClients:
- id: rocky8.gbsw.net
** redirectURIs:**
** - http://rocky8.gbsw.net/oidc**
** name: OnDemand**
** secret: d79a85fb-afdf-43ab-83df-5fdc57552e0b**
connectors:
- type: ldap
** id: ldap**
** name: LDAP**
** config:**
** host: ldapmaster.gbsw.net:636**
** insecureSkipVerify: true**
** bindDN: cn=admin,dc=ldapmaster,dc=gbsw,dc=net**
** bindPW: *******
** userSearch:**
** baseDN: ou=People,dc=ldapmaster,dc=gbsw,dc=net**
** filter: “(objectClass=posixAccount)”**
** username: uid**
** idAttr: uid**
** emailAttr: mail**
** nameAttr: gecos**
** preferredUsernameAttr: uid**
** groupSearch:**
** baseDN: ou=Groups,dc=ldapmaster,dc=gbsw,dc=net**
** filter: “(objectClass=posixGroup)”**
** userMatchers:**
** - userAttr: DN**
** groupAttr: develop**
** nameAttr: cn**
oauth2:
** skipApprovalScreen: true**
enablePasswordDB: false
frontend:
** dir: “/usr/share/ondemand-dex/web”**
** theme: ondemand**

Thanks for that info! This morning I noticed that the ondemand-dex service was not running because /etc/ood/dex/config.yaml does not exist.

Looking at this page (which I don’t recall so don’t think was part of my March install)… OpenID Connect with Dex — Open OnDemand 2.0.20 documentation

All it says is "The default location for Dex configurations is /etc/ood/dex/config.yaml". The OOD Dex installation did not create it.

What should that file be?


BTW, that page has an error. The following command on the page needs “ondemand-dex” added onto the end of it:

sudo useradd -r -d /var/lib/ondemand-dex -g ondemand-dex -s /sbin/nologin -c "OnDemand Dex"

Thanks for the info.

The content that I pasted in the previous post is the full content of my /etc/ood/dex/config.yaml file.

The path for the config.yaml is in /etc/ood/dex
The other files in that directory are:
config.yaml
config.yaml.20220114T112133
config.yaml.20220114T132545
dex.db
ondemand.secret

You have your “connectors: type: ldap: …” config in /etc/ood/dex/config,yaml? I was following these instructions:

https://osc.github.io/ood-documentation-test/develop/authentication/dex.html#configuring-ondemand-dex-for-ldap

So the “connectors: type: ldap: …” config is in a dex: section of /etc/ood/config/ood_portal.yml

Which worked in March (and I noticed then there was a ondemand-nginx user/group used instead of the current ondemand-dex user/group).

So I created a /etc/ood/dex/config.yaml, the non-LDAP part of which is:

issuer: https://lightshare.cshl.edu:5554
storage:
  type: sqlite3
  config:
    file: "/etc/ood/dex/dex.db"
web:
  http: 0.0.0.0:5556
  https: 0.0.0.0:5554
  tlsCert: "/etc/ood/dex/cshl.edu.crt"
  tlsKey: "/etc/ood/dex/cshl.edu.key"
telemetry:
  http: 0.0.0.0:5558
staticClients:
- id: lightshare.cshl.edu
  redirectURIs:
  - https://lightshare.cshl.edu/oidc
  name: OnDemand
  secret: "/etc/ood/dex/ondemand.secret"
connectors:
- type: ldap

  <snip>

That get ondemand-dex running, but reloading the page gives an internal server error, with this in the apache log:

[Tue May 10 14:05:45.449202 2022] [auth_openidc:error] [pid 2106852:tid 140057406928640] [client 143.48.8.161:63181] oidc_util_http_call: curl_easy_perform() failed on: https://lightshare.cshl.edu:5554/.well-known/openid-configuration (Failed to connect to lightshare.cshl.edu port 5554: Connection refused)
[Tue May 10 14:05:45.449275 2022] [auth_openidc:error] [pid 2106852:tid 140057406928640] [client 143.48.8.161:63181] oidc_provider_static_config: could not retrieve metadata from url: https://lightshare.cshl.edu:5554/.well-known/openid-configuration

I left the old dex stuff in /etc/ood/config/ood_portal.yml alone; not sure what that file is being used for now.

The documentation needs some attention :slight_smile:

Thanks for the feedback. We are definitely working on a documentation overhaul.

In regards to your failed connection, I would recommend ensuring that port 5554 is allowed in the firewall. It may be blocked.

Thanks,
-gerald

Thanks. The firewall rule for 5554 was there (from before), but doing “firewall-cmd --reload” fixed the issue.

Now it appears LDAP authentication is working (as per /var/log/messages), but the OOD page says: " Home directory not found… Your home directory appears to be missing. The home directory mount may be unavailable, or your home directory may need to still be created. Please contact support for help and attempt to restart your web server by clicking below when the problem has been fixed."

But the home directory does exist and I can log in and into the home directory via ssh.

The only logging I see is in /var/log/ondemand-nginx…

[root@lightshare ~]# cat /var/log/ondemand-nginx/error.log
2022/05/11 10:00:03 [notice] 2147931#0: signal process started
2022/05/11 10:36:46 [notice] 2149486#0: signal process started
2022/05/11 10:36:46 [error] 2149486#0: open() "/var/run/ondemand-nginx/patoddh/passenger.pid" failed (2: No such file or directory)
[root@lightshare ~]#

I appreciate any help/suggestions.

The error message could be a red herring. Looks like the log file is stating that passenger.pid is not found. You might want to ensure that passenger is actually running and that the pid file is located in the location that OOD is expecting.

Thanks for all your help here. I don’t know anything about passenger, but I restarted services. I’m still getting “home directory not found” on the web page, but the log file no longer has that message, and I now see the file /var/run/ondemand-nginx/patoddh/passenger.pid

The only new logging is in /var/log/ondemand-nginx/patoddh/error.log…

[ N 2022-05-11 14:21:25.2229 2157540/T1 age/Wat/WatchdogMain.cpp:1373 ]: Starting Passenger watchdog...
[ N 2022-05-11 14:21:25.2472 2157543/T1 age/Cor/CoreMain.cpp:1340 ]: Starting Passenger core...
[ N 2022-05-11 14:21:25.2474 2157543/T1 age/Cor/CoreMain.cpp:256 ]: Passenger core running in multi-application mode.
[ N 2022-05-11 14:21:25.2599 2157543/T1 age/Cor/CoreMain.cpp:1015 ]: Passenger core online, PID 2157543
[ E 2022-05-11 14:21:27.6456 2157543/T5 age/Cor/SecurityUpdateChecker.h:521 ]: A security update is available for your version (6.0.7) of Phusion Passenger. We strongly recommend upgrading to version 6.0.14.
[ E 2022-05-11 14:21:27.6456 2157543/T5 age/Cor/SecurityUpdateChecker.h:526 ]: Additional security update check information: 
- [Fixed in 6.0.14] [CVE-2018-25032] zlib before 1.2.12 allows memory corruption when deflating (i.e., when compressing) if the input has many distant matches.
- [Fixed in 6.0.14] A use after free memory safety issue was introduced in 6.0.12, and fixed in 6.0.14.

I see an “E” on the lefts side of a couple of lines that sound like a suggestion. Error?

Is Phusion Passenger part of OOD?

Thanks

Phusion is part of Ruby on Rails. With that being the case yes, it is part of OOD. I would recommend upgrading passenger.

How would I do that? “yum update” doesn’t show anything passenger-like and ruby.x86_64 is up to date.

Thanks!

Users cannot update passenger directly. I’ll submit a ticket to update it upstream.

That said - back to the original issue about HOME directory. Could be an SELinux issue. If you’ve enabled SELinux, have you also enabled the booleans? There’s another active discourse topic for SELinux if that’s the case.

1 Like

Thanks @jeff.ohrstrom

SELinux is on, but I don’t see any SELinux messages in the logs. I also did this when installing:

[root@lightshare ~]# setsebool -P ondemand_use_ssh=off
[root@lightshare ~]# setsebool -P ondemand_use_nfs=off
[root@lightshare ~]# setsebool -P ondemand_use_sssd=off

Is there another boolean?

Are you using local HOME directories? I don’t think we wrote policies for that, so I’d bet you’re getting denials there.

I’m guessing you’re looking for ondemand_use_nfs-ish type policy for local HOME, though I’m not an SELinux issue, if the directory exists but we complain about not seeing I’d bet money it’s an SELinux issue.

No, they are not local home directories, they are on NFS and mapped via SSSD. And they worked on the installation I did in March, prior to this re-installation.

I experimented with the “semodule -DB”, “semodule -B”, and “cat /var/log/audit/audit.log | audit2allow” things referred to in the other thread/ticket you pointed me to. I also turned the ondemand_manage_user_home_dir boolean on and off (restarting services). Nothing made any difference in the logging. I’m stumped at the moment.

Ah, got it. I did this:

setsebool -P ondemand_use_nfs=on