hi everyone,
Latest OnDemand + CentOS 8 here. It is rejecting my login attempts via LDAP (to Active Directory) but just giving a generic “username and/or pass are wrong” message and I can’t find where in /var/log the ondemand-dex logs are. Nothing in /var/log/httpd24/error.log is relevant.
ETA: perhaps relevant: there is a " character in the password for the bind service account (
). I can’t change this.
TIA,
J
Hello,
Have you checked to see if the logs are in /var/log/messages for the ondemand-dex output? I believe this is where they should be logging to so check if you can see anything there first.
Thanks,
Travis
Nothing relating to Dex or OD in there. It did have en error message about SELinux, but after disabling that I still have the same issue.
Have you enabled the LDAP module’s debug logs in the httpd/conf.d directory?
You should be able to change the file ldap_debug.conf to have Loglevel ldap_module:debug to help ensure that more helpful logs are at least being generated to troubleshoot the query being passed.
Hi,
There is no ldap_debug.conf in /etc/httpd. I thought maybe I was supposed to look in the “rh” chroot but there’s no /etc/httpd in there.
Hey sorry, I should be more clear. The idea is to make that file in that directory so that Apache will pick it up and combine it with the main apache config, but now with elevated log levels that are set in a file we are aware of. It’s just a way to partition/layer the configs for apache to use.
So on CentOS 8 I believe you could create this file under /etc/httpd/conf.d then restart apache and it should now have that configuration set to elevate the log levels for ldap.
Thanks. All I get out of it is this:
[Sun Aug 01 03:43:01.605308 2021] [mpm_event:notice] [pid 8373:tid 140450471594304] AH00489: Apache/2.4.37 (centos) OpenSSL/1.1.1g configured -- resuming normal operations
[Sun Aug 01 03:43:01.605314 2021] [core:notice] [pid 8373:tid 140450471594304] AH00094: Command line: '/usr/sbin/httpd -D FOREGROUND'
[Mon Aug 02 14:03:17.262892 2021] [mpm_event:notice] [pid 8373:tid 140450471594304] AH00491: caught SIGTERM, shutting down
[Mon Aug 02 14:03:17.606475 2021] [suexec:notice] [pid 59369:tid 140220059765056] AH01232: suEXEC mechanism enabled (wrapper: /usr/sbin/suexec)
[Mon Aug 02 14:03:17.624589 2021] [lbmethod_heartbeat:notice] [pid 59369:tid 140220059765056] AH02282: No slotmem from mod_heartmonitor
[Mon Aug 02 14:03:17.625001 2021] [ldap:debug] [pid 59369:tid 140220059765056] util_ldap.c(3023): AH01316: LDAP merging Shared Cache conf: shm=0x55cf00650bc0 rmm=0x55cf00650c18 for VHOST: nei-globus2.nei.nih.gov
[Mon Aug 02 14:03:17.625006 2021] [ldap:debug] [pid 59369:tid 140220059765056] util_ldap.c(3023): AH01316: LDAP merging Shared Cache conf: shm=0x55cf00650bc0 rmm=0x55cf00650c18 for VHOST: nei-globus2.nei.nih.gov
[Mon Aug 02 14:03:17.625214 2021] [ldap:info] [pid 59369:tid 140220059765056] AH01318: APR LDAP: Built with OpenLDAP LDAP SDK
[Mon Aug 02 14:03:17.625218 2021] [ldap:info] [pid 59369:tid 140220059765056] AH01319: LDAP: SSL support available
[Mon Aug 02 14:03:17.628583 2021] [mpm_event:notice] [pid 59369:tid 140220059765056] AH00489: Apache/2.4.37 (centos) OpenSSL/1.1.1g configured -- resuming normal operations
[Mon Aug 02 14:03:17.628593 2021] [core:notice] [pid 59369:tid 140220059765056] AH00094: Command line: '/usr/sbin/httpd -D FOREGROUND'
To start troubleshooting from here you’ll need ldapsearch queries to generate log entries to examine, and also view the returned information.
Figuring out the correct query might take some work, but starting with checking the cn would be good.
To check if the user (here user_of_interest as an example) is what you expect try something like ldapsearch cn=user_of_interest where this is of the form ldapsearch <filters> and add more filters as needed.
Check to make sure what is returned matches what you have set for things like the basedn and attributes in Apache are correct.
Make sure the user_of_interest has only one entry, that’s an issue that’s been seen before where there exists more than one entry and sometimes does not return the same user as expected. Hopefully this allows you to at least start generating queries to debug further.
it is secure ldap on port 636 on Active Directory. is there something special I have to do to get it to use the cert it’s supposed to connect with? I don’t see that option in ood_portal.yml.
There are some settings you can add which the following doc will help with:
https://osc.github.io/ood-documentation/latest/installation/add-ssl.html
Then with an update_ood_portal and service restart the certs should be picked up and the protocol should switch over to https.
right, that’s for incoming user connections, I mean when it reaches out to do LDAP over TLS.
ETA: actually found it, there’s a key for rootCA, but I added it and it didn’t help. Dex
with respect to ldapsearch, no matter how I configure it I get ldap_sasl_bind(SIMPLE): Can't contact LDAP server (-1) which is pretty remarkably useless, heh.
Ok, the first thing to try would be to remove Dex altogether for the moment, then see if you can at least get the ldapsearch command to connect to the LDAP server and give a response. Because at the moment it seems like something is not right with the bindPW or bindDN since it looks like the LDAP server won’t even let you connect with a request to get a response.
Using the ldapsearch with the needed flags for those keys and the values you think should be working would be helpful. Because if you pass them to ldapsearch yourself and they don’t work, then obviously that is a different issue then a configuration problem. I’m hesitant to try and troubleshoot the Dex portion until we know the LDAP server is allowing you to speak with it and giving back sensible responses.
If you are sure the correct password and user are being passed and still not able to connect, then you would need to check that the Certificate is correct/not corrupted using something like openssl and hitting the LDAP server to check.
so the issue with ldapsearch turned out to be that I had to import the root CA certificate into CentOS 8’s certificate store.
login to OOD, however, still does not work. question about that: when I make changes to /etc/ood/config/ood_portal.yml and then do sudo /opt/ood/ood-portal-generator/sbin/update_ood_portal --force, I still get
No change in Apache config.
No change in the Dex config.
Completed successfully!
as though it’s just ignoring the changes I make in there. Is that normal?
That’s strange. The location looks correct for /etc/ood/config/ood_portal.yml and since you’re passing the --force option it should still generate the config even with different.
Are there any old configs generated previously? Basically, what ood-portal.conf files do you see in /etc/httpd/conf.d right now?
Try to mv those ood-portal.conf files and then attempt to again make an edit to the ood_portal.yml and then issue the update_ood_portal command again. That should force it to make a new file as there is no longer an Apache config to check against.