DEX error: "bad record MAC"

Get Help
1

Hi. I’m trying to get OOD working with DEX for ldap integration against freeipa.

When I browse to my login node’s IP, I get an “internal server error”, with DEX (running on that login node) showing:

Dec 07 10:29:54 login-0.noetchosts.invalid ondemand-dex[93160]: 2022/12/07 10:29:54 http: TLS handshake error from 10.60.102.67:33420: local error: tls: bad record MAC

Note 10.60.102.67 is the login node’s address, i.e. this is dex trying to contact dex?

Both OOD and dex are running on the cluster’s login node and freeipa is running on the cluster’s control node. Nothing has floating IPs or “real” domain names, so I’m using ssh ProxyJump to browse to the login node’s private IP. Note without dex and freeipa, just using PAM, this all works fine (i.e. I can reach OOD from the browser and auth etc etc.). All nodes are RockyLinux8.6.

OOD is using the default self-signed cert at /etc/pki/tls/certs/localhost.crt which is generated on install of the ood-apache package, and I’m using the OOD DEX defaults so DEX is using that cert too.

Someone suggested the problem might be because the cert’s SAN doesn’t include the IP address which DEX is using. This seemed plausible as it doesn’t:

[root@login-0 rocky]# openssl x509 -noout -text -in /etc/ood/dex/localhost.crt | grep -A1 "Subject Alternative Name"
            X509v3 Subject Alternative Name: 
                DNS:login-0.noetchosts.invalid

However I’ve tried two things to fix that, and both still give the same error:

  1. Changing the dex config to point to the hostname, i.e. modifying /etc/ood/dex/config.yaml to have

    #issuer: https://10.60.102.67:5554
issuer: https://login-0.noetchosts.invalid:5554

    (commented-out line is original, OOD-generated version) then running systemctl restart ondemand-dex. I can see the new issuer in systemctl status ondemand-dex but I get the same error. (I know that if I restart apache, that rewrites dex’s config, so I didn’t do that)

  2. Creating a new cert with both IP and DNS in the SANs:

    openssl req -x509 -newkey rsa:4096 -sha256 -days 3650 -nodes   -keyout oodv2.key -out oodv2.crt -subj '/CN=login-0.noetchosts.invalid'   -addext 'subjectAltName=DNS:login-0.noetchosts.invalid,IP:10.60.102.67'

    then modified the dex config to point to that, and do systemctl restart ondemand-dex. Again, I get the same error.

Any more suggestions would be very much appreciated!

2

Hi.

Sorry you are having issues.

Could you please paste your /etc/hosts file for me to see?

Thanks,
-gerald

3

Thanks. On the login node its basically empty, as this is enroled with freeipa:

[rocky@login-0 ~]$ cat /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
4

Hi @tdockendorf .

Do you have any advise regarding this issue? We spoke this morning as a team, and no one had an answer.

Thanks,
-gerald

5

Using untrusted certs is going to be really complicated to get working because Dex expects certificates used for itself, OOD, and LDAP (FreeIPA) to all be trusted. You will have to add the CA for the certs to the system trust store otherwise Dex will never trust they are valid.

We have some documentation on updating the CA trust store: 3. Secure Apache httpd — Open OnDemand 2.0.20 documentation

Another issue is if you are just editing dex.yaml manually you will run into problems with the Apache OAuth configs not also being updated to reflect the correct name and if the issuer and OAuth Apache configs don’t match, that will cause issues. I’d recommend only using ood-portal-generator to modify the Apache configs as well as the Dex configs so you know they are in sync.

6

Thanks for the responses. The untrusted certs issue for DEX is definitely a pain-point for this use. For the moment it seems like using basic_auth and PAM is actually enough to allow OOD to authenticate on a node which is enroled with FreeIPA, so I’m going to go with that to start with.

7

This topic was automatically closed 180 days after the last reply. New replies are no longer allowed.