Bad record MAC with hpc-tools-tutorial

Hello!

Im using the HPC-Toolset Demo to test Open OnDemand locally.

The ColdFront container and xdmod are working fine, however when i access the Open OnDemand webpage, i get the 500 error.

When i see the container log, i get the following error:

http: TLS handshake error from [::1]:57156: local error: tls: bad record MAC

Do i need to do some additional steps?

Hi and welcome!

Sorry for the delay. I’m looking into this. Did you build the containers locally or pull them from Dockerhub? I believe the images on dockerhub have expired certificates, so they shouldn’t be used. You should build them locally to use them.

Hi, thanks for the reply!

Yes I built all the containers locally, without using Dockerhub, but thanks for the warning!

Well I’m looking to replicate and can’t build the containers due to some numpy error now…

I’m on this and will reply back when I’ve found something.

Ok, I’ll wait for your reply!

Thanks!

I’m unable to replicate. What’s your OS & browser and can you try in a different browser.

Im using Ubuntu 22.04.5 LTS with Firefox and Chrome for the tests.

I will check other browsers and give a reply.

Thanks!

Were there any modifications you made to these containers before you started them?

dex (built in the go language) seems to be encountering this, where either you’re trying to access this through http instead of https or your browser doesn’t like the certificates it’s been given.

I usually just login to OnDemand without the others - so it may be an order thing. Like somehow the certificates from coldfront conflict with the ones in ondemand, though I think I’m grasping at straws there.

I did not make any modifications to the containers, just followed the steps in the tutorial. I am using HTTPS only and tried with the Opera browser, but without success.

I am going to modify the certificates to see if anything changes.

Hi Jeff,

I changed the certificates for OOD, and now it seems to be working just fine!

For anyone who encounters the same problem, here is the script that I used:

# Generate CA
openssl genrsa -out ca.key 4096
openssl req -new -x509 -days 100000 -sha256 -key ca.key -extensions v3_ca -out ca.crt -subj "/C=US/ST=YourState/L=YourCity/O=YourOrganization/OU=YourDepartment/CN=localhost"
# Generate certificate request
openssl genrsa -out localhost.key 2048
openssl req -new -sha256 -key localhost.key -out localhost.csr -subj "/C=US/ST=YourState/L=YourCity/O=YourOrganization/OU=YourDepartment/CN=localhost"
# Config for signing cert
cat > localhost.ext << EOF
authorityKeyIdentifier=keyid,issuer
basicConstraints=CA:FALSE
keyUsage = digitalSignature, nonRepudiation, keyEncipherment, dataEncipherment
subjectAltName = DNS:localhost
extendedKeyUsage = serverAuth
EOF

# Sign cert request and generate cert
openssl x509 -req -CA ca.crt -CAkey ca.key -CAcreateserial \
  -in localhost.csr -out localhost.crt \
  -days 100000 -sha256 -extfile localhost.ext

#Add to Docker Container
docker cp ca.key ondemand:/etc/pki/tls/
docker cp ca.crt ondemand:/etc/pki/tls/
docker cp localhost.key ondemand:/etc/pki/tls/private/
docker cp localhost.csr ondemand:/etc/pki/tls/certs/
docker cp localhost.ext ondemand:/etc/pki/tls/
docker cp localhost.crt ondemand:/etc/pki/tls/certs/
docker cp ca.crt ondemand:/etc/pki/ca-trust/source/anchors/

Remember to execute “update-ca-trust extract” on the ondemand container after these steps!

Hope that helps, and I appreciate the help, Jeff. Thanks!

You’re welcome, I guess. I have to admit I didn’t do much for you!

Odd situation for you to be in, maybe it’s because you generated the certificates outside of the container? (when you built the images, it generated the certificates within the container).

In any case - all’s well that ends well in my book. Glad to see you figured it out!