Error internal 500 after change servername and adding certificates

Hi,

We have an OOD implementation with the following features:

  • OOD version 3.0
  • Rocky Linux 8.5

OOD was working perfectly, but we want to announce to the users its availability. We followed the following steps:

  1. We have changed the servername to its final value.
  2. We added the certificates.
  3. We have launched an update_ood_portal
  4. We restarted ondemand-dex and httpd.

And we got an “Internal Server Error”, and in the log output:

 - [03/May/2023:09:48:25 +0200] "GET / HTTP/1.1" 302 233 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/112.0.0.0 Safari/537.36"
 - [03/May/2023:09:48:25 +0200] "GET /pun/sys/dashboard HTTP/1.1" 500 527 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/112.0.0.0 Safari/537.36"
 - [03/May/2023:09:48:30 +0200] "GET /favicon.ico HTTP/1.1" 404 196 "https://myhpc.domain.es/pun/sys/dashboard" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/112.0.0.0 Safari/537.36

And in the error_log we have:

[Wed May 03 09:48:30.645786 2023] [auth_openidc:error] [pid 50875:tid 140181717690112] [client :46676] oidc_util_http_call: curl_easy_perform() failed on: https://myhpc.domain.es/dex/.well-known/openid-configuration (Connection timed out after 5000 milliseconds)
[Wed May 03 09:48:30.645859 2023] [auth_openidc:error] [pid 50875:tid 140181717690112] [client :46676] oidc_provider_static_config: could not retrieve metadata from url: https://myhpc.domain.es/dex/.well-known/openid-configuration

Regards

Timeouts are generally a connectivity issue. That is, you don’t seem to have connectivity to https://myhpc.domain.es/dex/.well-known/openid-configuration from the OnDemand instance. Seems like you need to open a network path from the OnDemand instance to that OIDC instance.

That looks like dex so maybe they’re on the same instance - myhpc.domain.es, but it can’t find itself? Perform a curl to that URL on that machine. Does it succeed?

Hi Jeff,

I have tried connecting both from outside the ondemand server and from the ondemand server itself to the public web access both port 80 and 443, and no problem. I have tried with telnet and seen traffic captured with tcpdump, there is no restriction on network access.

But then I had a doubt with the CA issue and looking at the documentation I have tried to put the intermediate certificates in /etc/pki/ca-trust/source/anchors/ directory and then updating but when I do the curl I still get this:

# cp full_chain.crt /etc/pki/ca-trust/source/anchors/
# update-ca-trust

# curl -vvvv https://myhpc.domain.es/dex/.well-known/openid-configuration
* Trying IP...
* TCP_NODELAY set
* Connected to myhpc.domain.es (IP) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
* CAfile: /etc/pki/tls/certs/ca-bundle.crt
  CApath: none
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* OpenSSL SSL_connect: SSL_ERROR_SYSCALL in connection to myhpc.domain.es:443 
* Closing connection 0
curl: (35) OpenSSL SSL_connect: SSL_ERROR_SYSCALL in connection to myhpc.domain.es:443

Regards

I’m not entirely sure what’s going on, but it’s clearly some SSL issue. Maybe an update to update-ca-certificates may help resolve?

Where’d you get the certificates from? I know for sure there are issues if they’re self signed. You need to make sure the certificate authority is in the right place if the certificates are self signed.