I have Open OnDemand and Keycloak set up on two separate hosts with SSL enabled on both. I can access Keycloak just fine and configured the ondemand_client just fine with redirects:
The OnDemand server redirects me to Keycloak authentication and I can enter my credentials, but upon logging in I see an error code 404: The requested URL was not found on this server
when redirected to ood-host.net/oidc.
In the httpd24 error.log I get the following error:
[Fri Mar 26 17:51:35.845975 2021] [auth_openidc:error] [pid 1810] [client 67.177.22.168:52446] oidc_authenticate_user: the URL scheme (https) of the configured OIDCRedirectURI does not match the URL scheme of the URL being accessed (http): the "state" and "session" cookies will not be shared between the two!
No logfiles have been generated at /var/log/ondemand-nginx.
Did I miss something in the configuration somewhere? I’ve done the same setup in a containerized environment before but I haven’t run into this issue.
OIDCProviderMetadataURL https://keycloak.example.net/auth/realms/ondemand/.well-known/openid-configuration
OIDCClientID "ondemand_client"
OIDCClientSecret "*****"
OIDCRedirectURI https://ondemand.example.net/oidc
OIDCCryptoPassphrase "****"
# Keep sessions alive for 8 hours
OIDCSessionInactivityTimeout 28800
OIDCSessionMaxDuration 28800
# Set REMOTE_USER
OIDCRemoteUserClaim preferred_username
# Don't pass claims to backend servers
OIDCPassClaimsAs environment
# Strip out session cookies before passing to backend
OIDCStripCookies mod_auth_openidc_session mod_auth_openidc_session_chunks mod_auth_openidc_session_0 mod_auth_openidc_session_1
And here’s my Keycloak vhost config:
<VirtualHost $server_ip:443>
ServerName keycloak.example.net
ErrorLog "/var/log/httpd/error_log"
CustomLog "/var/log/httpd/access_log" combined
SSLEngine on
SSLCertificateFile "/etc/letsencrypt/live/keycloak.example.net/cert.pem"
SSLCertificateKeyFile "/etc/letsencrypt/live/keycloak.example.net/privkey.pem"
SSLCertificateChainFile "/etc/letsencrypt/live/keycloak.example.net/chain.pem"
SSLCACertificatePath "/etc/letsencrypt/live/keycloak.example.net"
Include "/etc/letsencrypt/options-ssl-apache.conf"
ProxyRequests Off
ProxyPreserveHost On
ProxyPass / http://localhost:8080/
ProxyPassReverse / http://localhost:8080/
RequestHeader set X-Forwarded-Proto "https"
RequestHeader set X-Forwarded-Port "443"
</VirtualHost>
Both hosts have their own certs and TLS enabled. I did set them up on separate hosts so I think that might be part of the issue. Keycloak is using apache 2.4.6 (not from software collections).
Did you edit the ood-portal.conf manually? I think the .new file is still there because it doesn’t like the manual edits and didn’t overwrite the .conf.
You have to use the /opt/ood/ood-portal-generator/sbin/update_ood_portal to get updates, and it seems you may need to use -f to force the replacement.
Ah, well I haven’t edited the file manually and I used /opt/ood/ood-portal-generator/sbin/update_ood_portal to update the .conf file. I just tried -f but it returned:
No change in Apache config.
Completed successfully!
After that I tried renaming the old .conf file and switching the name of the .new file to ood-portal.conf, but still no luck.
I checked the modules running through apachectl but it says that oidc is working:
Yes it generated those lines in the conf file, but only in the .new file. I noticed that after doing some tweaking in the ood_portal.yml and running /opt/ood/ood-portal-generator/sbin/update_ood_portal it keeps saying there was no change in the apache config.
I’ll try rebuilding our instance and running the update_ood_portal binary at the end.
Okay still no change and this time I successfully updated the ood-portal.conf. Can you see any issues with the global configuration?
# /etc/ood/config/ood_portal.yml
---
# List of Apache authentication directives
# NB: Be sure the appropriate Apache module is installed for this
# Default: (see below, uses basic auth with an htpasswd file)
auth:
- 'AuthType openid-connect'
- 'Require valid-user'
# The server name used for name-based Virtual Host
# Example:
# servername: 'www.example.com'
# Default: null (don't use name-based Virtual Host)
servername: 'ondemand.example.net'
# Redirect user to the following URI when accessing logout URI
# Example:
# logout_redirect: '/oidc?logout=https%3A%2F%2Fwww.example.com'
# Default: '/pun/sys/dashboard/logout' (the Dashboard app provides a simple
# HTML page explaining logout to the user)
logout_redirect: '/oidc?logout=https%3A%2F%2Fondemand.example.net'
# Sub-uri used by mod_auth_openidc for authentication
# Example:
# oidc_uri: '/oidc'
# Default: null (disable OpenID Connect support)
oidc_uri: '/oidc'
# Certificates
ssl:
- 'SSLCertificateFile "/etc/letsencrypt/live/ondemand.example.net/cert.pem"'
- 'SSLCertificateKeyFile "/etc/letsencrypt/live/ondemand.example.net/privkey.pem"'
- 'SSLCertificateChainFile "/etc/letsencrypt/live/ondemand.example.net/chain.pem"'
You’re OOD configs look OK. I guess I’m wondering about the Keycloak settings, though I’m not familiar with them. @tdockendorf may know more, but we have a site downtime today so it may be a while before he can post here. If you have an old instance of Keycloak that works, I’d look at it. It looks like we use the hostname as the client id, not sure if that’s important or not (seems not) but I’d check the old configurations if you have them anyhow.
Maybe turn odic apache logging up to debug and see if there’s anything else we can find. And/or use the browsers network tab in dev tools to verify what the URL you’re requesting is. From the mod_auth_oidc github 404s seem to indicate some missing setting.
So I’m almost sure it’s a silly mistake in the configs like using http somewhere instead of https. Right now, that’s the only indicate we have to go on from the log message. I guess I’d confirm what the actual URL you’re being redirected to is (through the web tools)
Specifically what I’m looking for is this: when you POST to Keycloak, it’ll redirect to you OOD through a Location header. What’s that location header?
You don’t need to post the image (it’s a bit sensitive), just indicate that it is in fact redirecting to the https site and not http.
What errors are present at this time? Based on the original errors I’d guess that somewhere you configured something with http:// instead of https:// , maybe the client redirect URIs or other client settings inside Keycloak for the OnDemand client entry?
I had https:// specified in Keycloak, but I’ve been using a hostname alias assigned by our VM software, instead of the fully qualified DNS name. Once I changed the ood-portal config and Keycloak redirect URIs to the FQDN I was able to access the dashboard.
I’m not sure if it wasn’t working because open-id-connect doesn’t do well with aliases, or if the one I was using wasn’t a true CNAME record. Either way I’m glad it’s working now, thanks for all the help!
