Hello everyone,
Texas Tech here! I’m sure you have seen a few discourses opened by our staff members in the past requesting help. But as a matter of fact, despite our previous attempts at installing OOD in the test environment, we have never deployed the project for production to date. The good news is, we are very close to finalizing the deployment of OOD version 4.x and going fully online by the end of September.
However, we ran into a strange issue that is similar to the problems mentioned in various discourse:
-
I get “InvalidAuthenticityToken” on performing most actions - Get Help - Open OnDemand
-
HTTP 422 Error / InvalidAuthenticityToken · Issue #1193 · OSC/ondemand
In all cases above, the primary solution is to make sure a valid SSL certificate is installed and configured correctly.
The fact is, in our case, a valid SSL certificate is in place:
/etc/ood/config/ood_portal.yml:
...
servername: oodheadnode.hpcc.ttu.edu
server_aliases:
- ondemand.hpcc.ttu.edu
ssl:
- 'SSLCertificateFile "/etc/pki/tls/ssl/oodheadnode.hpcc.ttu.edu.pem"'
- 'SSLCertificateKeyFile "/etc/pki/tls/private/oodheadnode_multi.hpcc.ttu.edu.key"'
- 'SSLCertificateChainFile "/etc/pki/tls/ssl/oodheadnode.hpcc.ttu.edu.cer"'
...
And the configurations have been applied correctly in the HTTPD config files, and the portal loads with the HTTPS protocol and a valid SSL certificate:
/etc/httpd/conf.d/ood-portal.conf
...
<VirtualHost *:80>
ServerName oodheadnode.hpcc.ttu.edu
ServerAlias ondemand.hpcc.ttu.edu
RewriteEngine On
RewriteRule ^(.*) https://%{HTTP_HOST}:443$1 [R=301,NE,L]
</VirtualHost>
# The Open OnDemand portal VirtualHost
#
<VirtualHost *:443>
ServerName oodheadnode.hpcc.ttu.edu
ServerAlias ondemand.hpcc.ttu.edu
...
SSLEngine On
SSLCertificateFile "/etc/pki/tls/ssl/oodheadnode.hpcc.ttu.edu.pem"
SSLCertificateKeyFile "/etc/pki/tls/private/oodheadnode_multi.hpcc.ttu.edu.key"
SSLCertificateChainFile "/etc/pki/tls/ssl/oodheadnode.hpcc.ttu.edu.cer"
...
To clarify this further, the server’s hostname where the OOD service is running is (oodheadnode.hpcc.ttu.edu):
# hostname
oodheadnode.hpcc.ttu.edu
We also created a friendly alias (canonical) name to allow users to log in to the Portal via that URL (ondemand.hpcc.ttu.edu)
# nslookup ondemand.hpcc.ttu.edu
Server: 129.x.x.x
Address: 129.x.x.x#53
Non-authoritative answer:
ondemand.hpcc.ttu.edu canonical name = oodheadnode.hpcc.ttu.edu.
Name: oodheadnode.hpcc.ttu.edu
Address: 129.x.x.x.
And as you see in the configuration files above, we also generated the SSL certificates based on the server’s hostname, not the alias.
Below is the summary of what we have done:
-
The hostname is oodheadnode.hpcc.ttu.edu
-
SSL certificates are generated for oodheadnode.hpcc.ttu.edu
-
Portal’s alias is ondemand.hpcc.ttu.edu
-
The Shibboleth is set for ondemand.hpcc.ttu.edu(the alias)
-
Users can access the Portal via ondemand.hpcc.ttu.edu, where Shibboleth is set to.
However, when we log in to the ondemand.hpcc.ttu.edu, and select the “jobs → Job Composer” from the menu and click on “+New Job → From Default Template”, we encounter the following error:
The change you wanted was rejected.
Maybe you tried to change something you didn’t have access to.
If you are the application owner check the logs for more information.
That also leads to the following error we see in the log files:
/var/log/ondemand-nginx/<USER>/error.log
...
App 2316025 output: [2025-07-31 09:42:49 -0500 ] FATAL "[f203c030-5ab1-4711-bf6c-e56e11964548]
[f203c030-5ab1-4711-bf6c-e56e11964548] ActionController::InvalidAuthenticityToken (Can't verify CSRF token authenticity.):
[f203c030-5ab1-4711-bf6c-e56e11964548]
[f203c030-5ab1-4711-bf6c-e56e11964548] actionpack (7.0.8.5) lib/action_controller/metal/request_forgery_protection.rb:253:in `handle_unverified_request'
[f203c030-5ab1-4711-bf6c-e56e11964548] actionpack (7.0.8.5) lib/action_controller/metal/request_forgery_protection.rb:286:in `handle_unverified_request'
[f203c030-5ab1-4711-bf6c-e56e11964548] actionpack (7.0.8.5) lib/action_controller/metal/request_forgery_protection.rb:275:in `verify_authenticity_token'
[f203c030-5ab1-4711-bf6c-e56e11964548] activesupport (7.0.8.5) lib/active_support/callbacks.rb:400:in `block in make_lambda'
[f203c030-5ab1-4711-bf6c-e56e11964548] activesupport (7.0.8.5) lib/active_support/callbacks.rb:199:in `block (2 levels) in halting'
[f203c030-5ab1-4711-bf6c-e56e11964548] actionpack (7.0.8.5) lib/abstract_controller/callbacks.rb:34:in `block (2 levels) in <module:Callbacks>'
[f203c030-5ab1-4711-bf6c-e56e11964548] activesupport (7.0.8.5) lib/active_support/callbacks.rb:200:in `block in halting'
[f203c030-5ab1-4711-bf6c-e56e11964548] activesupport (7.0.8.5) lib/active_support/callbacks.rb:595:in `block in invoke_before'\n
...
We are not sure what we have done wrong with our configurations based on the server’s hostname, server alias, and SSL certificates. Or there may be a missing piece here that we have been overlooking as the documentations fall short in explaining the complexity we are faced with.
Please let us know if there is any other information we can provide. Thanks for your help!
Best Regards,
Misha
. I’m copying and pasting the Response and Request headers here: