# Dex Not Starting With SSL

**URL:** <https://discourse.openondemand.org/t/dex-not-starting-with-ssl/2719>\
**Category:** Get Help\
**Created:** [May 25, 2023, 7:48pm UTC](https://discourse.openondemand.org/t/dex-not-starting-with-ssl/2719 "2023-05-25T19:48:04Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![brad.traver](https://avatars.discourse-cdn.com/v4/letter/b/82dd89/32.png) [@brad.traver](https://discourse.openondemand.org/u/brad.traver)\
**Post date:** [May 25, 2023, 7:48pm UTC](https://discourse.openondemand.org/t/dex-not-starting-with-ssl/2719/1 "2023-05-25T19:48:05Z")

</div>

After doing a fresh install on RHEL 8.6, Dex doesn’t seem to be starting with SSL support. I see these errors:

```auto
[Thu May 25 14:44:09.041069 2023] [auth_openidc:error] [pid 168028:tid 140736280393472] [client 172.27.96.248:31820] oidc_util_http_call: curl_easy_perform() failed on: https://apps.talon.und.edu/dex/.well-known/openid-configuration (SSL certificate problem: unable to get local issuer certificate)
[Thu May 25 14:44:09.041108 2023] [auth_openidc:error] [pid 168028:tid 140736280393472] [client 172.27.96.248:31820] oidc_provider_static_config: could not retrieve metadata from url: https://apps.talon.und.edu/dex/.well-known/openid-configuration

```

I’m not sure what to do here. Is there another option I need to set in ood\_portal.yml?

---

<div class="post-metadata">

**Author:** ![jeff.ohrstrom](https://sea1.discourse-cdn.com/flex015/user_avatar/discourse.openondemand.org/jeff.ohrstrom/32/136_2.png) [@jeff.ohrstrom](https://discourse.openondemand.org/u/jeff.ohrstrom)\
**Post date:** [May 26, 2023, 2:08pm UTC](https://discourse.openondemand.org/t/dex-not-starting-with-ssl/2719/2 "2023-05-26T14:08:12Z")

</div>

Are you using self signed certificates? You have to be sure that the certificates you generated for dex come from the same certificate authority that apache has. I’m not 100% sure what that means exactly, but self signed certificates always give dex problems. I’ll look for similar topics to link here.

---

<div class="post-metadata">

**Author:** ![jeff.ohrstrom](https://sea1.discourse-cdn.com/flex015/user_avatar/discourse.openondemand.org/jeff.ohrstrom/32/136_2.png) [@jeff.ohrstrom](https://discourse.openondemand.org/u/jeff.ohrstrom)\
**Post date:** [May 26, 2023, 2:09pm UTC](https://discourse.openondemand.org/t/dex-not-starting-with-ssl/2719/3 "2023-05-26T14:09:42Z")

</div>

This may help.

> [@Unknown certificate authority when using openondemand-dex](https://discourse.openondemand.org/t/unknown-certificate-authority-when-using-openondemand-dex/1083):
>
> I am trying open ondemand v1.8 with dex authentication but I see an “Internal Server Error” when trying to access and the log shows: messages:Aug 26 10:44:52 ood ondemand-dex: http: TLS handshake error from 10.2.0.4:34412: remote error: tls: unknown certificate authority I have obtained a certificate with letsencrypt (full script to do this is here: [Azure-OnDemand/setup-https-certificate.sh at master · edwardsp/Azure-OnDemand · GitHub](https://github.com/edwardsp/Azure-OnDemand/blob/master/azod-slurm/scripts/setup-https-certificate.sh)) The certificate works ok in the webpage and previously wi…

---

<div class="post-metadata">

**Author:** ![brad.traver](https://avatars.discourse-cdn.com/v4/letter/b/82dd89/32.png) [@brad.traver](https://discourse.openondemand.org/u/brad.traver)\
**Post date:** [May 26, 2023, 3:00pm UTC](https://discourse.openondemand.org/t/dex-not-starting-with-ssl/2719/4 "2023-05-26T15:00:39Z")

</div>

Jeff,

We are not using any self-signed certificates in the ood\_portal config.

---

<div class="post-metadata">

**Author:** ![jeff.ohrstrom](https://sea1.discourse-cdn.com/flex015/user_avatar/discourse.openondemand.org/jeff.ohrstrom/32/136_2.png) [@jeff.ohrstrom](https://discourse.openondemand.org/u/jeff.ohrstrom)\
**Post date:** [May 26, 2023, 3:19pm UTC](https://discourse.openondemand.org/t/dex-not-starting-with-ssl/2719/5 "2023-05-26T15:19:22Z")

</div>

Googling this search string shows a few results. They seem to be related to the CA certificate and apache’s permission to find them. Stack overflow questions have some openssl utility commands to test this.

```auto
apache SSL certificate problem: unable to get local issuer certificate

```

Where did you get your certificates from and do you have the certificate authority in the right place? You may just need to issue `update-ca-trust extract`.

---

<div class="post-metadata">

**Author:** ![brad.traver](https://avatars.discourse-cdn.com/v4/letter/b/82dd89/32.png) [@brad.traver](https://discourse.openondemand.org/u/brad.traver)\
**Post date:** [May 26, 2023, 3:23pm UTC](https://discourse.openondemand.org/t/dex-not-starting-with-ssl/2719/6 "2023-05-26T15:23:18Z")

</div>

Jeff,

After adding the SSLCertificateChainFile to the ood\_portal.yml file, I’m now getting a login page. But after logging in, I get the attached error.

 ![ood_error](https://us1.discourse-cdn.com/flex015/uploads/osc/original/2X/4/448cf4cfeaeb1904ce71edfa0d4b0236c05887ad.png)

---

<div class="post-metadata">

**Author:** ![jeff.ohrstrom](https://sea1.discourse-cdn.com/flex015/user_avatar/discourse.openondemand.org/jeff.ohrstrom/32/136_2.png) [@jeff.ohrstrom](https://discourse.openondemand.org/u/jeff.ohrstrom)\
**Post date:** [May 26, 2023, 3:28pm UTC](https://discourse.openondemand.org/t/dex-not-starting-with-ssl/2719/7 "2023-05-26T15:28:38Z")

</div>

Check in `/var/log/ondemand-nginx/$USER/error.log` for the error message(s). There may also be a tmp html file in `/tmp` that you can download and view, though the error log may say the same thing.

---

<div class="post-metadata">

**Author:** ![brad.traver](https://avatars.discourse-cdn.com/v4/letter/b/82dd89/32.png) [@brad.traver](https://discourse.openondemand.org/u/brad.traver)\
**Post date:** [May 26, 2023, 3:34pm UTC](https://discourse.openondemand.org/t/dex-not-starting-with-ssl/2719/8 "2023-05-26T15:34:53Z")

</div>

This is the error log.

```auto
[N 2023-05-26 10:20:44.9282 298503/T1 age/Wat/WatchdogMain.cpp:1373]: Starting Passenger watchdog...
[N 2023-05-26 10:20:44.9469 298506/T1 age/Cor/CoreMain.cpp:1340]: Starting Passenger core...
[N 2023-05-26 10:20:44.9470 298506/T1 age/Cor/CoreMain.cpp:256]: Passenger core running in multi-application mode.
[N 2023-05-26 10:20:45.0716 298506/T1 age/Cor/CoreMain.cpp:1015]: Passenger core online, PID 298506
App 298650 output: /usr/share/gems/gems/bundler-2.2.33/lib/bundler/definition.rb:480:in `materialize': Could not find rails-5.2.8.1, jquery-rails-4.4.0, timecop-0.9.5, thor-0.19.1, dotenv-rails-2.7.6, font-awesome-sass-5.12.0, redcarpet-3.5.1, addressable-2.8.0, data-confirm-modal-1.6.3, turbolinks-5.2.1, nokogiri-1.12.5, ood_support-0.0.3, ood_appkit-1.1.5, ood_core-0.23.2, sinatra-2.2.0, sinatra-contrib-2.2.0, erubi-1.10.0, dalli-3.2.0, webpacker-5.4.3, actioncable-5.2.8.1, actionmailer-5.2.8.1, actionpack-5.2.8.1, actionview-5.2.8.1, activejob-5.2.8.1, activemodel-5.2.8.1, activerecord-5.2.8.1, activestorage-5.2.8.1, activesupport-5.2.8.1, railties-5.2.8.1, tilt-2.0.10, public_suffix-4.0.7, i18n-1.11.0, turbolinks-source-5.2.0, mini_portile2-2.6.1, racc-1.6.0, mustermann-1.1.1, rack-2.2.4, rack-protection-2.2.0, rack-proxy-0.7.2, semantic_range-3.0.0, nio4r-2.5.8, mail-2.7.1, rack-test-2.0.2, rails-html-sanitizer-1.4.3, globalid-1.0.0, arel-9.0.0, concurrent-ruby-1.1.10, minitest-5.15.0, tzinfo-1.2.9, loofah-2.18.0, thread_safe-0.3.6, rb-fsevent-0.11.1 in any of the sources (Bundler::GemNotFound)
App 298650 output: from /usr/share/gems/gems/bundler-2.2.33/lib/bundler/definition.rb:185:in `specs'
App 298650 output: from /usr/share/gems/gems/bundler-2.2.33/lib/bundler/definition.rb:233:in `specs_for'
App 298650 output: from /usr/share/gems/gems/bundler-2.2.33/lib/bundler/runtime.rb:18:in `setup'
App 298650 output: from /usr/share/gems/gems/bundler-2.2.33/lib/bundler.rb:150:in `setup'
App 298650 output: from /usr/share/gems/gems/bundler-2.2.33/lib/bundler/setup.rb:20:in `block in <top (required)>'
App 298650 output: from /usr/share/gems/gems/bundler-2.2.33/lib/bundler/ui/shell.rb:136:in `with_level'
App 298650 output: from /usr/share/gems/gems/bundler-2.2.33/lib/bundler/ui/shell.rb:88:in `silence'
App 298650 output: from /usr/share/gems/gems/bundler-2.2.33/lib/bundler/setup.rb:20:in `<top (required)>'
App 298650 output: from <internal:/usr/share/rubygems/rubygems/core_ext/kernel_require.rb>:160:in `require'
App 298650 output: from <internal:/usr/share/rubygems/rubygems/core_ext/kernel_require.rb>:160:in `rescue in require'
App 298650 output: from <internal:/usr/share/rubygems/rubygems/core_ext/kernel_require.rb>:149:in `require'
App 298650 output: <internal:/usr/share/rubygems/rubygems/core_ext/kernel_require.rb>:85:in `require': cannot load such file -- bundler/setup (LoadError)
App 298650 output: from <internal:/usr/share/rubygems/rubygems/core_ext/kernel_require.rb>:85:in `require'
[E 2023-05-26 10:20:45.3695 298506/T3v age/Cor/App/Implementation.cpp:221]: Could not spawn process for application /var/www/ood/apps/sys/dashboard: The application process exited prematurely.
  Error ID: 0803bd37
  Error details saved to: /tmp/passenger-error-JBVKIY.html

[E 2023-05-26 10:20:45.3746 298506/T8 age/Cor/Con/CheckoutSession.cpp:283]: [Client 1-1] Cannot checkout session because a spawning error occurred. The identifier of the error is 0803bd37. Please see earlier logs for details about the error.

```

---

<div class="post-metadata">

**Author:** ![jeff.ohrstrom](https://sea1.discourse-cdn.com/flex015/user_avatar/discourse.openondemand.org/jeff.ohrstrom/32/136_2.png) [@jeff.ohrstrom](https://discourse.openondemand.org/u/jeff.ohrstrom)\
**Post date:** [May 26, 2023, 3:37pm UTC](https://discourse.openondemand.org/t/dex-not-starting-with-ssl/2719/9 "2023-05-26T15:37:59Z")

</div>

That is very odd indeed. You are running 2.0? on what operating system? Did you install it from the packages (rpm or deb)?

---

<div class="post-metadata">

**Author:** ![jeff.ohrstrom](https://sea1.discourse-cdn.com/flex015/user_avatar/discourse.openondemand.org/jeff.ohrstrom/32/136_2.png) [@jeff.ohrstrom](https://discourse.openondemand.org/u/jeff.ohrstrom)\
**Post date:** [May 26, 2023, 3:38pm UTC](https://discourse.openondemand.org/t/dex-not-starting-with-ssl/2719/10 "2023-05-26T15:38:50Z")

</div>

I see you’re on RHEL/8. You’re installing 2.0.x on RHEL 8.6?

---

<div class="post-metadata">

**Author:** ![brad.traver](https://avatars.discourse-cdn.com/v4/letter/b/82dd89/32.png) [@brad.traver](https://discourse.openondemand.org/u/brad.traver)\
**Post date:** [May 26, 2023, 3:39pm UTC](https://discourse.openondemand.org/t/dex-not-starting-with-ssl/2719/11 "2023-05-26T15:39:08Z")

</div>

I’m running 3.0 on RHEL 8.6. It was installed from the yum repository.

---

<div class="post-metadata">

**Author:** ![jeff.ohrstrom](https://sea1.discourse-cdn.com/flex015/user_avatar/discourse.openondemand.org/jeff.ohrstrom/32/136_2.png) [@jeff.ohrstrom](https://discourse.openondemand.org/u/jeff.ohrstrom)\
**Post date:** [May 26, 2023, 3:45pm UTC](https://discourse.openondemand.org/t/dex-not-starting-with-ssl/2719/12 "2023-05-26T15:45:26Z")

</div>

It’s looking for `rails-5.2.8.1` which is the version of rails for 2.0. OOD 3.0 uses rails-6.1. So it’s unclear to me why it’s searching for rails 5.2.

Can you do a spot check on this file?

```auto
[johrstrom ~()] head -n 5 /var/www/ood/apps/sys/dashboard/Gemfile
source 'https://rubygems.org'

# Bundle edge Rails instead: gem 'rails', github: 'rails/rails'
gem 'rails', '6.1.7.3'

```

---

<div class="post-metadata">

**Author:** ![brad.traver](https://avatars.discourse-cdn.com/v4/letter/b/82dd89/32.png) [@brad.traver](https://discourse.openondemand.org/u/brad.traver)\
**Post date:** [May 26, 2023, 3:51pm UTC](https://discourse.openondemand.org/t/dex-not-starting-with-ssl/2719/13 "2023-05-26T15:51:07Z")

</div>

This is the output of that command:

```auto
[root@login1 ~]# head -n 5 /var/www/ood/apps/sys/dashboard/Gemfile
source 'https://rubygems.org'

# Bundle edge Rails instead: gem 'rails', github: 'rails/rails'
gem 'rails', '5.2.8.1'

```

This is the output of `yum list installed | grep ondemand`:

```auto
[root@login1 ~]# yum list installed | grep ondemand
cjose.x86_64 0.6.1-1.el8 @ondemand-web
mod_auth_openidc.x86_64 2.4.5-1.el8 @ondemand-web
ondemand.x86_64 3.0.1-1.el8 @ondemand-web
ondemand-apache.x86_64 3.0.0-1.el8 @ondemand-web
ondemand-dex.x86_64 2.32.0-1.el8 @ondemand-web
ondemand-gems-3.0.1-1.x86_64 3.0.1-1.el8 @ondemand-web
ondemand-nginx.x86_64 1.20.2-1.p6.0.14.ood3.0.0.el8 @ondemand-web
ondemand-nodejs.x86_64 3.0.0-1.el8 @ondemand-web
ondemand-passenger.x86_64 6.0.14-1.ood3.0.0.el8 @ondemand-web
ondemand-release-web.noarch 3.0-1 @@commandline
ondemand-ruby.x86_64 3.0.0-1.el8 @ondemand-web
ondemand-runtime.x86_64 3.0.0-1.el8 @ondemand-web

```

---

<div class="post-metadata">

**Author:** ![jeff.ohrstrom](https://sea1.discourse-cdn.com/flex015/user_avatar/discourse.openondemand.org/jeff.ohrstrom/32/136_2.png) [@jeff.ohrstrom](https://discourse.openondemand.org/u/jeff.ohrstrom)\
**Post date:** [May 26, 2023, 4:08pm UTC](https://discourse.openondemand.org/t/dex-not-starting-with-ssl/2719/14 "2023-05-26T16:08:31Z")

</div>

OK that is really strange. You say this was a fresh install - no other OOD installation was here previously?

I just tried this - pulled the rpm down, unpacked it and checked that file and confirmed the RPM package has the correct file.

```auto
wget https://yum.osc.edu/ondemand/3.0/web/el8/x86_64/ondemand-3.0.0-1.el8.x86_64.rpm
rpm2cpio ondemand-3.0.0-1.el8.x86_64.rpm | cpio -idmv
head -n 5 var/www/ood/apps/sys/dashboard/Gemfile

```

Do you have a mount or something on top of that directory? Something on your system has overwritten the ondemand-3.0 RPM with the ondemand-2.0 package.

---

<div class="post-metadata">

**Author:** ![brad.traver](https://avatars.discourse-cdn.com/v4/letter/b/82dd89/32.png) [@brad.traver](https://discourse.openondemand.org/u/brad.traver)\
**Post date:** [May 26, 2023, 4:14pm UTC](https://discourse.openondemand.org/t/dex-not-starting-with-ssl/2719/15 "2023-05-26T16:14:57Z")

</div>

It was a fresh install but I copied over our previous configuration files so I didn’t have to set everything up from scratch. The two directories I copied were `/etc/ood/config` and `/var/www/ood/apps/sys`

---

<div class="post-metadata">

**Author:** ![jeff.ohrstrom](https://sea1.discourse-cdn.com/flex015/user_avatar/discourse.openondemand.org/jeff.ohrstrom/32/136_2.png) [@jeff.ohrstrom](https://discourse.openondemand.org/u/jeff.ohrstrom)\
**Post date:** [May 26, 2023, 4:27pm UTC](https://discourse.openondemand.org/t/dex-not-starting-with-ssl/2719/16 "2023-05-26T16:27:54Z")

</div>

> [@brad.traver](#):
>
> `/var/www/ood/apps/sys`

Do not copy or modify this directory.

`/etc/ood/config` is fine, packages won’t update/overwrite these files, but they do overwrite files in `/var/www/ood/apps`.

---

<div class="post-metadata">

**Author:** ![brad.traver](https://avatars.discourse-cdn.com/v4/letter/b/82dd89/32.png) [@brad.traver](https://discourse.openondemand.org/u/brad.traver)\
**Post date:** [May 26, 2023, 4:43pm UTC](https://discourse.openondemand.org/t/dex-not-starting-with-ssl/2719/17 "2023-05-26T16:43:31Z")

</div>

Should I put my apps in /etc/ood/config/apps then?

---

<div class="post-metadata">

**Author:** ![jeff.ohrstrom](https://sea1.discourse-cdn.com/flex015/user_avatar/discourse.openondemand.org/jeff.ohrstrom/32/136_2.png) [@jeff.ohrstrom](https://discourse.openondemand.org/u/jeff.ohrstrom)\
**Post date:** [May 26, 2023, 5:18pm UTC](https://discourse.openondemand.org/t/dex-not-starting-with-ssl/2719/18 "2023-05-26T17:18:37Z")

</div>

Sorry - I forgot that’s where other apps are too.

You can add _new_ directories into `/var/www/ood/apps/sys`, but you shouldn’t overwrite/change what the RPM provides.

So yes, you can copy your _other_ applications into this directory, just don’t copy the applications (directories) that the RPM ships.

---

<div class="post-metadata">

**Author:** ![brad.traver](https://avatars.discourse-cdn.com/v4/letter/b/82dd89/32.png) [@brad.traver](https://discourse.openondemand.org/u/brad.traver)\
**Post date:** [May 26, 2023, 5:22pm UTC](https://discourse.openondemand.org/t/dex-not-starting-with-ssl/2719/19 "2023-05-26T17:22:29Z")

</div>

Should I just reinstall the ondemand package to restore those directories?

---

<div class="post-metadata">

**Author:** ![jeff.ohrstrom](https://sea1.discourse-cdn.com/flex015/user_avatar/discourse.openondemand.org/jeff.ohrstrom/32/136_2.png) [@jeff.ohrstrom](https://discourse.openondemand.org/u/jeff.ohrstrom)\
**Post date:** [May 26, 2023, 5:23pm UTC](https://discourse.openondemand.org/t/dex-not-starting-with-ssl/2719/20 "2023-05-26T17:23:06Z")

</div>

Yes that should take care of it.

[Next page](https://discourse.openondemand.org/t/dex-not-starting-with-ssl/2719.md?page=2)