# Configure OOD in Kubernetes with Dex

**URL:** <https://discourse.openondemand.org/t/configure-ood-in-kubernetes-with-dex/4016>\
**Category:** Get Help\
**Tags:** question\
**Created:** [February 5, 2025, 5:01am UTC](https://discourse.openondemand.org/t/configure-ood-in-kubernetes-with-dex/4016 "2025-02-05T05:01:40Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![irfanardi3001](https://sea1.discourse-cdn.com/flex015/user_avatar/discourse.openondemand.org/irfanardi3001/32/2593_2.png) [@irfanardi3001](https://discourse.openondemand.org/u/irfanardi3001)\
**Post date:** [February 5, 2025, 5:01am UTC](https://discourse.openondemand.org/t/configure-ood-in-kubernetes-with-dex/4016/1 "2025-02-05T05:01:40Z")

</div>

I have a successfull authentication login page with ondemand-Dex in with my FreeIPA server running on KVM in advance. But, I don’t know the OIDC Client and secret to deploy hooks to user as on the documentation mention.

Before we discuss it about that, I had some problems. I was tested the configuration with this command, it’s fine:

> [root@master dashboard]# bin/rake -T test:jobs  
> rake test:jobs # Test all clusters  
> rake test:jobs:my\_k8s\_cluster # Test the cluster: my\_k8s\_cluster

but when I wanna continue the test like in documentation with this command, I had some errors:

> [root@master dashboard]# sudo su $USER -c ‘source /opt/ood/ondemand/enable; bin/rake test:jobs:my\_k8s\_cluster RAILS\_ENV=production’  
> Testing cluster ‘my\_k8s\_cluster’…  
> Submitting job…  
> rake aborted!  
> TypeError: no implicit conversion of Symbol into Integer (TypeError)
> 
> ```
> native_data[:container][:supplemental_groups] = supplemental_groups(native_data[:container][:supplemental_groups])
> ^^^^^^^^^^
> 
> ```
> 
> /opt/ood/ondemand/root/usr/share/gems/3.3/ondemand/4.0.0-1/gems/ood\_core-0.27.0/lib/ood\_core/job/adapters/kubernetes/batch.rb:232:in `generate_id_yml' /opt/ood/ondemand/root/usr/share/gems/3.3/ondemand/4.0.0-1/gems/ood_core-0.27.0/lib/ood_core/job/adapters/kubernetes/batch.rb:46:in `submit’  
> /opt/ood/ondemand/root/usr/share/gems/3.3/ondemand/4.0.0-1/gems/ood\_core-0.27.0/lib/ood\_core/job/adapters/kubernetes.rb:57:in `submit' /var/www/ood/apps/sys/dashboard/lib/tasks/test.rake:31:in `block (4 levels) in \<top (required)\>’  
> Tasks: TOP =\> test:jobs:my\_k8s\_cluster  
> (See full trace by running task with --trace)

For detail my\_k8s\_cluster is:

> * * *
> 
> v2:  
> metadata:  
> title: “UNNES AI SERVER”
> 
> # you may not want a login section. There may not be a login node
> 
> # for your kuberenetes cluster
> 
> login:  
> host: 172.16.1.38  
> job:  
> adapter: “kubernetes”  
> config\_file: “~/.kube/config”  
> cluster: “ood-prod”  
> context: “ood-prod”  
> bin: “/usr/bin/kubectl”  
> username\_prefix: “prod-”  
> namespace\_prefix: “user-”  
> all\_namespaces: false  
> auto\_supplemental\_groups: false  
> server:  
> endpoint: 172.16.1.38  
> cert\_authority\_file: “/etc/kubernetes/pki/ca.crt”  
> auth:  
> type: “oidc”  
> mounts:   
> batch\_connect:  
> ssh\_allow: false

Am I had a wrong configuration?  
I have an objective to setup interactove jupyter apps in my cluster to do.  
I’ll be thankyou for someone that has any advice for me?  
Many thanks

---

<div class="post-metadata">

**Author:** ![jeff.ohrstrom](https://sea1.discourse-cdn.com/flex015/user_avatar/discourse.openondemand.org/jeff.ohrstrom/32/136_2.png) [@jeff.ohrstrom](https://discourse.openondemand.org/u/jeff.ohrstrom)\
**Post date:** [February 5, 2025, 2:24pm UTC](https://discourse.openondemand.org/t/configure-ood-in-kubernetes-with-dex/4016/2 "2025-02-05T14:24:23Z")

</div>

Hi and welcome!

Yea I don’t think that rake task is going to work for Kuberenetes. Seems like you’re going to have to build the Jupyter application to get it working. Here’s an app we deploy on Kubernetes (you’ll see a Slurm toggle there too)

> **[GitHub - OSC/bc\_classroom\_jupyter: Jupyter OnDemand app for use in classrooms](https://github.com/OSC/bc_classroom_jupyter/)**
>
> Jupyter OnDemand app for use in classrooms

Maybe start with just using a busybox to be sure you can create pods and so on then move on from there.

---

<div class="post-metadata">

**Author:** ![irfanardi3001](https://sea1.discourse-cdn.com/flex015/user_avatar/discourse.openondemand.org/irfanardi3001/32/2593_2.png) [@irfanardi3001](https://discourse.openondemand.org/u/irfanardi3001)\
**Post date:** [February 6, 2025, 5:34am UTC](https://discourse.openondemand.org/t/configure-ood-in-kubernetes-with-dex/4016/3 "2025-02-06T05:34:39Z")

</div>

Thanks, I can see interactive dropdown contain classrom jupyter. But when I want to launch the app, I see the error message

> You are not a part of any classroom project. Restarting your web server in the help menu may fix the issue. You may also have to wait if you’ve recently been added to the project. Reach out to your instructor or contact support through the help menu.

I see this message is writen in the submit.yml.erb on the your github given to me. For the cluster configuration I try to follow your instruction on the github like this

cluster.d/cluster.yml

> * * *
> 
> v2:  
> metadata:  
> title: “Jupyter Cluster”  
> you may not want a login section. There may not be a login node for your kubernetes cluster  
> login:  
> host: 172.16.1.38  
> job:  
> adapter: “kubernetes”  
> config\_file: “~/.kube/config”  
> cluster: “ood-prod”  
> context: “ood-prod”  
> bin: “/usr/bin/kubectl”  
> username\_prefix: “prod-”  
> namespace\_prefix: “user-”  
> all\_namespaces: false  
> auto\_supplemental\_groups: false  
> server:  
> endpoint: 172.16.1.38  
> cert\_authority\_file: “/etc/kubernetes/pki/ca.crt”  
> auth:  
> type: “oidc”  
> mounts:   
> batch\_connect:  
> ssh\_allow: false  
> custom:  
> classrooms:  
> juypter:  
> OSU\_MATH\_123:  
> project: OSU1  
> OSU\_BIOLOGY\_234:  
> project: OSU2  
> OU\_CHEMISTRY\_234:  
> size: xlarge  
> hours: 6  
> project: OU1

am I forgot several step?  
I hope your advice

---

<div class="post-metadata">

**Author:** ![irfanardi3001](https://sea1.discourse-cdn.com/flex015/user_avatar/discourse.openondemand.org/irfanardi3001/32/2593_2.png) [@irfanardi3001](https://discourse.openondemand.org/u/irfanardi3001)\
**Post date:** [February 6, 2025, 10:51am UTC](https://discourse.openondemand.org/t/configure-ood-in-kubernetes-with-dex/4016/4 "2025-02-06T10:51:02Z")

</div>

I had a temporary dumb solution with marking comment at **submit.yml.erb**

> raise StandardError, err\_msg if classroom == ‘no\_classroom’

and

> ‘support’ =\> OodSupport::User.new(‘support’).home,

Because, now I think I don’t need the Class yet.

But, Now when I click launch button I encountered this problem:

 ![image](https://us1.discourse-cdn.com/flex015/uploads/osc/original/2X/a/ac0541d638afeea4046c2b6c8c69083d2f1d0130.png)

I modified this into **form.yml.erb**

> attributes:  
> cluster:  
> widget: “select”  
> options:  
> - [“kubernetes”,“kubernetes”]

the cluster name I got from

> kubectl config view  
> apiVersion: v1  
> clusters:
> 
> - cluster:  
> certificate-authority: /etc/kubernetes/pki/ca.crt  
> server: https://:6443  
> name: kubernetes

And this is my **cluster.d/my\_k8s\_cluster.yml**

> * * *
> 
> v2:  
> metadata:  
> title: “kubernetes”  
> login:  
> host: external-ip  
> job:  
> adapter: kubernetes  
> bin: “/usr/bin/kubectl”  
> all\_namespaces: false  
> auto\_supplemental\_groups: true  
> server:  
> endpoint: external-ip  
> cert\_authority\_file: “/etc/kubernetes/pki/ca.crt”  
> auth:  
> type: “managed”  
> mounts:   
> batch\_connect:  
> ssh\_allow: false

Can anybody give me advice for this problem to configure the cluster and jupyter?  
Thank you

---

<div class="post-metadata">

**Author:** ![jeff.ohrstrom](https://sea1.discourse-cdn.com/flex015/user_avatar/discourse.openondemand.org/jeff.ohrstrom/32/136_2.png) [@jeff.ohrstrom](https://discourse.openondemand.org/u/jeff.ohrstrom)\
**Post date:** [February 6, 2025, 2:11pm UTC](https://discourse.openondemand.org/t/configure-ood-in-kubernetes-with-dex/4016/5 "2025-02-06T14:11:08Z")

</div>

> [@irfanardi3001](#):
>
> cluster.d/my\_k8s\_cluster.yml

This is the cluster name that you need to use. In your `form.yml.erb` you reference a cluster called `kubernetes` which doesn’t exist. Your cluster is called `my_k8s_cluster` based off of the file name of the cluster.d file.

That app I linked may not be well suited for testing actually. It has a bit going on with access control and so on.

This one is simpler, maybe it’ll prove to be easier to modify.

> **[GitHub - OSC/bc\_k8s\_jupyter: Run a batch connect Jupyter in a kuberentes...](https://github.com/OSC/bc_k8s_jupyter)**
>
> Run a batch connect Jupyter in a kuberentes cluster

---

<div class="post-metadata">

**Author:** ![irfanardi3001](https://sea1.discourse-cdn.com/flex015/user_avatar/discourse.openondemand.org/irfanardi3001/32/2593_2.png) [@irfanardi3001](https://discourse.openondemand.org/u/irfanardi3001)\
**Post date:** [February 7, 2025, 9:09am UTC](https://discourse.openondemand.org/t/configure-ood-in-kubernetes-with-dex/4016/6 "2025-02-07T09:09:53Z")

</div>

I follow the simple one and I encountered this problem:

#### Failed to submit session with the following error:

error: error loading config file “/home/rocky95/.kube/config”: open /home/rocky95/.kube/config: permission denied

here I was modified the cluster.d/my\_k8s\_cluster.yml

> v2:  
> metadata:  
> title: “My K8s Cluster”  
> login:  
> host: “172.16.1.38”  
> job:  
> adapter: “kubernetes”  
> config\_file: “/home/rocky95/.kube/config”  
> cluster: “ood-prod”  
> context: “ood-prod”  
> bin: “/usr/bin/kubectl”  
> username\_prefix: “prod-”  
> namespace\_prefix: “user-”  
> all\_namespaces: false  
> auto\_supplemental\_groups: false  
> server:  
> endpoint: “172.16.1.38:6443”  
> cert\_authority\_file: “/etc/kubernetes/pki/ca.crt”  
> auth:  
> type: “oidc”  
> mounts:   
> batch\_connect:  
> ssh\_allow: false

I try to change the chmod config for /home/rocky95/.kube/config from chmod 600, 666, or even 755 and it’s still has the same problem

I also use the  
config\_file: ~/.kube/config  
and also encountered the same problem.

I also try to follow this [previous discussion](https://discourse.openondemand.org/t/problems-with-creating-jupyter-session-with-k8s-cluster-configuration/2196), follow the cluster configuration and can’t solve this problem.

---

<div class="post-metadata">

**Author:** ![jeff.ohrstrom](https://sea1.discourse-cdn.com/flex015/user_avatar/discourse.openondemand.org/jeff.ohrstrom/32/136_2.png) [@jeff.ohrstrom](https://discourse.openondemand.org/u/jeff.ohrstrom)\
**Post date:** [February 7, 2025, 2:28pm UTC](https://discourse.openondemand.org/t/configure-ood-in-kubernetes-with-dex/4016/7 "2025-02-07T14:28:14Z")

</div>

> [@irfanardi3001](#):
>
> error: error loading config file “/home/rocky95/.kube/config”: open /home/rocky95/.kube/config: permission denied

I assume you’re the `rocky95` user? I.e., you’re also this user in OnDemand? You kind of need to take errors at face value - meaning the OnDemand user (whomever that may be) cannot read that file. If you’re checking the permissions on the file, you may also need to check permissions on the directory or just try to replicate in a shell with the same user.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/flex015/uploads/osc/original/2X/b/bae70bd0ed39a3ae769c2108155f4cb3e9da8385.png) [@system](https://discourse.openondemand.org/u/system)\
**Post date:** [August 6, 2025, 2:28pm UTC](https://discourse.openondemand.org/t/configure-ood-in-kubernetes-with-dex/4016/8 "2025-08-06T14:28:49Z")

</div>

Closed on August 6, 2025, 2:28pm UTC

This topic was automatically closed 180 days after the last reply. New replies are no longer allowed.