Configure OOD in Kubernetes with Dex

Get Help
1

I have a successfull authentication login page with ondemand-Dex in with my FreeIPA server running on KVM in advance. But, I don’t know the OIDC Client and secret to deploy hooks to user as on the documentation mention.

Before we discuss it about that, I had some problems. I was tested the configuration with this command, it’s fine:

[root@master dashboard]# bin/rake -T test:jobs
rake test:jobs # Test all clusters
rake test:jobs:my_k8s_cluster # Test the cluster: my_k8s_cluster

but when I wanna continue the test like in documentation with this command, I had some errors:

[root@master dashboard]# sudo su $USER -c ‘source /opt/ood/ondemand/enable; bin/rake test:jobs:my_k8s_cluster RAILS_ENV=production’
Testing cluster ‘my_k8s_cluster’…
Submitting job…
rake aborted!
TypeError: no implicit conversion of Symbol into Integer (TypeError)

native_data[:container][:supplemental_groups] = supplemental_groups(native_data[:container][:supplemental_groups])
            ^^^^^^^^^^

/opt/ood/ondemand/root/usr/share/gems/3.3/ondemand/4.0.0-1/gems/ood_core-0.27.0/lib/ood_core/job/adapters/kubernetes/batch.rb:232:in generate_id_yml' /opt/ood/ondemand/root/usr/share/gems/3.3/ondemand/4.0.0-1/gems/ood_core-0.27.0/lib/ood_core/job/adapters/kubernetes/batch.rb:46:in submit’
/opt/ood/ondemand/root/usr/share/gems/3.3/ondemand/4.0.0-1/gems/ood_core-0.27.0/lib/ood_core/job/adapters/kubernetes.rb:57:in submit' /var/www/ood/apps/sys/dashboard/lib/tasks/test.rake:31:in block (4 levels) in <top (required)>’
Tasks: TOP => test:jobs:my_k8s_cluster
(See full trace by running task with --trace)

For detail my_k8s_cluster is:

v2:
metadata:
title: “UNNES AI SERVER”

you may not want a login section. There may not be a login node

for your kuberenetes cluster

login:
host: 172.16.1.38
job:
adapter: “kubernetes”
config_file: “~/.kube/config”
cluster: “ood-prod”
context: “ood-prod”
bin: “/usr/bin/kubectl”
username_prefix: “prod-”
namespace_prefix: “user-”
all_namespaces: false
auto_supplemental_groups: false
server:
endpoint: 172.16.1.38
cert_authority_file: “/etc/kubernetes/pki/ca.crt”
auth:
type: “oidc”
mounts:
batch_connect:
ssh_allow: false

Am I had a wrong configuration?
I have an objective to setup interactove jupyter apps in my cluster to do.
I’ll be thankyou for someone that has any advice for me?
Many thanks

2

Hi and welcome!

Yea I don’t think that rake task is going to work for Kuberenetes. Seems like you’re going to have to build the Jupyter application to get it working. Here’s an app we deploy on Kubernetes (you’ll see a Slurm toggle there too)

Maybe start with just using a busybox to be sure you can create pods and so on then move on from there.

3

Thanks, I can see interactive dropdown contain classrom jupyter. But when I want to launch the app, I see the error message

You are not a part of any classroom project. Restarting your web server in the help menu may fix the issue. You may also have to wait if you’ve recently been added to the project. Reach out to your instructor or contact support through the help menu.

I see this message is writen in the submit.yml.erb on the your github given to me. For the cluster configuration I try to follow your instruction on the github like this

cluster.d/cluster.yml

v2:
metadata:
title: “Jupyter Cluster”
you may not want a login section. There may not be a login node for your kubernetes cluster
login:
host: 172.16.1.38
job:
adapter: “kubernetes”
config_file: “~/.kube/config”
cluster: “ood-prod”
context: “ood-prod”
bin: “/usr/bin/kubectl”
username_prefix: “prod-”
namespace_prefix: “user-”
all_namespaces: false
auto_supplemental_groups: false
server:
endpoint: 172.16.1.38
cert_authority_file: “/etc/kubernetes/pki/ca.crt”
auth:
type: “oidc”
mounts:
batch_connect:
ssh_allow: false
custom:
classrooms:
juypter:
OSU_MATH_123:
project: OSU1
OSU_BIOLOGY_234:
project: OSU2
OU_CHEMISTRY_234:
size: xlarge
hours: 6
project: OU1

am I forgot several step?
I hope your advice

4

I had a temporary dumb solution with marking comment at submit.yml.erb

raise StandardError, err_msg if classroom == ‘no_classroom’

and

‘support’ => OodSupport::User.new(‘support’).home,

Because, now I think I don’t need the Class yet.

But, Now when I click launch button I encountered this problem:

image
image1402×440 37 KB

I modified this into form.yml.erb

attributes:
cluster:
widget: “select”
options:
- [“kubernetes”,“kubernetes”]

the cluster name I got from

kubectl config view
apiVersion: v1
clusters:

  • cluster:
    certificate-authority: /etc/kubernetes/pki/ca.crt
    server: https://:6443
    name: kubernetes

And this is my cluster.d/my_k8s_cluster.yml

v2:
metadata:
title: “kubernetes”
login:
host: external-ip
job:
adapter: kubernetes
bin: “/usr/bin/kubectl”
all_namespaces: false
auto_supplemental_groups: true
server:
endpoint: external-ip
cert_authority_file: “/etc/kubernetes/pki/ca.crt”
auth:
type: “managed”
mounts:
batch_connect:
ssh_allow: false

Can anybody give me advice for this problem to configure the cluster and jupyter?
Thank you

5

This is the cluster name that you need to use. In your form.yml.erb you reference a cluster called kubernetes which doesn’t exist. Your cluster is called my_k8s_cluster based off of the file name of the cluster.d file.

That app I linked may not be well suited for testing actually. It has a bit going on with access control and so on.

This one is simpler, maybe it’ll prove to be easier to modify.

6

I follow the simple one and I encountered this problem:

Failed to submit session with the following error:

error: error loading config file “/home/rocky95/.kube/config”: open /home/rocky95/.kube/config: permission denied

here I was modified the cluster.d/my_k8s_cluster.yml

v2:
metadata:
title: “My K8s Cluster”
login:
host: “172.16.1.38”
job:
adapter: “kubernetes”
config_file: “/home/rocky95/.kube/config”
cluster: “ood-prod”
context: “ood-prod”
bin: “/usr/bin/kubectl”
username_prefix: “prod-”
namespace_prefix: “user-”
all_namespaces: false
auto_supplemental_groups: false
server:
endpoint: “172.16.1.38:6443”
cert_authority_file: “/etc/kubernetes/pki/ca.crt”
auth:
type: “oidc”
mounts:
batch_connect:
ssh_allow: false

I try to change the chmod config for /home/rocky95/.kube/config from chmod 600, 666, or even 755 and it’s still has the same problem

I also use the
config_file: ~/.kube/config
and also encountered the same problem.

I also try to follow this previous discussion, follow the cluster configuration and can’t solve this problem.

7

I assume you’re the rocky95 user? I.e., you’re also this user in OnDemand? You kind of need to take errors at face value - meaning the OnDemand user (whomever that may be) cannot read that file. If you’re checking the permissions on the file, you may also need to check permissions on the directory or just try to replicate in a shell with the same user.