Not able to reconnect to interactive desktops

Hi all,

Able to start interactive application / desktop without issue. If I disconnect / close out my browser, login go to “My interactive sessions”, click button “launch desktop”. I get “New connection has been rejected with reason: Authentication failed”. Try again, go to “My interactive sessions” page, refresh the page and try again same error, until 4th try I get “New connection has been rejected with reason: Too many
authentication failures. Connections temporarily blocked”.

Running on:
ondemand 2.0.34
Rocky Linux 8
selinux currently in permissive

I’ve seen few posts similar to this issue but none with a workaround or resolution.

already bumped pun_max_retries: 25

Looking for suggestions,

Regards,
Jesse Waters

Add additional information:
The usl is not being updated between re-launches.

1st Time I connect
https://www.mysite.loc/pun/sys/dashboard/noVNC-1.1.0/vnc.html?utf8=✓&autoconnect=true&path=rnode%2Fnode1.mysite.loc%2F63353%2Fwebsockify&resize=remote&password=YnJRDPA8&compressionsetting=9&qualitysetting=9&commit=Launch+Desktop+-+Ondemand+Shared+node

When I close out of my webbrowser and login and try to relaunch the desktop the url password is still the same. To get around “grep ^password connection.yml” and replace the new random password in the url and I can reconnect.

How is the url generated and password slurped in between re-launches?

Hi Jesse.

Thanks for the post. I’ll look into this today, and let you know what I find.

Thanks,
-gerald

Hey Jesse.

I believe this will solve your problem. Please see the following discourse topic.

Thank you for the response. This is not the issue I am experiencing.

After clicking launch button the url password is not getting updated with current content in connection.yml. After every successful connection, the connection.yml is updated with a new random password.

alittle more info
when I reboot the open ondemand web server and try to re-launch an interactive application it works. What ever process pulls in the current password from connection.yml.

restarting http and nginx_clean/restart web server from web-ui, does not pull in the current password.

This is not a software issue w/open ondemand. Its my underlying shared storage.
nfs fronted beegfs (bad idea)

1 Like

@jesse.waters
We experienced the same issue because our OnDemand node was VM and jobs were
launched over GPFS compute. We had the same issue with connections.yml not updating/syncing
GPFS vs NFS mounted GPFS. After trying everything, we decided to switch the OnDemand node
to have IB and mount GPFS directly which solved the issue.

But I am curious to see if you figured out the underlying shared storage issue and was it related to NFS?

Thanks!
Bhushan