OOD behind Citrix NetScaler - noVNC fails to connect to server

Get Help
1

Hello,

I am trying to set up an Open OnDemand portal with a Citrix NetScaler serving what will be the public-facing IP address. I have enabled SSL on port 443, and all appeared to be working without issue until we pointed the CNAME for our portal to the NetScaler. We can still connect and authenticate to the portal interface itself, however when we start a remote desktop on one of the nodes, we are unable to launch it in a new tab - the new session attempts to connect for a short time, but then fails with a noVNC “Failed to connect to server” error. Our ood_portal.yml file used to generate our ood-portal.conf looks like this:

servername: hpc.myschool.edu
ssl:

  • ‘SSLCertificateFile “/opt/rh/httpd24/root/etc/httpd/ssl/hpc_myschool_edu_shpc7wb01_2019-09-11.crt”’
  • ‘SSLCertificateKeyFile “/opt/rh/httpd24/root/etc/httpd/ssl/hpc_myschool_edu_shpc7wb01_2019-09-11.key”’
    auth:
  • ‘SSLRequireSSL’
  • “AuthType shibboleth”
  • “ShibRequestSetting requireSession true”
  • “RequestHeader set REMOTE_USER %{REMOTE_USER}s”
  • “RequestHeader edit* Cookie “(^shibsession[^;](;\s)?|;\s*shibsession[^;]*)” “””
  • “RequestHeader unset Cookie “expr=-z %{req:Cookie}””
  • “Require valid-user”
  • “Require shib-attr affiliation member@myschool.edu”
    logout_redirect: /Shibboleth.sso/Logout?return=https%3A%2F%2Fidp.myschool.edu%2Fidp%2Fprofile%2FLogout
    user_map_cmd: “/opt/ood/ood_auth_map/bin/ood_auth_map.regex --regex=’^(\w+)’”
    host_regex: ‘[\w]+.myschool.edu’
    node_uri: ‘/node’
    rnode_uri: ‘/rnode’

Can anyone tell me if there is anything “special” that needs to be added to the configuration to make this work? Or is there anything specific that needs to be configured on the NetScaler?

Thank you,

Richard

2

Below is the LocationMatch directive that’s giving you problems. Note how it proxies from the ondemand host directly to the node, probably on a strange port.

This seems like a time out. Notice how this proxy rule goes directly to the compute node from the ondemand host. That’s the first thing I would check, to see if you have connectivity between the two. Next time you test, jot down what port you’re trying to connect to (you’ll see it in the URL), and try a telnet to that port from the ondemand host. Off the top, as I recall we try to use a port above 2000 and less than ~64000. As an example I just got the port 57960.

  # Reverse "relative" proxy traffic to backend webserver through IP sockets:
  #
  #     https://myschool.edu:443/rnode/HOST/PORT/index.html
  #     #=> http://HOST:PORT/index.html
  #
  <LocationMatch "^/rnode/(?<host>[\w.-]+\.osc\.edu)/(?<port>\d+)(?<uri>/.*|)">
    AuthType openid-connect
    Require valid-user

    # ProxyPassReverse implementation
    Header edit Location "^([^/]+//[^/]+)|(?=/)" "/rnode/%{MATCH_HOST}e/%{MATCH_PORT}e"

    # ProxyPassReverseCookieDomain implemenation
    Header edit* Set-Cookie ";\s*(?i)Domain[^;]*" ""

    # ProxyPassReverseCookiePath implementation
    Header edit* Set-Cookie ";\s*(?i)Path[^;]*" ""
    Header edit  Set-Cookie "^([^;]+)" "$1; Path=/rnode/%{MATCH_HOST}e/%{MATCH_PORT}e"

    LuaHookFixups node_proxy.lua node_proxy_handler
  </LocationMatch>

If that’s not it, If you have connectivity between the two, then I’ll have to lookup what we’re doing with that domain cookie.

3

Thanks for your prompt reply. As it turns out, my browser was the culprit here. I was using Firefox, but when I switched to Chrome, I found that everything works. I have now also tried it with MS Edge and IE. MS Edge and Chrome seem to do just fine. IE fails miserably - both the shell access and the interactive desktop. Firefox (69.0.1), strangely enough, fails for me, but seems to work for one of my co-workers. Another co-worker with a Mac reports that he is unable to launch any apps using Safari 13, and that although the shell works, it prompts him for a password, even though he has already logged in using DUO 2FA. (REMOTE_USER is supposed to be passed in the request header by Shibboleth, but apparently that isn’t working when he uses Safari.)

Is there anything documented regarding accommodation for the various browsers? I’m not terribly concerned if IE doesn’t work at this point, but I’m pretty sure a lot of our users are MacBook owners and would prefer to use Safari.

Thank you!

4

Oh! OK yea, IE is really the only browser we actively don’t want to support.

As to Safari, I’m quite sure we want to have feature parity with other browsers. There are edge cases that come up here and there but basically our position on it is “we want to support most browsers with semi-current version” and certainly Safari 13 falls into that category.

I’m an active FF user, so that’s very strange that VNC doesn’t work for you.

Feel free to open bug tickets on the main repo or against the application specifically.

5

In regards to Safari, you do have to explicitly specify that it is supported in a configuration setting after you switch to using another authentication method besides BasicAuth. I haven’t tested the latest Safari 13, but I believe Safari 11 and before had problems with BasicAuth and websockets. We have been lax in our cross browser testing recently, and we have some accessibility and responsiveness problems we need to address (hopefully in the next 6 months).

6

The issue with Safari 13 has been resolved. I set the DISABLE_SAFARI_BASIC_AUTH_WARNING to 1 as per the instructions in the link provided by efranz. (Thank you!) I think the bigger issue, though, was that the user who complained was using zsh as his default shell. The particular server we installed OOD on is just a test server and is very bare-bones – we had not installed the zsh package. Once we did so, Safari 13 worked without issue for that particular user.

As for Firefox, I’m just befuddled. I’ve tried v69.0.1 and v69.0.2 on my own desktop, and neither works, however other users have sent word that both versions are working for them. It seems to be an issue with the browser on my particular machine. Are there any particular browser settings that I need to be investigating? Nothing looks out of the ordinary to me, except that for our OOD site I have created an exception to allow pop-ups, though I don’t even know if that’s necessary.

Thanks.