It was our load balancer (Citrix Netscaler). It was not configured to allow websockets (ws and wss).
This is what I did to debug. I looked at developer tools request headers when it made the wss request. I then simulated the request using curl on the command-line. Here is an example curl wss request with it rejecting the wss (openssl errno 104):
curl -vk -i -N -H "Host: carcaccount.usc.edu" -H "Origin: https://carcaccount.usc.edu" -H "Connection: Upgrade" -H "Upgrade: websocket" 'https://carcaccount.usc.edu/pun/sys/shell/discovery.usc.edu?csrf=Y6GDrzuL-dv6WrczQXBsax7mwOCasZ1Yp8e8'
> GET /pun/sys/shell/ssh/discovery.usc.edu?csrf=Y6GDrzuL-dv6WrczQXBsax7mwOCasZ1Yp8e8 HTTP/1.1
> Host: carcaccount.usc.edu
> User-Agent: curl/7.66.0
> Accept: */*
> Connection: Upgrade
> Upgrade: websocket
> Origin: https://carcaccount.usc.edu
>
* STATE: DO => DO_DONE handle 0x800083108; line 1756 (connection #0)
* STATE: DO_DONE => PERFORM handle 0x800083108; line 1877 (connection #0)
* OpenSSL SSL_read: SSL_ERROR_SYSCALL, errno 104
* Marked for [closure]: Transfer returned error
* multi_done
* Closing connection 0
* The cache now contains 0 members
* Expire cleared (transfer 0x800083108)
curl: (56) OpenSSL SSL_read: SSL_ERROR_SYSCALL, errno 104
I noticed the wss request never made it to apache httpd. At that point I did suspect the LB configuration. I found out that I had to make special configuration changes on our Citrix Netscaler LB to support ws and wss protocols. After that the wss curl request worked! And of course OOD shell worked too!
@jeff.ohrstrom - Thank you for your assistance and guess as the LB as the problem!