Linux host adapter, ssh issues

I’m having ssh issues while using Linux host adapter and looking for tips on how to debug my issue.

When I try to execute the ssh command with all the options manually from the OOD host to the target host it appears to be working:

[kmuriki@ood ~]$ hostname

[kmuriki@ood ~]$ ssh -t -o BatchMode=yes -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no kmuriki@n0003.testbed0 /usr/bin/env bash

[kmuriki@n0003 ~]$ hostname

But the same ssh command appears to be disconnecting when OOD executes

OOD logs below:

App 7947 output: [2020-12-14 16:10:25 -0800 ] INFO “execve = [{}, “ssh”, “-t”, “-o”, “BatchMode=yes”, “-o”, “UserKnownHostsFile=/dev/null”, “-o”, “StrictHostKeyChecking=no”, “kmuriki@n0003.testbed0”, “/usr/bin/env”, “bash”]”
App 7947 output: [2020-12-14 16:10:26 -0800 ] INFO "method=POST path=/pun/sys/dashboard/batch_connect/dev/jupyter-interactive/session_contexts format=html controller=BatchConnect::SessionContextsController action=create status=302 duration=1445.91 view=0.00

Syslog messages from the host n0003.testbed0

Dec 14 16:10:25 n0003.testbed0 sshd[29013]: Accepted publickey for kmuriki from port 58016 ssh2: DSA SHA256:fP5ARPAN3vihLme3gNxuDRK94/n8p3qRYXpfegcTdrU
Dec 14 16:10:26 n0003.testbed0 kmuriki: n0003.testbed0 kmuriki: module load vim
Dec 14 16:10:26 n0003.testbed0 sshd[29015]: Received disconnect from port 58016:11: disconnected by user
Dec 14 16:10:26 n0003.testbed0 sshd[29015]: Disconnected from port 58016

You can see ssh was successful and loaded the module which I have in my $HOME/.bashrc but immediately disconnects. Not able to figure out why the disconnects are happening. Any tips on how to debug this ?


I would say follow these directions for trouble shooting. It sounds like “The job just exists with no errors.” is the right description of the problem?

Yeh this is ‘job just exits with no errors’ and already documented troubleshooting tip helped me. Turns out the location of my singularity container needs to be changed and I got it working now. Thanks.