Missing Home Directory Error

Get Help
1

Hello All,

I am sure it is something simple I am missing but been racking my head around “Missing Home Directory” error we are getting. We are new to ondemand and I had a build working before I left for vacation last week. When I came back this week and tried to log in I am getting…

" Home directory not found. Your home directory appears to be missing. The home directory mount may be unavailable, or your home directory may need to still be created. Please contact support for help and attempt to restart your web server by clicking below when the problem has been fixed."

I verified the home directory is mounted and have restarted services. I don’t see any errors in the logs. What am I missing?

Thanks in advance!

Nate

1 Like
2

Sounds like maybe your $HOME isn’t bound to your account. Here are the locations we’re looking for it. More specifically we’re using the ruby etc module to do it.

Locally it’s here

[jeff@localhost misc]$ grep jeff /etc/passwd
jeff:x:1000:1000:Jeff Ohrstrom:/home/jeff:/bin/bash

In LDAP I can find it here.

[johrstrom@server ~]$ ldapsearch -x cn=johrstrom  | grep home
homeDirectory: /users/PZS0714/johrstrom
1 Like
3

Thanks John! We connect to Active Directory but we override it with sssd settings since the value in Active Directory does not match our system. We do this with “override_homedir = /users/%u” option. It is working fine on the command line for users just not through ondemand. Should that work?

4

I don’t think that will work.

I’ve created this ticket to enable this use case becuase I don’t think we can handle this out of the box.

I can give you directions to hack this file, but it would be overwritten when you upgrade.

Since it may be your only choice, I’ll try to work something up as an example for you now.

5

Hacking the file is just fine for now. Thank you! I am not sure what changed as this did work two weeks in testing but we did reboot since then so probably some config was reloaded. We are using CAS too if that makes any difference but I do not think it does?

Nate

6

First I’m going to say I would no longer advise doing this. After spending a couple of hours on this I can get somethings to work and somethings not. The HOME environment variable does not seem to stick, I don’t know how to set it and keep it set forever. I can set it, but sure enough, it comes back to the original, so I’m not sure how to persist it.

I’d implore you instead to correct your LDAP. Indeed I don’t know how far we’ll get on the Github issue I’ve created given the pervasive nature of this variable.

Here are the changes you can make in these commits. I would first back up every file that you’re going to modify. Note this is on my fork, not the official repo.

/opt/ood/nginx_stage/lib/nginx_stage.rb
/opt/ood/nginx_stage/lib/nginx_stage/configuration.rb
/opt/ood/nginx_stage/lib/nginx_stage/user.rb
/etc/ood/config/nginx_stage.yml

I would copy the rb files from the Github branch source to the destinations above. Then manually add the config entry to the yml (you don’t need the comments).

Again, this is going to fail a whole bunch of areas where the home is just not set correctly. Sometimes it may just be a pain to navigate back to somewhere else or sometimes it may cause failure, but in either case your customers are not going to be very happy.

7

Hi John,

Thanks again for the help. Unfortunately, I cannot update Active Directory field with the home directory. We are mandated to use the Universities Central AD service for authentication and since numerous groups utilize AD, I can’t change the home directory location as it would affect other systems. I like the options mentioned in the support ticket you opened. Any chance those would be implemented?

Lastly, is there some debug mode I can turn on to see what home directory it actually thinks the user has set? I want to be sure it is trying to pull it from Active Directory.

Thanks!

Nate

8

@negregg would you log into the web host running OnDemand and run this command:

scl enable ondemand -- ruby -e "require 'etc'; puts [ENV['HOME'], Dir.home, Dir.home(ENV['USER']), File.expand_path('~'), Etc.getpwnam(ENV['USER']).dir]"

Are any of the lines the correct home directory?

9

@efranz
Surprisingly they are all ok … see following:

scl enable ondemand – ruby -e “require ‘etc’; puts [ENV[‘HOME’], Dir.home, Dir.home(ENV[‘USER’]), File.expand_path(’~’), Etc.getpwnam(ENV[‘USER’]).dir]”
/users/negregg
/users/negregg
/users/negregg
/users/negregg
/users/negregg

10

Just to give you a quicker/easier change, I’ve found I can get the same results as that branch with this approach.

Just create an /etc/ood/profile file and override your HOME variable. No need to modify any source files. This will have the same affect and it is much simpler and easier to maintain.

HOME="/users/$USER"
11

@jeff.ohrstrom Thanks! However, this did not fix my issue. I almost wonder if there is some other problem I am hitting? As the command @efranz sent me shows all the possible home directory locations being returned correctly and the /etc/ood/profile update did not seem to fix the issue. Is there a way to trace what is going on with OOD when the page is loading?

Thanks!

12

I have an update for you that may work now. I’ve figured out it was Passenger that wasn’t passing $HOME down to it’s sub processes’ and found the option to pass it.

Copy all the files changed in these commits. to these locations (backing up the original files as you go!!!)

  • /opt/ood/nginx_stage/lib/nginx_stage.rb
  • /opt/ood/nginx_stage/lib/nginx_stage/configuration.rb
  • /opt/ood/nginx_stage/lib/nginx_stage/user.rb
  • /etc/ood/config/nginx_stage.yml
  • /opt/ood/nginx_stage/lib/nginx_stage/nginx_stage/templates/pun.conf.erb

Here’s a little info just for posterity. One can see that the watchdog as the right HOME but core doesn’t.

# output from ps -elf
4 S ood        710     1  3  80   0 - 91569 -      20:17 ?        00:00:00 Passenger watchdog
0 S ood        713   710  4  80   0 - 372460 -     20:17 ?        00:00:00 Passenger core


[root@localhost /]# cat /proc/710/environ | tr '\0' '\n' | grep HOME
HOME=/users/$USER
[root@localhost /]# cat /proc/713/environ | tr '\0' '\n' | grep HOME
HOME=/home/ood
13

@jeff.ohrstrom thanks for the help. I believe I copied all the appropriate files into place but still having same issue. Some things to note:

  • templates/pun.conf.erb did not exist so i created the directory and file
  • After not working, I did edit ood_portal.yml and add user_home_dir: '/users/%{user}' but it did not fix the issue either.

Note, the environment var looks to be ok based on your ps example:

[root@lrp-trcvm001 config]#  cat /proc/115988/environ | tr '\0' '\n'|grep -i home
HOME=/users/negregg
[root@lrp-trcvm001 config]#  cat /proc/115982/environ | tr '\0' '\n'|grep -i home
HOME=/users/negregg

Nate

14

Sorry, that file path given was way off. It’s /opt/ood/nginx_stage/templates/pun.conf.erb is correct path. It should already exist.

It’s not ood_portal.yml you need to add it it it’s nginx_stage.yml.

Note that after you replace everything you’ll probably have to bounce httpd24 and your own PUN web server. That could be all you need though I’d double check the file list just to be sure.

Your ENV output there is good! But the first and initial hurdle seems to be this Etc business.

15

Typo on my end … i meant to say I edited nginx_stage.yml. So after placing pun.conf.erb in the right location I am getting this:

Error -- nginx: [emerg] invalid number of arguments in "passenger_env_var" directive in /var/lib/ondemand-nginx/config/puns/negregg.conf:58

16

Sorry it needs a ; on the end.

passenger_env_var HOME '/users/<%= user %>';

17

No problem. So that error is gone away not but unfortunately back to:

Home directory not found
Your home directory appears to be missing. The home directory mount may be unavailable, or your home directory may need to still be created. Please contact support for help and attempt to restart your web server by clicking below when the problem has been fixed.

18

Are you sure when you checked out that git repo you switched to that branch? As a sanity check you can do a quick diff on one of the files you swapped and backed up.

git clone https://github.com/johrstrom/ondemand.git
cd ondemand
git checkout override-home-dir
19

Yup … see below md5sums to verify (note: /etc/ood/config/nginx_stage.yml not listed due to modification which makes the md5sum mismatch).

4e66710d145efbdfbf7cdb82172aec79 lib/nginx_stage.rb
4e66710d145efbdfbf7cdb82172aec79 /opt/ood/nginx_stage/lib/nginx_stage.rb
5de9bf9aa46ff0d60b3913baaeb31443 lib/nginx_stage/configuration.rb
5de9bf9aa46ff0d60b3913baaeb31443 /opt/ood/nginx_stage/lib/nginx_stage/configuration.rb
ecec30df5c28b0da579da0f08b612857 lib/nginx_stage/user.rb
ecec30df5c28b0da579da0f08b612857 /opt/ood/nginx_stage/lib/nginx_stage/user.rb
713fe9730e91e9b7a0ef2039cfc81846 templates/pun.conf.erb
713fe9730e91e9b7a0ef2039cfc81846 /opt/ood/nginx_stage/templates/pun.conf.er

20

That is a bummer. I’m able to replicate the issue and the fix locally, so there must be something more to this. I cannot replicate LDAP locally so I must be missing something there.

Here’s where you can circumvent showing that page. That was put in as a sort of safety measure because who knows what sort of side affects a missing home directory may have.

/opt/ood/nginx_stage/lib/nginx_stage/views/pun_config_view.rb

    def missing_home_directory?
      false
      # comment this bit out and just return false
      # ! Dir.exist?(user.dir)
    end