I’m working on deploying Open OnDemand on a large, shared Slurm cluster.
Context:
It is one central Slurm cluster.
Authentication is multi-tenant, using an identity broker (e.g., Keycloak).
EduGAIN is supported — so users from different institutions (mostly students) can authenticate using their student credentials.
Authentication of local user via local FreeIPA is supported as well
The Challenge:
We need a way to automatically create home directories on the Slurm cluster when a new user logs in for the first time (via Open OnDemand or directly via SSH, if allowed).
So far, I’ve identified a few potential challenges:
I don’t want to pre-provision users manually.
I need a secure, scalable, and ideally event-driven way to provision a home directory on first login.
I’m looking for guidance or solutions — PAM? OOD hooks? Integration with Keycloak?
Any best practices or tools others use?
Thanks in advance!
I noticed this thread and I hope it is not too late, or if so maybe my approach will benefit someone in the community.
First, a custom mapfile contains a few lines which create a file to serve as the trigger to the inotify daemon:
Check if the user already exists
if id “$INPUT_USER” &>/dev/null; then
echo “$INPUT_USER”
else
Create the file that will trigger inotify to run the user creation script
touch /var/www/html/user_triggers/$INPUT_USER
fi
I deployed an inotify daemon as an ansible task to take action when triggered by the creation of the user file. It could potentially be adapted for your deployment and fit it in to your configuration management tool of choice.
# Directory to watch
WATCH_DIR="/var/www/html/user_triggers"
# Process each file in the directory
for FILE in "$WATCH_DIR"/*; do
if [ -f "$FILE" ]; then
# Read the username from the filename
USERNAME=$(basename "$FILE")
# If the username is in an @myinstitution.edu email address, trim it
if [[ "$USERNAME" =~ @myinstitution.edu$ ]]; then
USERNAME=${USERNAME%@myinstitution.edu}
fi
# Create the user
useradd -m "$USERNAME"
# Wait for the user to be created
while ! id "$USERNAME" &>/dev/null; do
sleep 1
done
# Remove the file after processing
rm -f "$FILE"
fi
done
mode: ‘0755’
owner: root
group: root
name: Create systemd service for inotify
ansible.builtin.copy:
dest: /etc/systemd/system/inotify-user.service
content: |
[Unit]
Description=Inotify User Creation Service
Also, I should have given a high-level breakdown of why the process lacks simple elegance:
The trigger file is created by the apache user
The inotify daemon runs as root so that it can create the local linux account with the home directory.
If your process only needs to create the home directory without creating a local linux user, you would replace the “useradd” with just a mkdir for the home directory itself, another mkdir for the .ssh subdirectory, a recursive chown, and a restorecon -v -R to fix the selinux labels
Ok, so another update to my last update. I just started out a deployment on a new authentication infrastructure with saml in which I am not using the user_map script and this seems like it could be a sensible solution that would work independently any choices you have had to make regarding authentication mechanisms.
You’d think the script would be obvious but there is a gotcha here. This is why I am logging the output to /var/log/home_dir.
Using environment variables did not seem like an option because the context of the user changes a couple of times
1. Our researcher logs into Apache
2. The “apache” account executes the pre-pun hook - but with sudo
3. The script is executed as root.
Instead, the script parses out the username from the command line arguments passed to it.
name: Create user home directory script
ansible.builtin.copy:
dest: /usr/local/bin/create_home_dir.sh
content: |
#!/bin/bash
# Setup logging
LOG_DIR="/var/log/home_dir"
LOG_FILE="$LOG_DIR/homedir.log"
# Create log directory if it doesn't exist
mkdir -p $LOG_DIR
chmod 755 $LOG_DIR
# Log function
log() {
echo "$(date '+%Y-%m-%d %H:%M:%S') - $1" >> $LOG_FILE
}
# Start logging
log "Script started"
log "Environment variables:"
env | sort >> $LOG_FILE
log "Command line arguments: $@"
# Get username from command line arguments
USERNAME=""
while [[ $# -gt 0 ]]; do
case $1 in
-u|--user)
USERNAME="$2"
shift 2
;;
-P|--pre-hook)
# Skip the pre-hook argument and its value
shift 2
;;
*)
shift
;;
esac
done
if [ -z "$USERNAME" ]; then
log "ERROR: Username not provided in command line arguments"
exit 1
fi
log "Username from command line: $USERNAME"
# Remove @uchicago.edu if present
USERNAME=${USERNAME%@uchicago.edu}
log "Final username after processing: $USERNAME"
if [ -z "$USERNAME" ]; then
log "ERROR: Username is empty after processing"
exit 1
fi
# Check if user exists
if ! id "$USERNAME" &>/dev/null; then
log "ERROR: User $USERNAME does not exist"
exit 1
fi
# Create the home directory with .ssh directory and set permissions and selinux labels
log "Creating home directory for $USERNAME"
if [ ! -d "/home/$USERNAME" ]; then
mkdir -p /home/$USERNAME
if [ $? -ne 0 ]; then
log "ERROR: Failed to create home directory"
exit 1
fi
log "Created new home directory"
else
log "Home directory already exists, ensuring proper setup"
fi
# Create .bashrc with proper content
log "Creating .bashrc"
cat > /home/$USERNAME/.bashrc << 'EOL'
# .bashrc created by create_home_dir.sh
# Source global definitions
if [ -f /etc/bashrc ]; then
. /etc/bashrc
fi
# User specific environment
if ! [[ "$PATH" =~ "$HOME/.local/bin:$HOME/bin:" ]]
then
PATH="$HOME/.local/bin:$HOME/bin:$PATH"
fi
export PATH
# Uncomment the following line if you don't like systemctl's auto-paging feature:
# export SYSTEMD_PAGER=
# User specific aliases and functions
if [ -d ~/.bashrc.d ]; then
for rc in ~/.bashrc.d/*; do
if [ -f "$rc" ]; then
. "$rc"
fi
done
fi
unset rc
EOL
# Create .bash_profile with proper content
log "Creating .bash_profile"
cat > /home/$USERNAME/.bash_profile << 'EOL'
# .bash_profile created by create_home_dir.sh
# Source .bashrc
. ~/.bashrc
EOL
# Create and set .ssh directory permissions
mkdir -p /home/$USERNAME/.ssh
chown $USERNAME:$USERNAME /home/$USERNAME/.ssh
chmod 700 /home/$USERNAME/.ssh
# Ensure all files and directories have correct ownership and permissions
log "Ensuring correct ownership and permissions"
chown -R $USERNAME:$USERNAME /home/$USERNAME
if [ $? -ne 0 ]; then
log "ERROR: Failed to set ownership"
exit 1
fi
chmod 700 /home/$USERNAME
if [ $? -ne 0 ]; then
log "ERROR: Failed to set home directory permissions"
exit 1
fi
# Set file permissions
chmod 644 /home/$USERNAME/.bashrc
chmod 644 /home/$USERNAME/.bash_profile
chmod 644 /home/$USERNAME/.bash_logout
# Log SELinux operations
log "Setting SELinux contexts"
# Set context for home directory
semanage fcontext -a -t user_home_dir_t "/home/$USERNAME(/.*)?" 2>> $LOG_FILE
restorecon -Rv /home/$USERNAME 2>> $LOG_FILE
# Set specific contexts for files
chcon -t user_home_t /home/$USERNAME/.bashrc 2>> $LOG_FILE
chcon -t user_home_t /home/$USERNAME/.bash_profile 2>> $LOG_FILE
chcon -t user_home_t /home/$USERNAME/.bash_logout 2>> $LOG_FILE
# Set context for .ssh directory
semanage fcontext -a -t ssh_home_t "/home/$USERNAME/.ssh" 2>> $LOG_FILE
restorecon -Rv /home/$USERNAME/.ssh 2>> $LOG_FILE
log "Script completed successfully"
name: Set SELinux context for log directory
ansible.builtin.file:
path: /var/log/home_dir
state: directory
mode: ‘0755’
owner: root
group: root
setype: var_log_t
Or if you are not using selinux:
name: Create user home directory script
ansible.builtin.copy:
dest: /usr/local/bin/create_home_dir.sh
content: |
#!/bin/bash
# Setup logging
LOG_DIR="/var/log/home_dir"
LOG_FILE="$LOG_DIR/homedir.log"
# Create log directory if it doesn't exist
mkdir -p $LOG_DIR
chmod 755 $LOG_DIR
# Log function
log() {
echo "$(date '+%Y-%m-%d %H:%M:%S') - $1" >> $LOG_FILE
}
# Start logging
log "Script started"
log "Environment variables:"
env | sort >> $LOG_FILE
log "Command line arguments: $@"
# Get username from command line arguments
USERNAME=""
while [[ $# -gt 0 ]]; do
case $1 in
-u|--user)
USERNAME="$2"
shift 2
;;
-P|--pre-hook)
# Skip the pre-hook argument and its value
shift 2
;;
*)
shift
;;
esac
done
if [ -z "$USERNAME" ]; then
log "ERROR: Username not provided in command line arguments"
exit 1
fi
log "Username from command line: $USERNAME"
# Remove @uchicago.edu if present
USERNAME=${USERNAME%@uchicago.edu}
log "Final username after processing: $USERNAME"
if [ -z "$USERNAME" ]; then
log "ERROR: Username is empty after processing"
exit 1
fi
# Check if user exists
if ! id "$USERNAME" &>/dev/null; then
log "ERROR: User $USERNAME does not exist"
exit 1
fi
# Create the home directory with .ssh directory and set permissions
log "Creating home directory for $USERNAME"
if [ ! -d "/home/$USERNAME" ]; then
mkdir -p /home/$USERNAME
if [ $? -ne 0 ]; then
log "ERROR: Failed to create home directory"
exit 1
fi
log "Created new home directory"
else
log "Home directory already exists, ensuring proper setup"
fi
# Create .bashrc with proper content
log "Creating .bashrc"
cat > /home/$USERNAME/.bashrc << 'EOL'
# .bashrc created by create_home_dir.sh
# Source global definitions
if [ -f /etc/bashrc ]; then
. /etc/bashrc
fi
# User specific environment
if ! [[ "$PATH" =~ "$HOME/.local/bin:$HOME/bin:" ]]
then
PATH="$HOME/.local/bin:$HOME/bin:$PATH"
fi
export PATH
# Uncomment the following line if you don't like systemctl's auto-paging feature:
# export SYSTEMD_PAGER=
# User specific aliases and functions
if [ -d ~/.bashrc.d ]; then
for rc in ~/.bashrc.d/*; do
if [ -f "$rc" ]; then
. "$rc"
fi
done
fi
unset rc
EOL
# Create .bash_profile with proper content
log "Creating .bash_profile"
cat > /home/$USERNAME/.bash_profile << 'EOL'
# .bash_profile created by create_home_dir.sh
# Source .bashrc
. ~/.bashrc
EOL
# Create and set .ssh directory permissions
mkdir -p /home/$USERNAME/.ssh
chown $USERNAME:$USERNAME /home/$USERNAME/.ssh
chmod 700 /home/$USERNAME/.ssh
# Ensure all files and directories have correct ownership and permissions
log "Ensuring correct ownership and permissions"
chown -R $USERNAME:$USERNAME /home/$USERNAME
if [ $? -ne 0 ]; then
log "ERROR: Failed to set ownership"
exit 1
fi
chmod 700 /home/$USERNAME
if [ $? -ne 0 ]; then
log "ERROR: Failed to set home directory permissions"
exit 1
fi
# Set file permissions
chmod 644 /home/$USERNAME/.bashrc
chmod 644 /home/$USERNAME/.bash_profile
chmod 644 /home/$USERNAME/.bash_logout
log "Script completed successfully"