Introduction
Linux administration is the discipline of keeping servers available, secure, and predictable — from the filesystem layer up through users, services, and the network stack. Whether you run RHEL, Ubuntu, or Debian, the same core subsystems apply: systemd for service management, a POSIX-style permission model, logical volume management for storage, and a consistent set of diagnostic tools.
This guide walks through the exact skill set a production Linux administrator uses daily — filesystem hierarchy, users and permissions, services and logging, networking and SSH, disk and LVM management, process control, and the troubleshooting instincts that come from having broken (and fixed) real servers. It closes with the tips that matter most for the RHCSA exam, since the exam essentially certifies this exact skill set.
Who this is for
- Engineers moving from a single workstation into managing production servers.
- Anyone preparing for the RHCSA (EX200) certification.
- DevOps engineers who need Linux fundamentals to support CI/CD and cloud work.
Prerequisites
You should have the following before working through the commands in this guide:
- Terminal access to a Linux machine — a VM (VirtualBox/VMware/UTM) or a cloud instance.
- sudo or root access on that machine.
- A RHEL 9 / Rocky Linux 9 VM if you intend to follow the RHCSA-specific sections.
- Basic comfort navigating a shell (cd, ls, cat) — this guide builds from there.
Distribution differences
Commands are shown for both Debian-based (Ubuntu/Debian, usingapt) and Red Hat-based (RHEL/Rocky/CentOS, using dnf) systems where package management differs. Core concepts — permissions, systemd, LVM — are identical across both families.Theory
Everything is a file
Linux represents devices, sockets, and processes as files under /dev, /proc, and /sys, not just documents under /home. A single, consistent read/write/permission model applies to all of them — this is why tools like chmod and file redirection work uniformly across the entire system.
The Filesystem Hierarchy Standard (FHS)
Every major distribution follows the same top-level layout:
| Path | Purpose |
|---|---|
/etc | System-wide configuration files |
/var | Variable data — logs, mail, spool queues |
/home | Per-user home directories |
/usr | Installed software, libraries, binaries |
/opt | Third-party / self-contained application installs |
/proc | Virtual filesystem exposing kernel & process state |
/boot | Kernel image, initramfs, and bootloader files |
/dev | Device nodes |
The boot process, conceptually
Firmware (BIOS/UEFI) hands off to a bootloader (GRUB2), which loads the kernel and an initial RAM filesystem (initramfs). The kernel then starts systemd as PID 1, which brings the system up through a sequence of targets— systemd's replacement for old-style runlevels.
Architecture
The diagram below shows the boot sequence from power-on to a usable login shell:
Firmware → GRUB2 → Kernel → initramfs → systemd → target → login
Targets vs. runlevels
multi-user.target is the modern equivalent of runlevel 3 (CLI, networked), and graphical.target is the equivalent of runlevel 5. Check the current default with systemctl get-default.Installation & Initial Setup
The first actions on any freshly provisioned server are the same regardless of distribution: set the hostname, sync time, update packages, and create a non-root administrative user.
# Update package index and upgrade
sudo apt update && sudo apt upgrade -y
# Set hostname
sudo hostnamectl set-hostname web01
# Sync time via systemd-timesyncd
sudo timedatectl set-ntp true
timedatectl status
# Create an admin user and grant sudo
sudo adduser deploy
sudo usermod -aG sudo deployDisable password SSH before exposing to the internet
Once key-based SSH access works for your new user (see the Configuration section), disable root login and password authentication insshd_config before this host is reachable from the public internet.Configuration
Users and groups
User accounts live in /etc/passwd (identity), hashed passwords in /etc/shadow (root-readable only), and group membership in /etc/group.
# Create a user with a home directory and specific shell
sudo useradd -m -s /bin/bash jsmith
# Set/reset password
sudo passwd jsmith
# Add an existing user to a secondary group
sudo usermod -aG docker jsmith
# Create a group
sudo groupadd deployers
# Lock and delete a user (keep or remove home dir)
sudo usermod -L jsmith
sudo userdel -r jsmithPermissions, ownership, and special bits
Every file has an owner, a group, and three permission triads (owner/group/other), each representing read (4), write (2), and execute (1).
# Symbolic and octal chmod
chmod u+x deploy.sh
chmod 750 /var/www/app
# Change owner and group
sudo chown jsmith:deployers /var/www/app -R
# View permissions
ls -l /var/www/app
# Default umask (subtracted from 666/777 for new files/dirs)
umask
umask 027- SUID (4000)— a binary runs with the file owner's privileges (e.g.
passwdruns as root even when invoked by a normal user). - SGID (2000)— new files in a directory inherit the directory's group; useful for shared team directories.
- Sticky bit (1000) — in a shared directory (like
/tmp), only the file owner can delete their own files.
# Shared team directory: group-writable, new files inherit group, no cross-deletes
sudo chmod 2775 /srv/shared
sudo chmod +t /srv/shared
# ACLs for permissions beyond owner/group/other
sudo setfacl -m u:jsmith:rwx /srv/shared
getfacl /srv/sharedSSH configuration
Harden /etc/ssh/sshd_config once key-based auth is confirmed working:
PermitRootLogin no
PasswordAuthentication no
PubkeyAuthentication yes
X11Forwarding no
MaxAuthTries 3
AllowUsers deploy jsmith# Generate a key pair on your local machine
ssh-keygen -t ed25519 -C "jsmith@laptop"
# Copy the public key to the server
ssh-copy-id jsmith@web01
# Apply sshd config changes
sudo sshd -t && sudo systemctl reload sshdScheduling: cron and systemd timers
Crontab syntax is five time fields followed by the command:
| Field | Range |
|---|---|
| Minute | 0-59 |
| Hour | 0-23 |
| Day of month | 1-31 |
| Month | 1-12 |
| Day of week | 0-6 (Sun-Sat) |
# Edit the current user's crontab
crontab -e
# Run a backup script every day at 2:30 AM
30 2 * * * /usr/local/bin/backup.sh >> /var/log/backup.log 2>&1
# List and remove
crontab -l
crontab -rPrefer systemd timers for new work
Timers integrate withjournalctl, support dependency ordering, and survive missed runs via Persistent=true — worth adopting over cron for new services.[Unit]
Description=Run backup daily
[Timer]
OnCalendar=*-*-* 02:30:00
Persistent=true
[Install]
WantedBy=timers.targetCommands
systemctl — service management
sudo systemctl start nginx
sudo systemctl stop nginx
sudo systemctl restart nginx
sudo systemctl reload nginx
sudo systemctl enable --now nginx # start now + boot
sudo systemctl disable nginx
sudo systemctl status nginx
sudo systemctl mask nginx # hard-block starting
systemctl list-units --type=service --state=runningjournalctl — the systemd log
journalctl -u nginx # logs for one unit
journalctl -u nginx -f # follow live
journalctl --since "1 hour ago"
journalctl -p err -b # errors since last boot
journalctl --disk-usage
sudo journalctl --vacuum-time=7d # trim retained logsProcess management
ps aux | grep nginx
top # or: htop
kill -15 1234 # graceful (SIGTERM)
kill -9 1234 # force (SIGKILL)
nice -n 10 ./heavy-job.sh
renice -n 5 -p 1234
jobs; bg; fg # background/foreground job controlNetworking
ip addr show
ip route show
nmcli device status
nmcli connection up eth0
ss -tulnp # listening sockets
sudo firewall-cmd --add-service=http --permanent # RHEL
sudo firewall-cmd --reload
sudo ufw allow 443/tcp # Ubuntu
sudo ufw enableDisk and LVM management
lsblk # view block devices
df -h # filesystem usage
du -sh /var/log/* # directory sizes
sudo fdisk /dev/sdb # partition a disk (or parted)
sudo mkfs.xfs /dev/sdb1 # create a filesystem
sudo mount /dev/sdb1 /data
echo '/dev/sdb1 /data xfs defaults 0 0' | sudo tee -a /etc/fstab
# LVM: physical volume -> volume group -> logical volume
sudo pvcreate /dev/sdc
sudo vgcreate data_vg /dev/sdc
sudo lvcreate -L 20G -n data_lv data_vg
sudo mkfs.xfs /dev/data_vg/data_lv
# Extend a logical volume online
sudo lvextend -L +10G /dev/data_vg/data_lv
sudo xfs_growfs /data # xfs; use resize2fs for ext4Examples
Two common end-to-end tasks, shown as full sequences:
Onboard a new engineer with sudo + SSH key access
useradd -m -s /bin/bash newdev, add to thewheel/sudogroup, create~/.ssh/authorized_keyswith their public key at mode600, and confirm login withssh newdev@hostbefore disabling password auth.Replace a cron backup job with a systemd timer
Write the backup logic into a oneshot service unit, create a matching.timerunit withOnCalendar, runsystemctl daemon-reload, thenenable --now backup.timerand verify withsystemctl list-timers.
Screenshots
htop — live process and resource view
Screenshot placeholder
df -h and lsblk output after extending a logical volume
Screenshot placeholder
Real World Example
Scenario: a production API server starts returning 500 errors and monitoring shows the disk at 100%.
Confirm and scope the problem
df -hconfirms/varis full.du -sh /var/* | sort -rh | headnarrows it down to/var/log.Free space immediately
journalctl --vacuum-size=200Mand truncate/rotate the offending app log to restore headroom without deleting files apps still have file handles open on (which wouldn't free space).Fix the root cause
Add alogrotatepolicy for the application's log file so it never grows unbounded again.Prevent recurrence
Add a disk-usage alert threshold in monitoring (see the Nagios guide) well before 100%, and if the volume is LVM-backed, extend it proactively withlvextend+xfs_growfs.
Common Issues
Service won't start
Runsystemctl status <service> first, then journalctl -u <service> -xe for the full error trace — the status summary is often truncated.Permission denied on a file you own
Check for a restrictive parent directory permission or an SELinux context mismatch (ls -Z, restorecon -Rv /path) before assuming the mode bits are wrong.Locked out of SSH after a config change
Never close your existing SSH session while testingsshd_config changes — validate with sudo sshd -t and reload in a second terminal so the current session stays as a fallback.System won't boot after editing /etc/fstab
A bad fstab entry drops the boot into emergency mode. Boot into rescue mode or usemount -o remount,rw / from the emergency shell, fix the line, then mount -a to validate before rebooting.Disk shows full but du doesn't add up
A process is likely holding a deleted file open. Find it withlsof +L1 and restart the owning process to actually release the space.Best Practices
- Never operate as root directly — use a named user with
sudoand full audit trail. - Automate repeatable setup with Ansible (see the Ansible guide) instead of manual server-by-server changes.
- Keep
/var,/home, and/on separate logical volumes so one filling up doesn't take down the whole system. - Centralize logs off-host (rsyslog forwarding or an ELK/Wazuh stack) so a crashed disk doesn't take the evidence with it.
- Patch on a schedule, not reactively — track CVEs for the exact package versions you run.
Security Hardening
- SSH — key-only auth, no root login, non-default port optional,
fail2banto throttle brute force. - SELinux/AppArmor — leave enforcing/enabled; fix context/policy issues rather than disabling mandatory access control.
- Firewall — default-deny with
firewalld/ufw, explicitly allow only required ports. - auditd — enable for tracking access to sensitive files (
/etc/shadow, sudoers) on regulated systems. - Least privilege — scope
sudoersentries to specific commands where possible instead of blanketALL=(ALL).
Interview Questions
Cheat Sheet
# --- Services ---
systemctl status|start|stop|restart|enable|disable <unit>
journalctl -u <unit> -f
# --- Users/Groups ---
useradd -m -s /bin/bash <user> && passwd <user>
usermod -aG <group> <user>
# --- Permissions ---
chmod 750 <path> ; chown user:group <path> -R
# --- Disk/LVM ---
df -h ; lsblk ; du -sh *
pvcreate|vgcreate|lvcreate|lvextend
# --- Process ---
ps aux | grep <name> ; top ; kill -15 <pid>
# --- Network ---
ip addr ; ss -tulnp ; firewall-cmd --add-service=<svc> --permanent
# --- Logs ---
journalctl --since "1 hour ago" ; tail -f /var/log/syslogSummary
Solid Linux administration comes down to a handful of layers you now have the vocabulary and commands for: the filesystem hierarchy and permissions model, systemd for services and boot, journald for diagnostics, cron/timers for scheduling, LVM for flexible storage, and a hardened SSH/firewall configuration for access control. These are the same fundamentals tested by RHCSA and used daily in production — practice them on a disposable VM until the commands are muscle memory.
Resources
- Red Hat Enterprise Linux documentation —
docs.redhat.com - Linux man-pages project —
man7.org - The Linux kernel archives —
kernel.org - systemd documentation —
freedesktop.org/software/systemd/man - Arch Wiki (distro-agnostic depth on almost every topic) —
wiki.archlinux.org