What Is systemd?
systemd is the system and service manager used by most modern Linux distributions (Ubuntu, CentOS, Fedora, Debian, etc.). It starts processes at boot, manages service lifecycles, and collects logs. It replaced the sequential boot process of SysVinit with parallel boot support, resulting in faster startup times.
Here are the core components of systemd.
| Component | Role |
|---|---|
| systemctl | CLI tool for controlling services |
| journalctl | Log viewing tool |
| unit files | Service definition files (.service, .timer, etc.) |
| target | Service groups (replaces runlevels) |
Service State Management
Use the systemctl command to start, stop, restart services and check their status.
# Check service status
sudo systemctl status nginx
# ● nginx.service - A high performance web server
# Loaded: loaded (/lib/systemd/system/nginx.service; enabled)
# Active: active (running) since Mon 2026-02-19 09:00:00 KST
# Main PID: 1234 (nginx)
# Start / stop / restart service
sudo systemctl start nginx
sudo systemctl stop nginx
sudo systemctl restart nginx
# Reload config without restarting the process
sudo systemctl reload nginx
# Enable / disable auto-start on boot
sudo systemctl enable nginx
sudo systemctl disable nginx
# Enable and start immediately
sudo systemctl enable --now nginx
enable registers auto-start on boot, while start starts the service immediately. When registering a new service, use enable --now to handle both in one command.
Writing Custom Service Files
Here’s an example of registering a Node.js application as a systemd service. Service files are created in the /etc/systemd/system/ directory with the .service extension.
# /etc/systemd/system/my-app.service
[Unit]
# Service description
Description=My Node.js Application
# Start after network is up
After=network.target
# Start after PostgreSQL service
Wants=postgresql.service
[Service]
# Service type (simple: the process itself is the service)
Type=simple
# Execution user/group
User=deploy
Group=deploy
# Working directory
WorkingDirectory=/opt/my-app
# Load environment variables file
EnvironmentFile=/opt/my-app/.env
# Execution command
ExecStart=/usr/bin/node /opt/my-app/dist/server.js
# Restart policy (restart on abnormal exit)
Restart=on-failure
# Wait time before restart (seconds)
RestartSec=5
# Maximum file descriptor count
LimitNOFILE=65535
# Send stdout/stderr to journal
StandardOutput=journal
StandardError=journal
# Tag for identification in journal
SyslogIdentifier=my-app
[Install]
# Include in multi-user.target (starts during normal boot)
WantedBy=multi-user.target
After writing the service file, register it in the following order.
# Reload service file changes
sudo systemctl daemon-reload
# Enable service + start immediately
sudo systemctl enable --now my-app.service
# Check status
sudo systemctl status my-app.service
# ● my-app.service - My Node.js Application
# Loaded: loaded (/etc/systemd/system/my-app.service; enabled)
# Active: active (running) since Mon 2026-02-19 09:10:00 KST
# Main PID: 5678 (node)
# Verify service file syntax
sudo systemd-analyze verify /etc/systemd/system/my-app.service
daemon-reload must be run every time you modify a service file. Changes are not applied without this command.
Key Service Section Options
Options for fine-grained control over service behavior.
| Option | Value | Description |
|---|---|---|
| Type | simple | The process itself is the service (default) |
| Type | forking | Daemon-style (parent process forks then exits) |
| Type | oneshot | Runs once then exits (for scripts, etc.) |
| Type | notify | Service notifies systemd when ready |
| Restart | no | No restart (default) |
| Restart | on-failure | Restart only on abnormal exit |
| Restart | always | Restart for any reason |
| RestartSec | seconds | Wait time before restart |
| TimeoutStartSec | seconds | Start timeout |
| TimeoutStopSec | seconds | Stop timeout |
Viewing Logs with journalctl
systemd stores all service logs in a unified journal. Use journalctl to query them.
# Specific service logs (latest 50 lines)
sudo journalctl -u my-app.service -n 50
# Stream logs in real time (like tail -f)
sudo journalctl -u my-app.service -f
# Today's logs only
sudo journalctl -u my-app.service --since today
# Logs within a specific time range
sudo journalctl -u my-app.service \
--since "2026-02-19 09:00" \
--until "2026-02-19 12:00"
# Filter error logs only (priority: emerg, alert, crit, err, warning, notice, info, debug)
sudo journalctl -u my-app.service -p err
# Logs since last boot only
sudo journalctl -u my-app.service -b
# Output in JSON format (for parsing)
sudo journalctl -u my-app.service -o json-pretty -n 5
# Check and clean journal disk usage
sudo journalctl --disk-usage
sudo journalctl --vacuum-size=500M
sudo journalctl --vacuum-time=7d
The -f option is useful for viewing real-time logs during incident response. Filtering with -p err helps quickly find problems in large volumes of logs.
systemd Timers
Using systemd timers instead of cron provides benefits like integrated journal logging, dependency management, and retry on execution failure. A timer consists of a pair: a .timer file and a .service file.
Here’s an example that runs a backup script daily at 3:00 AM.
# /etc/systemd/system/backup.service
[Unit]
Description=Daily Database Backup
[Service]
Type=oneshot
User=deploy
ExecStart=/opt/scripts/backup.sh
# Execution time limit (30 minutes)
TimeoutStartSec=1800
# /etc/systemd/system/backup.timer
[Unit]
Description=Run backup daily at 3 AM
[Timer]
# Run daily at 3:00 AM
OnCalendar=*-*-* 03:00:00
# If server was off, run missed executions immediately on startup
Persistent=true
# Random delay of 0-5 minutes (prevents multiple timers from running simultaneously)
RandomizedDelaySec=300
[Install]
WantedBy=timers.target
# Enable + start timer
sudo systemctl daemon-reload
sudo systemctl enable --now backup.timer
# List registered timers
sudo systemctl list-timers --all
# NEXT LEFT LAST PASSED UNIT ACTIVATES
# Tue 2026-02-20 03:00:00 KST 17h left - - backup.timer backup.service
# Manually trigger timer (for testing)
sudo systemctl start backup.service
# Check timer execution logs
sudo journalctl -u backup.service --since today
OnCalendar expressions are more readable than cron. Formats like Mon *-*-* 09:00:00 (every Monday at 9 AM) and *-*-01 00:00:00 (first day of every month at midnight) are also supported.
Debugging Services
Here’s the order to follow when a service fails to start.
# 1. Check status — verify Active state and error messages
sudo systemctl status my-app.service
# 2. Check full logs — logs from the start attempt onward
sudo journalctl -u my-app.service --no-pager
# 3. Verify service file syntax
sudo systemd-analyze verify /etc/systemd/system/my-app.service
# 4. Check dependency graph
sudo systemctl list-dependencies my-app.service
# 5. Analyze boot time bottlenecks
sudo systemd-analyze blame
# 3.456s my-app.service
# 2.123s postgresql.service
# 1.234s nginx.service
Common issues and their solutions:
| Symptom | Cause | Solution |
|---|---|---|
code=exited, status=203 | ExecStart path error | Check executable path, verify with which |
code=exited, status=217 | User does not exist | Verify with id username |
activating (auto-restart) looping | Service exits immediately and restarts | Check logs, inspect environment variables/config files |
inactive (dead) | Not enabled | Run systemctl enable --now |
Practical Tips
- Service file locations: Custom services go in
/etc/systemd/system/, while package manager-installed services are in/lib/systemd/system/. To modify a packaged service, don’t edit the original — create an override file withsystemctl edit nginx.serviceinstead. - Restart=always vs on-failure: Use
alwaysfor services that must stay running like web servers. Useon-failurefor batch jobs where a normal exit is meaningful. - Timers vs cron: systemd timers are recommended for new projects. They offer integrated journal logging,
Persistent=trueto catch up on missed runs, and dependency management — all of which provide better manageability than cron. - Security hardening: Options like
ProtectSystem=strict,ProtectHome=true, andNoNewPrivileges=truecan restrict a service’s filesystem access. Use these actively for production services. - Log management: By default, the journal is reset on each boot. Creating the
/var/log/journal/directory enables persistent storage. Add settings likeSystemMaxUse=1Gin/etc/systemd/journald.confto manage disk space.