systemd: how to create and manage services

You bring the app up on the VPS with nohup node server.js &, close the SSH session, and sleep easy. In the morning the service is down: the process died overnight and nothing brought it back, or the SSH session dropped and took everything with it. Worst case: the machine rebooted for a kernel update and your app simply didn't come back. nohup and screen were built to keep a command alive while you're not watching, not to manage a production service. That's what systemd is for, and it's already installed on your distro.

This article shows how to turn an application into a properly managed service: one that restarts on its own, comes up at boot, and has centralized logging. If at some point the machine runs out of space because of a misconfigured log, it's worth reading later how to investigate low disk space on Linux — a full journald is a common cause.

Why systemd over nohup or screen

nohup and screen/tmux don't monitor the process. If it crashes, it stays dead. They don't come up at boot. They have no structured logging. They don't manage startup ordering dependencies. systemd does all of it: restarts on failure, integrates with boot, centralizes logs in journald, and lets you express that your service depends on the network or the database being up.

The trade is one-time: you write a unit file and never think about keeping the process alive by hand again.

Anatomy of a .service file

A unit lives in /etc/systemd/system/<name>.service and has three sections:

• [Unit] — metadata and dependencies: description, what must be up first.
• [Service] — how to run it: the command, the user, the restart policy, environment variables.
• [Install] — when enabled at boot, which "target" it belongs to.

Here's a real example for a Node application:

[Unit]
Description=Quick Tools API
After=network-online.target
Wants=network-online.target

[Service]
Type=simple
User=deploy
Group=deploy
WorkingDirectory=/opt/quicktools
EnvironmentFile=/opt/quicktools/.env
ExecStart=/usr/bin/node /opt/quicktools/server.js
Restart=on-failure
RestartSec=5
StandardOutput=journal
StandardError=journal

[Install]
WantedBy=multi-user.target

Every line here solves a problem nohup left open: runs as a non-root user, reads env from a file, restarts on failure with a 5-second wait, and comes up at boot.

Restart=: the policy that separates a service from a script

The Restart= directive defines when systemd brings the process back:

• no — never restarts (default). It's effectively a nohup.
• on-failure — restarts only if the process exits with a non-zero code or is killed by a signal. This is what you want in most cases.
• always — restarts even on a clean exit (code 0). Useful for services that should never terminate, but it masks intentional-exit bugs.

Combine it with RestartSec= so you don't get an instant restart loop when the service crashes at startup. And if you want a ceiling, StartLimitIntervalSec + StartLimitBurst in [Unit] stop systemd from restarting a broken service forever.

journalctl: where the logs went

Since the example sends stdout/stderr to journal, all logging goes to journald. The day-to-day commands:

# service logs, most recent first
journalctl -u quicktools.service -n 100 --no-pager

# follow in real time (like tail -f)
journalctl -u quicktools.service -f

# today only, error priority
journalctl -u quicktools.service --since today -p err

No more log file lost in /var/log or redirected to nohup.out. Everything is indexed by unit, by time, and by priority. The price is managing the journal's size — hence the connection to low disk space.

enable vs start: the classic confusion

The two commands look identical and do different things:

• systemctl start quicktools — starts it now, in this session. Doesn't survive a reboot.
• systemctl enable quicktools — registers it to come up at the next boot. Doesn't start it now.

You almost always want both. The shortcut is systemctl enable --now quicktools: enables at boot and starts immediately. Forgetting the enable is the number-one reason for "it worked, I rebooted the machine, and it vanished."

Reading status and diagnosing

systemctl status quicktools is the first command when something goes wrong:

systemctl status quicktools.service

It shows whether it's active (running), failed, or inactive, the PID, how long it's been up, and the last few log lines. If you see failed, go straight to journalctl -u for the stack trace.

Common gotchas

Three things that trip up almost everyone the first time:

• daemon-reload: after editing the .service, run systemctl daemon-reload before restarting the service. Without it systemd keeps using the cached old version and you'll swear your change did nothing.
• WorkingDirectory: if the application reads files by relative path, set WorkingDirectory explicitly. systemd starts in / by default, not in the binary's directory, and relative paths break.
• Environment variables: systemd does not load your ~/.bashrc or any .env automatically. Use Environment= for one-off variables or EnvironmentFile= pointing at a file. PATH is minimal too — that's why ExecStart uses an absolute path to the binary.

Frequently asked questions

What's the difference between enable and start in systemd?

start starts the service immediately in the current session and doesn't survive a reboot. enable registers the service to come up automatically at the next boot, but doesn't start it now. Use systemctl enable --now <service> to do both at once.

Why didn't my change to the .service file take effect?

You probably forgot systemctl daemon-reload. systemd keeps units cached in memory and only rereads the files when you tell it to reload. Run daemon-reload and then restart the service every time you edit the unit.

Should I use Restart=always or Restart=on-failure?

on-failure in most cases: it restarts only when the process exits with an error or is killed by a signal, and respects a clean exit. always restarts even on exit code 0, which is useful for services that should never terminate, but masks situations where the process exited on purpose.

Does systemd load my .env automatically?

No. It doesn't read ~/.bashrc or any .env on its own. You need to declare EnvironmentFile=/path/.env in the [Service] section, or define variables one by one with Environment=.

What systemd leaves in your workflow

Swapping nohup for systemd is half an hour of work that pays off the first night the service crashes and brings itself back without waking you. Write the unit with the right WorkingDirectory and EnvironmentFile, choose Restart=on-failure with RestartSec, run enable --now, and use journalctl -u -f when you need to look. Remember daemon-reload every time you edit the file. From then on your service comes up at boot, survives crashes, and has centralized logs — three things nohup & will never give you.