systems-guide/systemd.md

Configuring, running and logging with systemd

Why you want this

systemd provides an out-of-the box linux native way of running and logging for your services.

You could as well use things like tmux, screen, pm2 or just a background task, but they dont cut all the cases. systemd provides a way to declaratively define your app startup, re-startup, service dependencies and pipe its outputs in a structured manner to journalctl where they are stored in the most efficient manner.

How to create a new service that runs persistently even if server restarts

First create your new service file

sudo nvim /etc/systemd/system/$SERVICE_NAME.service

then paste and edit this template:

[Unit]
Description=Service name
After=network.target
Wants=network.target

[Service]
# Set the working directory where your binary resides
WorkingDirectory=/usr/local/bin/YOUR_PATH
# Absolute path to your binary
ExecStart=/usr/local/bin/YOUR_PATH/target/release/BINARY_NAME

# Ensure the service restarts on failure
Restart=always
RestartSec=5

# Optional: prevent rapid restart loops
StartLimitIntervalSec=60
StartLimitBurst=3

# Direct stdout and stderr to the system journal
StandardOutput=journal
StandardError=journal
SyslogIdentifier=YOUR_SERVICE

# Optional: increase file descriptor limits if needed
LimitNOFILE=4096

[Install]
WantedBy=multi-user.target

then enable and start your service

sudo systemctl enable $YOUR_SERVICE
sudo systemctl start $YOUR_SERVICE

it will now keep running even after restarts.

Logging your server

A better guide: click here

To get all logs simply do

journalctl -u your_service

To follow logs append -f option.

journalctl -u your_service -f

To query logs by time use --since and --until

# it uses a human readable way. You can operate with hours, seconds etc.
journalctl -u your_service --since "2 hours ago"
# no timestamps, you need the below format to do precise times
journalctl -u your_service --since "2015-06-26 23:15:00" --until "2015-06-26 23:20:00"

To get precise number of logs, use -n option.

journalctl -u your_service -n 100

If you want to get latest N messages, use also the -r option to reverse the log order to get latest messages.

# Get last 100 logs
journalctl -u your_service -r -n 100