Pick the right shape first
Ask one question: does your script finish, or does it wait? That answer decides nearly everything else.
| Your script | Examples | Run it as |
|---|---|---|
| Does a job, then ends | A scraper, a price alert, a nightly report | A timer |
| Loops with a long sleep in the middle | while True: with time.sleep(3600) | A timer, after you delete the loop |
| Has to keep listening | A sensor logger, a websocket feed, a message queue | An always-on service |
Why rewrite the sleeping loop? An hourly job that crashes on one bad web page just runs again next hour. A loop that crashes takes all its future runs with it.
You may have heard of cron, the old scheduler. It's fine for a one-line job. This guide uses systemd timers because a slow run can't overlap the next one, a run missed while the server was off can be made up, and the output reaches a log with no extra setup.
Find it a home
A home Linux computer that stays on works, and it's free. When you can't count on that, a cloud server gives you a small Linux machine with root access (full control of it) in New York. Size S, with 1 GB of memory, is a fair start for a script that fetches pages and writes a file. If your script is really a small website, our web hosting runs Python apps on your own domain instead.
Choose Ubuntu (22.04 or later) when you order, then sign in as the ubuntu user with your SSH key. At home, any current Ubuntu works, once you've run sudo apt install -y openssh-server. The first lines switch your firewall on and let SSH through, nothing else. If your script receives connections, open its port, for example sudo ufw allow 8080/tcp, and only if it needs to be reachable. The steps also make an account just for the script, plus a venv, a private folder for its Python packages. You don't have to activate it: calling the Python inside is enough. Your script goes in /opt/watcher/app. Passwords and API keys go in /etc/watcher.env, as lines like API_KEY=abc123. Ordinary accounts can't read it.
cd /opt
sudo sed -i 's/^IPV6=.*/IPV6=yes/' /etc/default/ufw
sudo ufw allow OpenSSH && sudo ufw --force enable
sudo apt update && sudo apt install -y python3-venv
sudo useradd --system --create-home --home-dir /opt/watcher --shell /usr/sbin/nologin watcher
sudo install -d -o watcher -g watcher /opt/watcher/app /opt/watcher/data
sudo install -m 600 /dev/null /etc/watcher.env
sudo -u watcher python3 -m venv /opt/watcher/venv
sudo -u watcher /opt/watcher/venv/bin/pip install requestsAdd whatever else your script imports to that last line, and keep requests if you'll use the check-in below. Copy your script to your home folder with scp or any SFTP app, then put it in place with sudo install -o watcher -g watcher ~/check.py /opt/watcher/app/. These steps call it check.py. If yours has another name, rename it or change the ExecStart line below to match.
Schedule a job with a timer
A timer needs two small files: one says what to run, the other says when. Listener? Read the next section first.
sudo tee /etc/systemd/system/watcher.service > /dev/null <<'EOF'
[Unit]
Description=Price check
Wants=network-online.target
After=network-online.target
[Service]
Type=oneshot
User=watcher
WorkingDirectory=/opt/watcher/app
EnvironmentFile=/etc/watcher.env
Environment=PYTHONUNBUFFERED=1
ExecStart=/opt/watcher/venv/bin/python check.py
TimeoutStartSec=15min
EOF
sudo tee /etc/systemd/system/watcher.timer > /dev/null <<'EOF'
[Unit]
Description=Run the price check every hour
[Timer]
OnCalendar=hourly
Persistent=true
[Install]
WantedBy=timers.target
EOF
sudo systemctl daemon-reload
sudo systemctl start watcher.service && sudo systemctl enable --now watcher.timerThe last line runs the script once by hand, and turns the timer on only if that run works. If it fails, read why with journalctl -u watcher -n 50, fix the problem, put the script in place again and rerun that last line. A few lines above do quiet work:
TimeoutStartSec=15minmatters most. A job like this has no time limit unless you set one, and while a run is stuck, the timer won't start another. One frozen web request could halt your script for good with no sign of trouble. Set it a bit above your slowest normal run.Persistent=truemakes up a missed run once the server is back after being off. The two network lines at the top make that run wait until the server is online.PYTHONUNBUFFERED=1makesprint()reach the log straight away. Otherwise Python holds output back when nobody's at a terminal, and your log looks empty.
Check the schedule with systemctl list-timers watcher.timer.
Keep a listener running
From the block above, write only the first file (watcher.service, down to the first EOF), and skip the timer file and the last line. If you already turned the timer on, run sudo systemctl disable --now watcher.timer first. Then edit the file with sudo nano /etc/systemd/system/watcher.service: remove Type=oneshot and TimeoutStartSec, and add the lines below to [Service], plus an [Install] section with WantedBy=multi-user.target. Run sudo systemctl daemon-reload. Last comes sudo systemctl enable --now watcher, which starts it.
Restart=always
RestartSec=30Keep RestartSec. The default wait is a tenth of a second, so a script that dies at once hits systemd's limit of 5 starts in 10 seconds and then stays off.
always also covers a script that ends cleanly, say when its connection closes, which on-failure would leave off.
Restarting only helps if your script actually dies. One stuck on a network call looks perfectly healthy. The Requests docs warn that calls without a timeout can hang, so always write requests.get(url, timeout=30), and do the same in any other library.
Get the time right
Your server's clock may not be set to your time zone, and timedatectl will show you what it uses. Rather than changing it, name the zone in the schedule. OnCalendar=*-*-* 07:00:00 Europe/London means 7 in the morning in London all year, through clock changes.
To change the schedule, open the file with sudo nano /etc/systemd/system/watcher.timer, edit the OnCalendar line, and apply it with sudo systemctl daemon-reload && sudo systemctl restart watcher.timer. After editing watcher.service, sudo systemctl daemon-reload is enough for a timer job; a listener also needs sudo systemctl restart watcher.
Inside your script, save times in UTC with datetime.now(timezone.utc), and convert to local time only when you show them to a person. Python's zoneinfo module does that. Our suggestion: avoid scheduling anything in the hour when your clocks change, since that hour can be skipped or happen twice.
Stop the logs filling the disk
With print() or Python's logging going to the screen, there's nothing to set up. Read the output with journalctl -u watcher -n 50. The journal limits itself, by default to 10% of the disk and never more than 4 GB.
Writing your own log file? Use logging.handlers.RotatingFileHandler with maxBytes and backupCount, which starts a new file at a size you choose and deletes the oldest.
Nothing trims what your script collects, so have it delete old files in /opt/watcher/data on a schedule you choose. That folder exists only on that machine, so copy what matters somewhere else regularly.
Find out when it stops
Plenty of people leave this out. A script can't report that it has stopped, and neither can a machine that's switched off. So turn the question around: have the script check in after each good run, and let something outside raise the alarm when the check-ins stop.
- Create a check at a monitoring service that accepts simple web pings. Healthchecks.io is one, and its docs cover this pattern for scheduled jobs.
- Set the expected gap to your schedule, plus some slack.
- Open your settings file with
sudo nano /etc/watcher.envand add a line withPING_URL=and your check's address. - Add
import osandimport requestsat the top of your script, and this line at the very end, after the work is done:requests.get(os.environ['PING_URL'], timeout=10). Then put the script in place again. - On a timer job, run
sudo systemctl stop watcher.timerfor a while, and make sure the alert really reaches your phone, then start it again. An alarm you've never heard isn't an alarm.
For a listener, ping whenever real work gets done, so a frozen script goes quiet, and wrap the ping in try: / except requests.RequestException as e: pass so a failed ping can't stop the script; run sudo systemctl restart watcher after editing the env file, and test the alarm with sudo systemctl stop watcher, starting it again afterwards. Stuck on any of this? Our support page lists how to reach us.
Change the script later
Copy the new version in the way you did the first time. A timer job picks it up on its next run; a listener needs sudo systemctl restart watcher. Save your package list while things work with sudo -u watcher sh -c '/opt/watcher/venv/bin/pip freeze > /opt/watcher/requirements.txt'. Python's docs treat a venv as disposable, so after a big Ubuntu upgrade, delete it with sudo rm -rf /opt/watcher/venv, run cd /opt and the python3 -m venv line again, then sudo -u watcher /opt/watcher/venv/bin/pip install -r /opt/watcher/requirements.txt.