Running Podman containers with Systemd

One of the biggest things that I (and others I would assume) miss about docker when switching over to podman is that magic --restart=always flag. Basically that config flag tells the daemon to do a few things:

  1. Restart the container if it exits (duh)
  2. As a side-effect, it also makes the container start at boot

The second point which isn't really the meaning of the flag is probably the biggest piece of functionality that most docker users miss. e.g. "I rebooted and none of my containers came back, wtf >:("

So today we're going to go through using podman to generate systemd services so the containers get recreated at boot

For this you'll need podman installed - I think the functionality for this command was added long ago - so almost any podman will do.

Step 1: Getting the container configuration correct

This is probably the lengthiest part. Let's say you want to spin up an nginx container serving the files in the current directory. One would probably run the command like this:

podman run -it --rm \
    --name homepage \
    -p 127.0.0.1:8000:80 \
    -v .:/usr/local/apache2/htdocs \
    docker.io/httpd:alpine

That command runs nginx in the foreground - and if you throw an index.html file in there it gets served!

Using this running container to generate a systemd file would work - but there is one flaw with it: the way the volume gets mapped. Without going into too much detail (and without editing the generated service unit) it should be a full path. e.g. change the -v line to -v $PWD:/usr/local/apache2/htdocs this would run the container with the full path mounted.

We are now ready to get generating! Because no one has time to write systemd unit files from scratch.

Step 2: Generating the systemd unit file

Luckily podman has us covered here - we can generate a systemd unit file for running the current container in a single command:

podman generate systemd homepage

(where homepage is the name of the running container)

On my machine that generates something like this:

# container-4d9eb8c2f5d65319bfc61b9ef7ba7bee84fe794c370db5b04742f78f73fd922f.service
# autogenerated by Podman 3.4.0
# Thu Oct 21 12:19:24 CDT 2021

[Unit]
Description=Podman container-4d9eb8c2f5d65319bfc61b9ef7ba7bee84fe794c370db5b04742f78f73fd922f.service
Documentation=man:podman-generate-systemd(1)
Wants=network-online.target
After=network-online.target
RequiresMountsFor=/run/user/1000/containers

[Service]
Environment=PODMAN_SYSTEMD_UNIT=%n
Restart=on-failure
TimeoutStopSec=70
ExecStart=/usr/bin/podman start 4d9eb8c2f5d65319bfc61b9ef7ba7bee84fe794c370db5b04742f78f73fd922f
ExecStop=/usr/bin/podman stop -t 10 4d9eb8c2f5d65319bfc61b9ef7ba7bee84fe794c370db5b04742f78f73fd922f
ExecStopPost=/usr/bin/podman stop -t 10 4d9eb8c2f5d65319bfc61b9ef7ba7bee84fe794c370db5b04742f78f73fd922f
PIDFile=/run/user/1000/containers/overlay-containers/4d9eb8c2f5d65319bfc61b9ef7ba7bee84fe794c370db5b04742f78f73fd922f/userdata/conmon.pid
Type=forking

[Install]
WantedBy=multi-user.target default.target

now this is a pretty brittle unit - it basically just starts/stops/restarts that same container we ran. Thats not very container-ey of us now is it! Lets make this completely stateless (because thats the way it should be).

Step 2.5: Making the unit file run a new container every time

Once again podman has our back here - because who would have thought we would want to rely on container state.

The command to always run a new container is mostly the same except one addition:

podman generate systemd --new homepage

The --new flag tells systemd to always run a new instance. Handy. The output should be something like this now:

# container-homepage.service
# autogenerated by Podman 3.4.0
# Thu Oct 21 12:24:10 CDT 2021

[Unit]
Description=Podman container-homepage.service
Documentation=man:podman-generate-systemd(1)
Wants=network-online.target
After=network-online.target
RequiresMountsFor=%t/containers

[Service]
Environment=PODMAN_SYSTEMD_UNIT=%n
Restart=on-failure
TimeoutStopSec=70
ExecStartPre=/bin/rm -f %t/%n.ctr-id
ExecStart=/usr/bin/podman run --cidfile=%t/%n.ctr-id --cgroups=no-conmon --rm --sdnotify=conmon -d --replace -it --name homepage -p 127.0.0.1:8000:80 -v /home/jlindgren/homepage:/usr/local/apache2/htdocs docker.io/httpd:alpine
ExecStop=/usr/bin/podman stop --ignore --cidfile=%t/%n.ctr-id
ExecStopPost=/usr/bin/podman rm -f --ignore --cidfile=%t/%n.ctr-id
Type=notify
NotifyAccess=all

[Install]
WantedBy=multi-user.target default.target

Now we're in business. This unit will run the container, restart it if it dies, and restart it on boot (if enabled). Now we just need to put it somewhere where systemd knows it.

Step 3: Setting the container up with systemd

If you're already familiar with systemd this part will be simple, otherwise read on.

We're going to set up whats called a systemd "user" service since we don't need these to run as root. In order to do that we need to create the systemd user directory, which lives here:

mkdir -p ~/.config/systemd/user/

Now either save that stdout to a file, or cd to this directory and redirect to create container-homepage.service

OR

cd to the directory and run podman generate systemd --new --file homepage and it will save a file with the fresh unit file.

From here it's as easy as reloading the systemd user context with:

systemd daemon-reload --user

and then starting/enabling the unit with:

systemctl --user enable --now container-homepage

In the background, systemd is:

  1. pulling the image
  2. starting the container
  3. monitoring it for failure, restarting if necessary
  4. forwarding all logs to the journal, which can be viewed with journalctl --user -u container-homepage

Well thats it. You should now have a container thats managed by systemd - and you can just forget about it until it starts giving you issues.

This feature alone has made my switchover to podman mostly painless - I have been running all the services I used to just run with docker-compose via systemd with no issues for months.

Jacob Lindgren

Jacob Lindgren

Nebraska, USA