Hi everyone,
I recently ran into an issue where the Hestia firewall (hestia-iptables.service) kept failing to start on server reboots when used alongside CrowdSec. I managed to figure out a clean and deterministic solution and wanted to share it here in case anyone else faces the same problem.
The Problem
On boot, hestia-iptables.service enters a failed state with the following error in the logs:
iptables-restore v1.8.7 (nf_tables): Set crowdsec-blacklists-1 doesn’t exist.
Error occurred at line: 32
hestia-iptables.service: Main process exited, code=exited, status=2/INVALIDARGUMENT
The Root Cause
It’s a classic startup race condition. Hestia is designed to load its firewall rules extremely early in the boot process (Before=network-pre.target). However, crowdsec-firewall-bouncer needs a few seconds to boot up and create its ipset lists. Because Hestia acts faster, iptables-restore crashes completely since it references an ipset that doesn’t exist yet.
Note: You cannot fix this simply by adding After=crowdsec.service to Hestia’s systemd unit. Doing so creates a circular dependency (ordering cycle) with network-pre.target, and systemd will silently drop the Hestia service from the boot queue (inactive (dead)).
The Solution
The safest and cleanest workaround is to use a systemd drop-in file to pre-create empty ipset “buckets” right before Hestia attempts to load the firewall rules.
This satisfies the strict iptables-restore command. When CrowdSec eventually boots up, it sees the lists are already there and simply starts populating them.
Run the following block in your terminal to create the override file:
mkdir -p /etc/systemd/system/hestia-iptables.service.d/
cat << 'EOF' > /etc/systemd/system/hestia-iptables.service.d/override.conf
[Service]
ExecStartPre=-/sbin/ipset -exist create crowdsec-blacklists-0 hash:net
ExecStartPre=-/sbin/ipset -exist create crowdsec-blacklists-1 hash:net
ExecStartPre=-/sbin/ipset -exist create crowdsec6-blacklists-0 hash:net family inet6
ExecStartPre=-/sbin/ipset -exist create crowdsec6-blacklists-1 hash:net family inet6
EOF
systemctl daemon-reload
Why this works flawlessly:
The - prefix in ExecStartPre=- is a systemd safety net. It ensures that if you manually restart the Hestia firewall while the server is running (meaning CrowdSec has already created these lists), systemd will ignore the “set already exists” error and proceed to load the firewall successfully.
Hope this helps anyone running the HestiaCP + CrowdSec combo!
Server: Ubuntu 20.04/22.04
HestiaCP: 1.9.3,1.9.4,1.9.5,1.9.6