Create ipset from multiple files

[jlguerrero]

I want to create several ipsets:

• Spain + Latin America
• USA + UK + Australia + Canada
• Asian Countries
• African countries
• European countries

The command here creates the ipset but only from ONE source
https://docs.hestiacp.com/cli_commands/commands.html#v-add-firewall-ipset
(The documentation doesn’t even say how is a valid source.)

But I need multiple sources from here: Index of /ipblocks/data/countries

Is there a way to provide v-add-firewall-ipset with multiple sources?

Alternatively I could set a cron job, download those files and “cat” them into a file to feed v-add-firewall-ipset but I want to know if it is possible to add different sources.

[eris]

For example for the NL / BE / LU

https://gist.github.com/jaapmarcus/2d894af99297437ef90d192a8dacf4a3

Chmod the file with 755 !

[jlguerrero]

Thank you very much @eris that script is perfect! Much better than the one that I was about to write.

One more thing, the default ipsets in hestia are from ipverse?

[eris]

Yes.

hestiacp/hestiacp/blob/f5e82b73bede8e640287c490095f777a6e1754cb/web/templates/pages/add_firewall_ipset.html#L116-L128

If you know the country code it is always: http://ipverse.net/ipblocks/data/countries/tld.zone

[eris]

The script does exactly the same:
Download every day (Cronjob) a list of files and merge them together…

IP_TMP=$(mktemp)
(( HTTP_RC=$(curl -L --connect-timeout 10 --max-time 10 -o "$IP_TMP" -s -w "%{http_code}" "$i") ))
if (( HTTP_RC == 200 || HTTP_RC == 302 || HTTP_RC == 0 )); then # "0" because file:/// returns 000
    command grep -Po '^(?:\d{1,3}\.){3}\d{1,3}(?:/\d{1,2})?' "$IP_TMP" | sed -r 's/^0*([0-9]+)\.0*([0-9]+)\.0*([0-9]+)\.0*([0-9]+)$/\1.\2.\3.\4/' >> "$IP_BEL_TMP"
elif (( HTTP_RC == 503 )); then
    echo >&2 -e "\\nUnavailable (${HTTP_RC}): $i"
else
    echo >&2 -e "\\nWarning: curl returned HTTP response code $HTTP_RC for URL $i"
fi
rm -f "$IP_TMP"
done

sed -r -e '/^(0\.0\.0\.0|10\.|127\.|172\.1[6-9]\.|172\.2[0-9]\.|172\.3[0-1]\.|192\.168\.|22[4-9]\.|23[0-9]\.)/d' "$IP_BEL_TMP"|sort -n|sort -mu
rm -f "$IP_BEL_TMP"

This part will only make sure that any comments are removed and all ip adress are converted to CIRD format

[ibarreda]

In the case of having ipset, but wanting to allow a specific subnet… what priority does it have? Could you add a rule to allow only that subnet … or would ipset have more weight?

Thank you very much

[eris]

Ipset need to have at least ip subnet. So use firewall rule instead