Block abusers with ipset

Community Support General Questions
1

Hey all,
To protect the web servers, I have built this simple system:

  1. Parse web server logs looking for specific patterns. Take note of requesting IP(s) that match those patterns
  2. Check these IPs against abuseipdb.com via API. If confidence of abuse is above 20%, add the IP to ipset list
  3. Distribute the ipset list to other servers (that do not have this system in place)

It is based on: GitHub - ShaneOss/AbuseIPDB-cPanel-CSF: Script to parse the IPs in the cPanel Apache error log. Checks them against AbuseIPDB and add to CSF deny list if certain criteria is met. configured for my needs.

This system works well for each one web server, but -as it is now- it is not efficient. All servers need to run the same checks (waste of resources). Plus, each server has its unique ipset. Moreover, I would like to collect possible abusive IPs from other services as well, like mail servers and create a much wider ipset list, for all servers to have.

To accomplish this, I suppose that I would need to collect all the suspicious IPs to a “central place”, run the checks there and then create one ipset list.

The version I am working on right now, is like this:

a) At the Source Servers like web/mail servers, parse the logs to look for suspicious IPs and then use code like below to send data to the Data Collector Server :

curl --data "param1=IP&param2=Whatever" http://url/upload.php

b) At the Data Collector Server I would run some code (upload.php) to write the collected IPs to a file, run checks and create the ipset list.

c) Place the list on a web accessible URL, so that the Source Servers can download it and load it in their ipset.

What do you think? Overkill? Suggestions? What protection systems do you have in place?

P.S. This is not an actual support question, rather a discussion.

1 Like
2

Hi @Felix

I always love creative ways to block malicious bots and spammers, so this is cool :wink:

The main issue I see is that, with the free API version, you can only check 1000 IPs per day. Depending on your servers, this limit could be sufficient or too low.

Edit: I forgot to say that I also use AbuseIPDB but I download the ip list. Maybe you could take a look.

I fill my ipset blocks adding all the sources from these sites.

"https://www.projecthoneypot.org/list_of_ips.php?t=d&rss=1"            # Project Honey Pot Directory of Dictionary Attacker IPs
"https://check.torproject.org/cgi-bin/TorBulkExitList.py?ip=1.1.1.1"   # TOR Exit Nodes
"https://danger.rulez.sk/projects/bruteforceblocker/blist.php"         # BruteForceBlocker IP List
"https://www.spamhaus.org/drop/drop_v4.json"                           # Spamhaus Don't Route Or Peer List (DROP)
"https://cinsscore.com/list/ci-badguys.txt"                            # C.I. Army Malicious IP List
"https://lists.blocklist.de/lists/all.txt"                             # blocklist.de attackers
"https://blocklist.greensnow.co/greensnow.txt"                         # GreenSnow
"https://iplists.firehol.org/files/firehol_level1.netset"              # Firehol Level 1
"https://iplists.firehol.org/files/stopforumspam_7d.ipset"             # Stopforumspam via Firehol
"https://raw.githubusercontent.com/borestad/blocklist-abuseipdb/main/abuseipdb-s100-14d.ipv4"  # AbuseIPdb 100% 14 days
"https://raw.githubusercontent.com/stamparm/ipsum/master/levels/1.txt" # IPsum Level 1

Then, I count how many times an IP appears across all lists. If an IP appears once, I add it to my ipset lv1; if it appears twice, it goes to lv2; and if it appears three or more times, it goes to lv3. Of course, I remove duplicated ips. After that, I populate my already created ipsets with this data. The iptables ipset rules start with lv3, then lv2, and finally lv1.

I’ve also added all IPs that triggered a Fail2Ban RECIDIVE to a dedicated block list.

1 Like