Adding Abuseipdb in IPtables and yet I see hits from abusive IPs in logs

Community Support Firewall
1

First of all, thank you everyone for this wonderful software and everyone who helps support it! Can’t thank you all enough.

And now - I’ll share my challenge that I’ve observed recently.

Based on reply of @sahsanu over here - Adding in abuseipdb into the firewall - #2 by sahsanu → I added the abuseipdb list as an IP list and then I also created a Drop firewall rule in the firewall for anything that matches IPs in that IP list.

However - I noticed that on one of my sites, I was still getting automated attacks from one of the IPs in that list and that left me confused.

Why is that the case?

  • Isn’t the IPtable suppose to drop any hits to the server?

  • Or is it going to only drop hits that are made directly to the IP address of the server?

  • Or is the reason that the website domain is behind Cloudflare that IPtables firewall isn’t seeing the real IP of the website visitor?

    However, i’m not sure about this one - I’m assuming that if the apache access logs are showing the real IP of the user then Cloudflare IPtables should see the real ip as well, right?

2

No, the web server sees the real IP because it extracts it from a header (CF-Connecting-IP) sent by Cloudflare. And no, iptables doesn’t see the real IP, it only sees Cloudflare’s IP. If you want to block that IP, you should create a deny rule on the web server, which is a pain, or better, block it on Cloudflare’s end.

3

Ok perfect, thank you! This clarifies a lot. This gives me peace of mind that the solution I’ve developed is the best way to move forward →

I’ve implemented the block on apache level. It doesn’t make sense to use Cloudflare because Cloudflare IP Lists only allow up to 10,000 entries per IP list and the IP list has more than 100,000 entries.

So - i’ve created an automated script that updates the IP list every day and used the Web Templates to configure Apache to deny hits to any IP in that list. The script also has test feature to allow you to test if it works or not.

Will share the process shortly, I’m writing a blog post on it :slight_smile:

1 Like
4

How did you implement it? I’m asking because denying more than 100,000 IPs in Apache can cause performance issues.

1 Like
5

I’ll create instructions in a detailed post but for your reference here’s what I’m doing in template file.

RewriteMap abuseipdb_blocklist "txt:/etc/apache2/abuseipdb_blocklist.txt"
RewriteEngine On
RewriteCond ${abuseipdb_blocklist:%{REMOTE_ADDR}} "blocked"
RewriteRule ^ - [F]

Rewritemap from what i’ve researched is pretty performant. And to further improve the performance - i’m going to change the implementation from txt to dbm file instead.

2 Likes
6

For a list with more than 100,000 IPs, keep in mind that every HTTP request will be checked. Well, Apache2 loads that list into memory so it shouldn’t be an issue, it could still cause slowness when restarting/reloading Apache2.

Using the DBM format will indeed be more efficient.

I don’t really use Apache myself, so go ahead, I’d be interested to know how well it performs on a medium to high traffic site.

I forgot to say that if the IP list contains networks like 203.0.113.0/24 (as they usually do) that RewriteCond won’t work.

4 Likes
7

Thankfully the list I have doesn’t have these CIDR notation

My sites don’t get so much traffic. I’m interested in testing this solution to ensure that i’m creating a solution that’s useful not just for me but others as well. My goal is to first ensure that this works with txt version properly.

Can you please help me understand, how much traffic according to you is medium to high traffic.

Personally for me - i’m confident that this solution is a good solution because →

  1. I’m caching the site’s HTML on Cloudflare completely
  2. I have rate limiting rules set on Cloudflare

So the hits on the apache are quite low for me. But - as I said - your inputs will help me refine this approach and make a more generic solution that can be applied by others.

1 Like
8

This is subjective but I would consider a medium traffic site if it has 1000 to 15000 daily visits and a high traffic site with more than 15000.

2 Likes
9

Hi, did you happen to make this script? Thanks

10

Hi @asiago87,

Sorry, I don’t know which script you are talking about.

1 Like
11

Yes I did.

  1. Feel free to verify the above script by asking chatgpt what it does.
  2. Put this shell file in root in a folder… say ~/tools/abuseipblocker.sh
  3. make it executable chmod +x abuseipblocker.sh
  4. I’m assuming that you are using apache as webserver along with PHP-FPM: for that →

    1. Copy default.tpl & default.stpl as blocklist.tpl and blocklist.stpl over here (/usr/local/hestia/data/templates/web/apache2/php-fpm)

    2. edit BOTH blocklist.tpl and blocklist.stpl to add the bold line after this

      <VirtualHost %ip%:%web_port%>

ServerName %domain_idn%
%alias_string%
ServerAdmin %email%

      RewriteMap abuseipdb_blocklist “dbm:/etc/apache2/abuseipdb_blocklist.dbm”

  5. In hestiacp, edit the domain and select the blocklist template for webserver
  6. run the script… ~/tools/abuseipblocker.sh update
  7. Test if you are getting blocked by apache or not by running -
    ~/tools/abuseipblocker.sh test
    the test script will unblock you as well. But if it works then you’ve done everything correctly.
  8. Add a cron to update the iplist regularly by running this command ~/tools/abuseipblocker.sh update

Please note - This is the fully functioning version, the rest of the repo has another advanced version that i’m working on, which will use nginx instead of Apache. Unfortunately that nginx version isn’t working properly yet - so stick with the instructions shared here and use abuseipblock-original-dbm.sh script only.