ModSecurity and OWASP rule set?

[Lupu]

Unfortunately it won’t be easier the compilation takes more time and it’s a bit more complex.

But don’t get discouraged, the process might have improved since I last played with it.

Let us know what you find

[neven]

You were right. I spent several hours setting up the environment, required modules and compiling, but nginx hanged on start every time. I obviously missed something.

Your approach was rather quick and worked like a charm.
Now I really want to implement this in production, because not only it’s nice to see A+ rating test.vinsdigital.com now gets on immuniweb, looking at modsec_audit.log after just a few minutes of work freaked me out.

[salnz]

You can achieve auto update with nginx dynamic modules in Debian / Ubuntu by using 3 files

1, Script to install module,
2. A hook file in /etc/apt/apt.conf.d/ - to build and install dynamic modules before nginx upgrades
3. Worker script - to build scripts and pass error codes to apt hook

I have this working with 1 extra module and set the build path under /nginx/modules/ for now. You could add a copy option to move the files into that location once completed

[neven]

Definately better then just putting nginx package on hold and forgeting I did that (and why).

[AlwaysSkint]

Mod security should really be a priority, to bolster security in general. You should see my server logs and how many malicious attacks the free OWASP rules trapped!
Combined with CSF, thankfully my servers have remained safe for many years.
Unfortunately my skill set doesn’t cover integrating with Hestia.

[AlwaysSkint]

Is a recompile all that onerous? DirectAdmin for example needs rebuilds for the slightest change! I assume that you install ccache by default, as that helps considerably.
Any version of mod_sec and free OWASP rules is better than none at all.

Including mod_sec/OWASP/CSF would make HestiaCP a much more viable replacement for CWP.

[AlwaysSkint]

Any movement on this, guys?

[Raphael]

We discussed it internaly and would love to implement it. Currently, we haven’t made any step forward due to other pending work.

[AlwaysSkint]

Just a wee reminder, given a recent bad bot thread.

[Lupu]

Link to that thread pls

[eris]

I think this one…

Max CPU/RAM and ClamAV disabled

You might have been hit by an aggressive bot (or more), that were scanning your site(s). Have a look at the web server logs and if this is the case you might want to block some bots with something like this in your .htaccess file(s): SetEnvIfNoCase User-Agent .*rogerbot.* bad_bot SetEnvIfNoCase User-Agent .*exabot.* bad_bot SetEnvIfNoCase User-Agent .*mj12bot.* bad_bot SetEnvIfNoCase User-Agent .*dotbot.* bad_bot SetEnvIfNoCase User-Agent .*gigabot.* bad_bot SetEnvIfNoCase User-Agent .*ahrefsbo…

[AlwaysSkint]

@eris got it in one.

[Lupu]

@AlwaysSkint I have started working on modsecurity for Hestia recently but unfortunately it will take some time before it’s released, it might not even end up there if I find any showstoppers.

Can you share the extra rulesets you mentioned on the other thead besides the default owasp ones?

[AlwaysSkint]

TBH, in 30 years on *nix, I’ve never taken the time/effort to get to grips with regex. So, I end up appending to an existing rule, even though creating a custom one is much more appropriate.

In CWP I’d use these (haven’t got around to it )
/usr/local/apache/modsecurity-owasp-old/base_rules/modsecurity_crs_35_bad_robots.conf
/usr/local/apache/modsecurity-owasp-old/base_rulesmodsecurity_35_bad_robots.data

In WHM/cPanel ( OWASP ModSecurity Core Rule Set V3.0 ) I use the following
and append as follows (my emboldening of text):

SecRule HTTP_User-Agent “(?:\b(?:m(?:ozilla/4.0 (compatible)|etis)|webtrends security analyzer|pmafind)\b|n(?:-stealth|sauditor|essus|ikto)|b(?:lack ?widow|rutus|ilbo)|(?:jaascoi|paro)s|internet explorer|webinspect|.nasl|majestic|mj12bot|istellabot|baidu|yandex|ahrefs|VoilaBot|DotBot|spbot|grader|ezooms|seokicks|xovibot|semalt|semrush|LinksCrawler|aiHitBot|BUbiNG)”
“deny,log,auditlog,msg:‘Request Indicates a Security Scanner Scanned the Site’,id:‘1’,severity:‘2’”

The excellent configserver folks have a free modsec GUI for WHM/cPanel that might be useful to HestiaCP. The CWP GUI may also give you some ideas but is more basic.

Note that regardless of which ruleset is chosen, it can be necessary to define some rules as disabled or create exception based on a particular account/website. This is typical of Wordpress and some e-commerce packages, in addition to control panels. This can usually be handled within a GUI. Here’s a small sample:

CWP
SecRuleRemoveById 960017
Joomla
SecRuleRemoveById 960024
Wordpress
SecRuleRemoveById 981242
Drupal
SecRuleRemoveById 981231
PHPmyadmin
SecRuleRemoveById 981205
oscommerce
SecRuleRemoveById 959070

HTH.

[jibon]

Will you please share the script that you added in /etc/apt/apt.conf.d/? At present I am doing it manually but with automatic script will be helpful. At present I am using this script:

#!/bin/bash

command="nginx -v"
nginxv=$( ${command} 2>&1 )
nginxVersion=$(echo $nginxv | grep -o '[0-9.]*$')

echo version is http://nginx.org/download/$nginxVersion.tar.gz

wget http://nginx.org/download/nginx-$nginxVersion.tar.gz
tar -xvf nginx-$nginxVersion.tar.gz

cd nginx-$nginxVersion
./configure --with-compat --add-dynamic-module=../ModSecurity-nginx && make modules 

rm -rf /etc/nginx/modules/ngx_http_modsecurity_module.so
cp -rf objs/ngx_http_modsecurity_module.so /etc/nginx/modules 
service nginx restart

cd ../
rm -rf nginx-$nginxVersion.tar.gz nginx-$nginxVersion

[salnz]

I will check my backup install ups as I am no longer using Hestia in production as need extra features not available. I did not install modsecurity but perhaps you can replace the lines to what you require

[salnz]

You need to create a file like this in /etc/apt/apt.conf.d/

Mine is simply called 06nginxmodules with the below line that hooks to your file /usr/local/sbin/nginx-mod-preinstall when update occurs.

// Hook to build and install dynamic modules before NGINX upgrades
// Script calls individual build scripts and passes back error codes
// Place this file in /etc/apt/apt.conf.d/

DPkg::Pre-Install-Pkgs {“/usr/local/sbin/nginx-mod-preinstall”;};

in nginx-mod-preinstall you need the below but I only use brotli

nginx-mod-preinstal contents

#!/bin/bash
Call NGINX module build scripts and pass error codes to apt hook

Get NGINX version to upgrade to

read ngfile < <(grep ‘/nginx_’) || exit 0
ngver=$(echo $ngfile | sed ‘s/-.//’ | sed 's/._//’)

List of build scripts to run:

/usr/local/sbin/makebrotli $ngver || exit $?

/usr/local/sbin/makemodsec $ngver || exit $? ← or what your build file is called

/usr/local/sbin/makepagespeed $ngver || exit $?

[jibon]

Thanks for hints @salnz. I have managed to work like this:

So, I have created a hook file like this:

/etc/apt/apt.conf.d/handleModSec

Code:
DPkg::Pre-Install-Pkgs {"/root/modsec/handleModSec"; }

Now in /root/modsec/handleModSec

#!/bin/bash

read debfile < <(grep '/nginx_') || exit 0
nginxVersion=$(echo $debfile | grep -o -e 'nginx_.*-' | sed 's/[^.0-9]*//g')
echo version is https://nginx.org/download/nginx-$nginxVersion.tar.gz

cd /root/modsec
wget https://nginx.org/download/nginx-$nginxVersion.tar.gz
if [ $? -ne 0 ] 
	then exit 0
fi
tar -xvf nginx-$nginxVersion.tar.gz

cd nginx-$nginxVersion
./configure --with-compat --add-dynamic-module=../ModSecurity-nginx && make modules 

rm -rf /etc/nginx/modules/ngx_http_modsecurity_module.so
cp -rf objs/ngx_http_modsecurity_module.so /etc/nginx/modules 

cd ../
rm -rf nginx-$nginxVersion.tar.gz nginx-$nginxVersion

Looks working fine for me. So, before install nginx it’s checking & building module. I ain’t expert on this. Just tried by myself

[kpv]

Regarding aggressive bots, couldn’t you also disallow them in robots.txt (as long as they don’t violate standards) ? For some Websites I only allow Googlebot and Bingbot:

# robots.txt

User-agent: Googlebot
Disallow: /cgi-bin/
Disallow: /tmp/

User-agent: bingbot
Disallow: /cgi-bin/
Disallow: /tmp/

User-agent: *
Disallow: /

and if any bots ignore robots.txt and still hit the website, then I usually block their IP ranges and/or their agent string in Apache.

[AlwaysSkint]

@kpv

and if any bots ignore robots.txt and still hit the website, then I usually block their IP ranges and/or their agent string in Apache.

I went with approach approx. 15 years ago - was a total PITA. Google/Bing also like to ignore robots.txt, BTW; for example ignoring /? directives to prevent crawling parameters. Consider this: in adding a Disallow, you are advertising a location that you don’t want crawlers to go - counter-productive? I actually use this technique for a honeypot on one of my servers.
Why re-invent the wheel when mod_sec can do an excellent job?.. here’s one from yesterday:

[Thu Jul 16 22:04:30.300767 2020] [:error] [pid 16368] [client 154.113.16.226:62796] [client 154.113.16.226] ModSecurity: Access denied with code 406 (phase 2). Pattern match “^[1]+$” at REQUEST_HEADERS:Host. [file “/etc/modsecurity.d/REQUEST-920-PROTOCOL-ENFORCEMENT.conf”] [line “735”] [id “920350”] [msg “Host header is a numeric IP address”] [data “xxx.xxx.xxx.xxx”] [severity “WARNING”] [ver “OWASP_CRS/3.3.0”] [tag “application-multi”] [tag “language-multi”] [tag “platform-multi”] [tag “attack-protocol”] [tag “paranoia-level/1”] [tag “OWASP_CRS”] [tag “capec/1000/210/272”] [tag “PCI/6.5.10”] [hostname “xxx.xxx.xxx.xxx”] [uri “/thinkphp/html/public/index.php”] [unique_id “XxCyzp3GaVmVrWnpbrcUNAAAAAU”]
[Thu Jul 16 22:04:30.531760 2020] [:error] [pid 8218] [client 154.113.16.226:63176] [client 154.113.16.226] ModSecurity: Access denied with code 406 (phase 2). Pattern match “^[2]+$” at REQUEST_HEADERS:Host. [file “/etc/modsecurity.d/REQUEST-920-PROTOCOL-ENFORCEMENT.conf”] [line “735”] [id “920350”] [msg “Host header is a numeric IP address”] [data “xxx.xxx.xxx.xxx”] [severity “WARNING”] [ver “OWASP_CRS/3.3.0”] [tag “application-multi”] [tag “language-multi”] [tag “platform-multi”] [tag “attack-protocol”] [tag “paranoia-level/1”] [tag “OWASP_CRS”] [tag “capec/1000/210/272”] [tag “PCI/6.5.10”] [hostname “xxx.xxx.xxx.xxx”] [uri “/html/public/index.php”] [unique_id “XxCyzrgCK5nTxWyAK6gzeQAAAAA”]

Note the scans for inexistent files, as well as the protocol violation. 5 Attempts and they’re blocked. A Nigerian hack attempt by the looks of things.

Another one:

[Tue Jul 14 00:30:27.732034 2020] [:error] [pid 5299] [client 93.158.66.41:38414] [client 93.158.66.41] ModSecurity: Access denied with code 406 (phase 2). Matched phrase “/.git/” at REQUEST_FILENAME. [file “/etc/modsecurity.d/REQUEST-930-APPLICATION-ATTACK-LFI.conf”] [line “124”] [id “930130”] [msg “Restricted File Access Attempt”] [data “Matched Data: /.git/ found within REQUEST_FILENAME: /.git/head”] [severity “CRITICAL”] [ver “OWASP_CRS/3.3.0”] [tag “application-multi”] [tag “language-multi”] [tag “platform-multi”] [tag “attack-lfi”] [tag “paranoia-level/1”] [tag “OWASP_CRS”] [tag “capec/1000/255/153/126”] [tag “PCI/6.5.4”] [hostname “idle-spare.server.com”] [uri “/.git/HEAD”] [unique_id “Xwzgg1VF4Q3efdj6-fzgmAAAAAE”]

Mod Security is much more powerful/effective than just blocking the many rogue web crawlers.

---

1. \\d.: ↩︎

2. \\d.: ↩︎