[AntiDDoS] for HestiaCP with vDDoS Proxy Protection

duy13 · June 10, 2022, 6:47am

STEP 1: Install HestiaCP (Debian)

wget https://raw.githubusercontent.com/hestiacp/hestiacp/release/install/hst-install.sh
bash hst-install.sh

More documentation:
https://hestiacp.com

STEP 2: Install vDDoS Proxy Protection

vDDoS Proxy Protection is free software to provide a Reverse Proxy Server HTTP(S) protocols. It act as a Layer 7 Firewall Filter & Mitigate DOS, DDOS, SYN Floods, or HTTP Floods attack to protect your website.

wget https://files.voduy.com/vDDoS-Proxy-Protection/latest.sh ; chmod 700 latest.sh ; bash latest.sh

More documentation:
https://vddos.voduy.com

STEP 3: Change Apache Default Port

By default HestiaCP uses Apache or Nginx running on port 80, 443. Or Nginx proxy Apache too, we can reconfigure these Webservers to listen on another port so that vDDoS can proxy on them.

More documentation:
https://vestacp.com/docs/#how-to-remove-nginx-debian-ubuntu

Stop & Remove nginx

service nginx stop
apt-get -y remove nginx

Change Hestia configuration:

cp /usr/local/hestia/conf/hestia.conf /usr/local/hestia/conf/hestia.conf.vddos.bak
cd /usr/local/hestia/conf
sed -i "/PROXY_*/d" hestia.conf

For example, I want Apache listen on port 8080 and 8443:

nano /usr/local/hestia/conf/hestia.conf

...
WEB_PORT='8080'
WEB_RGROUPS='www-data'
WEB_SSL='mod_ssl'
WEB_SSL_PORT='8443'
WEB_SYSTEM='apache2'

Rebuild vhost configs:

for user in $(v-list-sys-users plain); do v-rebuild-web-domains $user; done

mv /etc/apache2/conf.d/phpmyadmin.inc /etc/apache2/conf.d/phpmyadmin.conf

Re-Check Apache port:

root@HestiaCP:~ # service apache2 restart
root@HestiaCP:~ # netstat -lntup|grep apache
tcp        0      0 1.2.3.4:8080           0.0.0.0:*               LISTEN      722/apache2
tcp        0      0 127.0.0.1:8081          0.0.0.0:*               LISTEN      722/apache2
tcp        0      0 1.2.3.4:8443           0.0.0.0:*               LISTEN      722/apache2

STEP 4: Config vDDoS Proxy Protection

For example, the IP Addr of your server is 1.2.3.4:

nano /vddos/conf.d/website.conf

# Website       Listen               Backend                  Cache Security SSL-Prikey   SSL-CRTkey
default         http://0.0.0.0:80    http://1.2.3.4:8080    no    no      no           no
default         https://0.0.0.0:443  https://1.2.3.4:8443   no    no      /vddos/ssl/your-domain.com.pri /vddos/ssl/your-domain.com.crt

Restart vDDoS service after you have configured:

/usr/bin/vddos restart

STEP 5: Config vDDoS Auto Add

vDDoS Auto Add is a addon support for vDDoS Proxy Protection - Monitor Domains/Aliasdomains/Subdomains in Panel Hosting, Web Server, List Domain, Virtual Host… and automatically add them into the website.conf file.

nano /vddos/auto-add/setting.conf

# Default Setting for vddos-add command:

SSL				auto
DNS_sleep 		66
DNS_alias_mode	no
Cache			no
Security		no
HTTP_Listen		http://0.0.0.0:80
HTTPS_Listen	https://0.0.0.0:443
HTTP_Backend	http://1.2.3.4:8080
HTTPS_Backend	https://1.2.3.4:8443

Set Crontab:

ln -s /usr/sbin/apache2ctl /usr/sbin/httpd
echo '*/15 * * * * root /usr/bin/vddos-autoadd webserver apache' >> /etc/crontab

STEP 6: Config vDDoS Auto Switch

vDDoS Auto Switch is a addon support for vDDoS Proxy Protection - Automatically identifies overloaded websites and changes their Security Mode.

nano /vddos/auto-switch/setting.conf

# This is the default configuration for "sensor-switch.sh" and "vddos-autoswitch.sh"

hostname="vDDoS Master"							#(Name this server, it will show up in Email notifications)

vddos_master_slave_mode="no"					#(Turn on "yes" if your system has slave servers, want to sync affter switch like master)
backend_url_check="no"			#(Put the URL of the backend. Ex: https://1.1.1.1:8443/ (make sure Backend status response is "200"))

send_notifications="no"						#(Turn on "yes" if you want receive notification)
smtp_server="smtps://smtp.gmail.com"		#(SMTP Server)
smtp_username="[email protected]"				#(Your Mail)
smtp_password="xxxxxxxxxxxxx" 				#(Get your Apps password for Gmail from https://security.google.com/settings/security/apppasswords)
send_notifications_to="[email protected]"		#(Your Email Address will receive notification)


maximum_allowable_delay_for_backend=2 			#(Means: If Backend (status response "200") is slower than 2s, vDDoS will enable challenge mode)
maximum_allowable_delay_for_website=2 			#(Means: If Website (status response "200") is slower than 2s, vDDoS will enable challenge mode)

default_switch_mode_not_attack="no"				#(Default Mode vDDoS use when it's not under attacked)
default_switch_mode_under_attack="high"			#(Default Mode vDDoS use when it's under attack)
default_waiting_time_to_release="60"			#(For example 60 minutes, release time from challenge)

Crontab vDDoS Auto Switch:

echo '*/5 * * * * root /usr/bin/vddos-autoswitch checkalldomain high' >> /etc/crontab
echo '0 */3 * * * root /usr/bin/vddos-switch allsite no && /usr/bin/vddos reload' >> /etc/crontab
echo '* * * * * root /usr/bin/vddos-sensor' >> /etc/crontab

More documentation:
https://github.com/duy13/vDDoS-Auto-Switch
https://groups.google.com/g/voduydotcom/c/uqKZjFh9b78

eris · June 10, 2022, 6:56am

Please note you will lose some certain features that currently only work with Nginx + Apache2 / Nginx only due to the dropped support for Apache2 only in 1.4!

eris · June 10, 2022, 7:01am

@duy13

Please explain how vDDOS is cable to stop an DDOS if the DDOS is flooding your server. The only thing it might solve it adds a rate limit and prevent apache2 from overloading and add an minor security layer with mode security.

duy13 · June 10, 2022, 8:00am

Thank you for your attention!

How to stop DDOS? There is no way, a DDOS attack only ends if the person who caused it decides to stop the attack and the attacker only stops when he feels his attack is not enough to paralyze the target.
So if the website is still normally accessible while the attack is going on, it will prompt the attack to stop prematurely because the attack is not effective.
The sole philosophy of anti-ddos is to dedicate existing resources CPU, RAM, IO, NET… to serve maximum service to the person or device that needs that service. instead of devoting resources to serving malicious queries.
so we need a filter to determine if a query is satisfactory enough to access the website, currently besides cloudflare (provided 5s javascript check + CAPTCHA) I don’t see any other filter help us do it. vDDoS L7 helps me do that, it helps to remove unsolicited queries from the backend web service:

kwebdeveloper · June 11, 2022, 8:26am

so in order to install that, it’s required to remove nginx on server?

Raphael · June 11, 2022, 9:20am

Personaly I don’t think this makes much sense. As @eris already wrote, apache2 only support has been stopped since around 1.4.

An option would be to get rate limits in place, but as already written, this can’t prevent you from a “real” ddos attack - basicly you’ve 1 gbit/s available, my last ddos attack arrived with more than 24 gbit/s. A ddos protection needs to be done a step higher at the provider/datacenter leven as its provided by cloudflare or also hetzner.

duy13 · June 11, 2022, 1:57pm

Thanks for your attention!

You are confusing layer 7 and layer 3-4 attacks. A ddos attack doesn’t mean always using up network bandwidth. If layer 7 DDoS attack happens your CPU, RAM, IO… will be working at full capacity, webserver (including nginx or whatever web server you recommend), php server, mysql… will crashes quickly. This tool can help mitigate that by refusing to serve “hostile connections”.

If the DDOS attack is at layer 3-4, it will quickly fill up the network card, and if your network is really taking up all the bandwidth, there is no software installed on the server that can help the server increase bandwidth. Are you putting me in a position to prove that installing vDDoS will increase bandwidth? Remember that an attack will include multiple “attack patterns” from layer 3-4 to layer 7, vDDoS can reduce bandwidth usage, because its challenge page is only a few KB compared to the resource. resources actually contained in the site if the site does not have a protective filter.

Let’s admit, if one just installs Hestacp and hosts a few websites on it, if he gets a DDoS attack his website will surely die. I did not find any topic about anti-DDoS for your control panel. (aapanel has nginx waf - one has to pay to use it however)

Layer 3-4 attacks have become obsolete since the advent of cloudflare, incapsula, sucuri… it’s almost impossible to bring down a website that uses cloudflare (Who would try to take down a website lies behind cloudflare by sending layer 3-4 packets hitting the IP of the cloudflare proxy? to flood cloudflare’s bandwidth?) modern attacks only use layer 7 ddos in the hope of sending packets sophisticated and cunning to exhaust the CPU, RAM… of the target. So even though cloudflare has a layer 3-4 system with almost infinite bandwidth, they still have to make 2 layer 7 filters, javascript check 5s and hcaptcha.
I’ve gone through several cases where layer 7 attackers actually bypassed cloudflare javascript check and even cloudflare hcaptcha… what would you do then? contact your ISP or network provider to increase your bandwidth?

(sorry for the clumsy english style)

duy13 · June 11, 2022, 4:12pm

In case you install Hestiacp and for some reason cannot use cloudflare. What makes you confident against layer 7 DDOS attacks? nothing, you don’t have any layer 7 defense tools in your hand. Litespeed has recaptcha support, but it’s rudimentary and not easy to customize:

“rate limit?” → unfortunately, if rate limit can withstand layer 7 attack then all webservers with rate limit can withstand layer 7 attack. But the PHP server + Mysql Server will die before that webserver is crippled, because below the rate limit with tens of thousands of hostile connections is more than enough to cripple the PHP server.
Or worse, an attacker with multiple IPs in hand can also cause a slow attack, sending packets at a rate below your rate limit setting.
What makes you confident to confront such a layer 7 attack? nothing!

I assume you have 100% CPU power, IO, RAM… your website only 1% traffic is real people and you want to serve resources to them (css, js, img…) the remaining 99% of traffic is from hostile connections, even if you use “rate limit” for example you set it to 100 connections / 1 second / 1 IP, the attacker still consumes a lot of your resources out of your 100% CPU, IO, RAM… power, for example he has 10k IPs, can he take “100 connections /1s/1 IP” x 10k = ?, while traffic from people real use only “100 connections /1s/1 IP” x 100 = ?, did you spend too much in “100% of your CPU, IO, RAM… power” to serve useless for attackers? While you only need a layer 7 filter, removing in the first place does not serve hostile connections?

And if layer 3-4 bandwidth attack happens, “vDDoS can’t increase your server’s network bandwidth, because no software can help increase server’s network bandwidth” there is only one way The best way to increase network bandwidth is to buy more at ISP. But vDDoS can reduce “outbound bandwidth”, since the challenge page is only a few KB, compared to an unprotected site, for example a hostile query asking to serve only 1 10MB image, you have lost going 10MB/s of bandwidth, and repeatedly requesting that picture will cause “outbound bandwidth” to congest, while “inbound bandwidth” will not. This should be solvable with the challenge layer 7 page html, it’s only a few KB, after verifying the connection they can download the image 10MB/s and actually take 10MB/s of bandwidth in seconds.
In general vDDOS is not designed for Layer 3-4 attack, that is the task of Iptables, cissco firewall, ISP router… but it reduces unnecessary serving bandwidth for hostile connections enemy.

Raphael · June 11, 2022, 4:18pm

I’m aware of the different layers, thanks for the detailed explanation.

Still, the problem is that hestia builds now up on nginx, apache2 only support has been dropped - so your solution implementation isnt in a good way. Is there any way you could it implement it infront of nginx instead removing it?

duy13 · June 11, 2022, 4:21pm

I just did a tutorial how everything should work with default setting of nginx proxy apache, whether you install nginx only or whatever, usage pattern is same: “vDDOS reverse proxy for webserver”

For those of you who are reading this guide, please don’t do it mechanically without really understanding what you’re trying to do. And test (at least on your PC’s virtual machine) before making a negative comment or applying to a website in production environments.

Thank you all for your interest!

Just change the port of nginx, i guess it is in:

/usr/local/hestia/conf/hestia.conf

eris · June 11, 2022, 4:46pm

I have hosted many websites before and in the 10 years I twice had an DDOS for 1 website. So unless you have websites that attract DDOS you don’t really need it. Basic nginx setup + caching will take care of a lot of traffic…

duy13 · June 11, 2022, 5:30pm

In my experience it’s different, no matter how small a website (even if it only sells toothpaste, dietary supplements, politically controversial articles…) is likely to be attacked by their competitors. Just to compete for the TOP position of Google search, they are ready to destroy each other by hiring a 3rd party, a 3rd party has a rental service to attack DDOS, Stresser, booter…

Aim to take down a competitor’s website.
Help them get ahead in Google search results and get more customers.

“A 3rd party” has a rental service to attack DDOS, Stresser, booter… believe me, they offer it at a cheap price and it’s too easy to attack any website that they itch. DDoS a website is really too easy nowadays.

And guess who gets the most DDoS, who is the customer using the most vDDOS? “What must be, must be”, the people most attacked are “DDOS, Stresser, booster provider…” People who build botnets and do business from them - they turn to attack each other for profit and win customers, or just to show power…, they fought often “All in a day’s work”, and many of the admins of these botnets used vDDoS to defend their botnet site.

They use cloudflare (so funny, cloudflare goes to protect antiddos for one site - specializes in ddos others, they even upgraded the business package) + vDDoS Free. Why don’t they use it, it free? It can be said that the main donation to maintain the vDDoS project comes from those who seek to bypass it.

The HestiaCP project is a free community project, so the benefits of the site for everyone are numerous, there is almost no competition here, so you are less likely to encounter really attacks. One day, we may encounter a bypass attack cloudflare javascript check, captcha… then maybe we will remember this topic.

eris · June 11, 2022, 7:07pm

First of all I am not taking in this case over HestiaCP website / forum. I know that the traffic is here not “large” enough even to kill any server. And if a server dies most likely it will be the apt server when all the users run the updates after release …

But on Wordpress websites with +100k visitors a day. With peaks of 15k similar visits / min at some point.

And I have ran al lot of websites with even without Cloudflare or any other firewall (except my dedicated provider provides).

Maybe if you website attracts DDOS / “bots” due to kind of visitors yes it might be needed to use a firewall but currently the performance of Hestia is keeping it up very well with minor changes to the default settings.

duy13 · June 12, 2022, 4:02am

I totally agree with you, cache server will help reduce server load during attacks, but what if the attacker doesn’t attack into what you have cached?
The essence of that cache is to process it once (if it’s new and not already in the cache) keep what has been processed and return results for subsequent queries. So what happens if the attacker doesn’t query what’s cached? Repeatedly querying for new URLs, for example a 404 attack:

49.48.251.222 - - [11/Jun/2022:09:26:09 +0000] "GET /axqlgstk HTTP/2.0" "202" 1402 "http://google-analytics.cloud/" "Mozilla/5.0 (Windows) NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.48 CitizenFX/1.0.0.5631 Safari/537.36" "49.48.251.222" "https://xzxzxzx.com:443/axqlgstk" "Acceptcookie =0"

Random URLs are sent continuously in requests, and they are definitely not available in the cache:

/axqlgstk
/1xblgstp
/rxqlnsto
...

The PHP server must definitely process them to check if they are 404s or not, nginx has no way of knowing if those URLs are 404s or not.
Besides another type of attack using POST, how do you cache the POST protocol?
If I decide to ddos a website, I will definitely find a way to bypass the cache server, and no website is 100% cached.

You can help a wordpress site serve 15k people per minute with cache, because those people are basically accessing old cached URLs, and make more use of the cache stored in real users’ browsers, helping reduce server load. But with targeted attack tools, things will not happen like that, you can withstand 15k connections from a real person’s browser but will only die at the hands of 7k hostile connections. The hostile connection is not comparable to the connection from the real user, the connection from the real user is much softer.
The user’s browser in general is not designed to kill the serving server, it will be “very soft” with servers that appear overloaded, by having them wait to be served instead of sending the request continuous. And users do the same with overloaded websites, they simply turn it off and go to another site, but not for the attacker. They will still keep the connection and continue the query.

And what about “outbound bandwidth” connections? A good cache server serving hostile queries means reducing CPU but also increasing network “outbound bandwidth” serving useless resources for attackers.

eris · June 12, 2022, 8:20am

Yes offcourse but not in the amount that I should care about. Maybe 0.1% or even less and I don’t give a *** about. As my server is capable of handling such traffic so why put an firewall that does also take resources and so far I can see is a blackbox…

Unless you know you are getting hacked alot there should be no reason to need a lot of websites run fine with it.

Off course you as the developer of the software thinks different about the needs for it. But I don’t see the need for it…

Also the code is closed source. But I will not surprise me it an nginx webserver with some added layers on top.

Surjo · June 13, 2022, 4:32pm

How to use for nginx + Php-FPM??

eris · June 13, 2022, 4:56pm

I suggest not to use it.

VDDOS protection is Nginx with some custom nginx modules that are not supported…

For example…

Asuming provided the last version is 2 years old you might install non supported software with vulnrebilities include…

eris · June 13, 2022, 5:12pm

Change: 升级core代码版本到Nginx-1.18.0 (lianglli)

Current version is 1.22.0

Tengine is a web server originated by Taobao, the largest e-commerce website in Asia. It is based on the Nginx HTTP server and has many advanced features. Tengine has proven to be very stable and efficient on some of the top 100 websites in the world, including taobao.com and tmall.com.

Tengine has been an open source project since December 2011. It is being actively developed by the Tengine team, whose core members are from Taobao, Sogou and other Internet companies. Tengine is a community effort and everyone is encouraged to get involved.

Features

All features of nginx-1.18.0 are inherited, i.e., it is 100% compatible with nginx.

duy13 · June 14, 2022, 2:45am

what make you sure my team is not modifying this module to fit our project.

And are you sure? in kyprizel’s original project, last update was in April 2022

tengine’s last update was in August 2021

I decided to use Tengine for the project instead of Nginx latest version because of Tengine’s great performance (it supports SO_REUSEPORT), this is the favorite webserver of systemadmins experienced with DDOS, I guess you don’t use it it ever.

These projects all have vulnerability reporting and support.

If open source software can solve all human needs, I wonder why people spend money on licensing software every day?
https://vestacp.com/pricing/

I wonder, vestacp’s SFTP CHROOT and filemanager feature charge users, do they provide open source code?
when customers pay to buy nginx plus from nginx.com do they also require open source software?

And since waking up, how many devices have you touched that aren’t actually 100% open source?
When you pay for an airline or train ticket, do you ask if the software that operates those devices is open source or closed?

We are still waiting for your anti-ddos version to be released

Raphael · June 14, 2022, 3:50am

Ehm well, vestacp released their code for filemanager and sftp chroot (and yes, just add a var at the right place and you can use it for free)… Hestia includes such functions and/or modules for free.

Beside that, I dont think @eris has any issues with closed source, personaly I use a lot of Windows myself, which is clearly not an example for open source code. I think taking different opensource code and obfuscate the code looks, at least for me personally, a bit sus.

PS: Also the “i would not use it” was also due to the fact, that the suggested way above to implement your solution would probaly break a lot of nginx based functions in hestia - as already written, apache2 only systems arent supported anymore.