Hello,
I have a server for my client that’s under DDOS few times a day. Now they’re only hitting wordpress homepage. The load is very low for a crash, which is strange.
This is dedicated server with 97gb of ram and 72 cpu cores.
Now the php-fpm just stopping without any log in error.
Today morning I had to increasenginx worker_connections to 30000.
10000 was not enough ( I saw this in logs).
Even under small spikes in traffic php just crashing.
Please remember that’s 72 cpu cores, so 3.5% cpu usage is nothing.
Example of attack:
So during the attack php just stop responding I have to ssh to server and do:
You could also check pm.start_servers, pm.min_spare_servers and pm.max_spare_servers but I would double check the limits php-fpm is using, if you didn’t change it, it should be using 1024 open files and you should raise it.
pid=$(ps -ef | grep '[p]hp-fpm: master.*8.3' | awk '{print $2}'); echo "Memory used by pid $pid"; cat /proc/$pid/status | grep -i vmrss; echo -e "\nLimits for pid $pid";cat /proc/$pid/limits
And you will see something like this:
Memory used by pid 1300
VmRSS: 12708 kB
Limits for pid 1300
Limit Soft Limit Hard Limit Units
Max cpu time unlimited unlimited seconds
Max file size unlimited unlimited bytes
Max data size unlimited unlimited bytes
Max stack size 8388608 unlimited bytes
Max core file size 0 unlimited bytes
Max resident set unlimited unlimited bytes
Max processes 7685 7685 processes
Max open files 1024 524288 files
Max locked memory 8388608 8388608 bytes
Max address space unlimited unlimited bytes
Max file locks unlimited unlimited locks
Max pending signals 7685 7685 signals
Max msgqueue size 819200 819200 bytes
Max nice priority 0 0
Max realtime priority 0 0
Max realtime timeout unlimited unlimited us
Note: I added the command to display VmRSS for the process, VmRSS is the memory that it is using… useful if you are going to raise max_childrens… servers, etc.
As I said, first check open files limit and raise it editing systemd service.
systemctl edit php8.3-fpm.service
And add the directive LimitNOFILE, something like this:
### Editing /etc/systemd/system/php8.3-fpm.service.d/override.conf
### Anything between here and the comment below will become the new contents of the file
[Service]
LimitNOFILE=65535
### Lines below this comment will be discarded
### /lib/systemd/system/php8.3-fpm.service
# [Unit]
[...]
Memory used by pid 681423
VmRSS: 174788 kB
Limits for pid 681423
Limit Soft Limit Hard Limit Units
Max cpu time unlimited unlimited seconds
Max file size unlimited unlimited bytes
Max data size unlimited unlimited bytes
Max stack size 8388608 unlimited bytes
Max core file size 0 unlimited bytes
Max resident set unlimited unlimited bytes
Max processes 384814 384814 processes
Max open files 1024 524288 files
Max locked memory 65536 65536 bytes
Max address space unlimited unlimited bytes
Max file locks unlimited unlimited locks
Max pending signals 384814 384814 signals
Max msgqueue size 819200 819200 bytes
Max nice priority 0 0
Max realtime priority 0 0
Max realtime timeout unlimited unlimited us
