User www-data takes 100% of cpu, what it is?

saleseo · July 10, 2023, 8:45pm

Hello!
Could you help to clear an issue…
Server under debian + hestia started to load on 100%
In processes i found so:

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
755 www-data 20 0 231336 150120 24932 R 100.0 3.7 8:41.89 nginx
753 www-data 20 0 359620 275228 24920 R 100.0 6.8 8:40.77 nginx
754 www-data 20 0 324084 249828 24916 R 100.0 6.2 8:44.57 nginx
756 www-data 20 0 269456 195792 24992 R 100.0 4.9 8:42.79 nginx

What is it?

ddos
virus
something etc.

eris · July 10, 2023, 9:03pm

With out any more info I really don’t know…

saleseo · July 10, 2023, 9:26pm

Strange situation with one account in server bootma

top - 00:25:35 up 4 min, 1 user, load average: 4.16, 2.76, 1.21
Tasks: 167 total, 7 running, 160 sleeping, 0 stopped, 0 zombie
%Cpu(s): 91.9 us, 5.7 sy, 0.0 ni, 0.6 id, 0.0 wa, 0.0 hi, 1.8 si, 0.0 st
MiB Mem : 3930.7 total, 130.7 free, 2516.0 used, 1284.0 buff/cache
MiB Swap: 255.0 total, 195.6 free, 59.4 used. 864.3 avail Mem

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
754 www-data 20 0 186252 115148 17836 R 93.7 2.9 4:16.27 nginx
750 www-data 20 0 212224 141644 18064 R 92.0 3.5 4:14.56 nginx
751 www-data 20 0 232704 161904 18192 R 91.7 4.0 4:14.32 nginx
752 www-data 20 0 183132 112432 17996 R 87.0 2.8 4:13.65 nginx
611 mysql 20 0 2207856 199700 16680 S 9.7 5.0 0:24.12 mariadbd
993 bootma 20 0 265000 31416 22280 S 4.3 0.8 0:04.56 php-fpm7.3
7360 anekdot+ 20 0 314628 69308 47332 S 3.7 1.7 0:00.11 php-fpm7.4
980 bootma 20 0 265000 31368 22208 R 3.0 0.8 0:04.82 php-fpm7.3
1011 bootma 20 0 265000 31152 22084 R 2.3 0.8 0:04.30 php-fpm7.3
1014 bootma 20 0 265000 31536 22412 S 2.3 0.8 0:04.93 php-fpm7.3
1016 bootma 20 0 265000 31188 22060 S 1.7 0.8 0:04.38 php-fpm7.3
981 bootma 20 0 265000 31232 22084 S 1.0 0.8 0:05.75 php-fpm7.3
979 bootma 20 0 265000 31400 22252 S 0.7 0.8 0:06.35 php-fpm7.3
982 bootma 20 0 265000 31244 22108 S 0.7 0.8 0:05.32 php-fpm7.3
21 root 20 0 0 0 0 S 0.3 0.0 0:00.28 ksoftirqd/1
27 root 20 0 0 0 0 S 0.3 0.0 0:00.83 ksoftirqd/2
33 root 20 0 0 0 0 S 0.3 0.0 0:00.34 ksoftirqd/3
42 root 20 0 0 0 0 S 0.3 0.0 0:00.14 kcompactd0
105 root 20 0 0 0 0 S 0.3 0.0 0:00.38 kswapd0
115 root 20 0 0 0 0 I 0.3 0.0 0:00.01 kworker/u8:3-flush-252:0
546 root 20 0 987164 21592 9476 S 0.3 0.5 0:00.74 fail2ban-server
1 root 20 0 98668 9568 6824 S 0.0 0.2 0:00.58 systemd
2 root 20 0 0 0 0 S 0.0 0.0 0:00.00 kthreadd
3 root 0 -20 0 0 0 I 0.0 0.0 0:00.00 rcu_gp
4 root 0 -20 0 0 0 I 0.0 0.0 0:00.00 rcu_par_gp
5 root 0 -20 0 0 0 I 0.0 0.0 0:00.00 netns
6 root 20 0 0 0 0 I 0.0 0.0 0:00.00 kworker/0:0-events
7 root 0 -20 0 0 0 I 0.0 0.0 0:00.00 kworker/0:0H-events_highpri
8 root 20 0 0 0 0 I 0.0 0.0 0:00.33 kworker/u8:0-flush-252:0
9 root 0 -20 0 0 0 I 0.0 0.0 0:00.00 mm_percpu_wq
10 root 20 0 0 0 0 S 0.0 0.0 0:00.00 rcu_tasks_rude_
11 root 20 0 0 0 0 S 0.0 0.0 0:00.00 rcu_tasks_trace
12 root 20 0 0 0 0 S 0.0 0.0 0:00.30 ksoftirqd/0
13 root 20 0 0 0 0 I 0.0 0.0 0:00.30 rcu_sched
14 root rt 0 0 0 0 S 0.0 0.0 0:00.00 migration/0
15 root -51 0 0 0 0 S 0.0 0.0 0:00.00 idle_inject/0
16 root 20 0 0 0 0 I 0.0 0.0 0:00.14 kworker/0:1-events

saleseo · July 10, 2023, 9:33pm

If i turn off bootma account, load of server takes ngnix user www-data

top - 00:32:26 up 11 min, 1 user, load average: 3.46, 3.72, 2.29
Tasks: 155 total, 5 running, 150 sleeping, 0 stopped, 0 zombie
%Cpu(s): 87.5 us, 6.0 sy, 0.0 ni, 2.1 id, 0.1 wa, 0.0 hi, 4.3 si, 0.0 st
MiB Mem : 3930.7 total, 119.1 free, 2935.0 used, 876.6 buff/cache
MiB Swap: 255.0 total, 58.3 free, 196.7 used. 476.1 avail Mem

PID USER 11539 www-data 20 11538 www-data 20 11536 www-data 20 11537 www-data 20 105 root 20 0 21 root 20 0 27 root 20 0 162 root 0 -20 1 root 2 root 20 0 3 root 0 -20 4 root 0 -20 5 root 0 -20 7 root 0 -20 8 root 20 0 9 root 0 -20 10 root 20 0 11 root 20 0 12 root 20 0 13 root 20 0 14 root rt 0 15 root -51 0 16 root 20 0 17 root 20 0 18 root 20 0 19 root -51 0 20 root rt 0 23 root 0 -20 24 root 20 0 25 root -51 0 26 root rt 0 29 root 0 -20 30 root 20 0 31 root -51 0 32 root rt 0 33 root 20 0 35 root 0 -20 PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
0 380348 295840 7408 R 99.7 7.3 0:20.33 nginx
0 359012 271816 7380 R 99.3 6.8 0:21.93 nginx
0 234404 150648 7440 R 99.0 3.7 0:22.47 nginx
0 468996 376528 7324 R 91.0 9.4 0:21.51 nginx
0 0 0 S 0.7 0.0 0:01.10 kswapd0
0 0 0 S 0.3 0.0 0:00.52 ksoftirqd/1
0 0 0 S 0.3 0.0 0:01.83 ksoftirqd/2
0 0 0 I 0.3 0.0 0:00.08 kworker/3:1H-events_highpri
20 0 98668 8108 5356 S 0.0 0.2 0:00.68 systemd
0 0 0 S 0.0 0.0 0:00.00 kthreadd
0 0 0 I 0.0 0.0 0:00.00 rcu_gp
0 0 0 I 0.0 0.0 0:00.00 rcu_par_gp
0 0 0 I 0.0 0.0 0:00.00 netns
0 0 0 I 0.0 0.0 0:00.00 kworker/0:0H-events_highpri
0 0 0 I 0.0 0.0 0:00.44 kworker/u8:0-events_power_efficient
0 0 0 I 0.0 0.0 0:00.00 mm_percpu_wq
0 0 0 S 0.0 0.0 0:00.00 rcu_tasks_rude_
0 0 0 S 0.0 0.0 0:00.00 rcu_tasks_trace
0 0 0 S 0.0 0.0 0:00.59 ksoftirqd/0
0 0 0 I 0.0 0.0 0:00.67 rcu_sched
0 0 0 S 0.0 0.0 0:00.00 migration/0
0 0 0 S 0.0 0.0 0:00.00 idle_inject/0
0 0 0 I 0.0 0.0 0:00.17 kworker/0:1-events
0 0 0 S 0.0 0.0 0:00.00 cpuhp/0
0 0 0 S 0.0 0.0 0:00.00 cpuhp/1
0 0 0 S 0.0 0.0 0:00.00 idle_inject/1
0 0 0 S 0.0 0.0 0:00.04 migration/1
0 0 0 I 0.0 0.0 0:00.00 kworker/1:0H-events_highpri
0 0 0 S 0.0 0.0 0:00.00 cpuhp/2
0 0 0 S 0.0 0.0 0:00.00 idle_inject/2
0 0 0 S 0.0 0.0 0:00.05 migration/2
0 0 0 I 0.0 0.0 0:00.00 kworker/2:0H-kblockd
0 0 0 S 0.0 0.0 0:00.00 cpuhp/3
0 0 0 S 0.0 0.0 0:00.00 idle_inject/3
0 0 0 S 0.0 0.0 0:00.04 migration/3
0 0 0 S 0.0 0.0 0:00.63 ksoftirqd/3
0 0 0 I 0.0 0.0 0:00.00 kworker/3:0H-events_highpri

saleseo · July 10, 2023, 10:07pm

I found an issue, somebody do many requests same
How i can close this flood /?..

185.220.101.0 - - [11/Jul/2023:01:05:08 +0300] “GET /?H0DF0=jDgsufAm HTTP/2.0” 502 2869 “-” “Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/108.0
.4521.47 Safari/537.36”
185.220.101.0 - - [11/Jul/2023:01:05:08 +0300] “GET /?uXMmy=julHPedy HTTP/2.0” 502 2869 “-” “Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/108.0
.4521.47 Safari/537.36”
185.220.101.0 - - [11/Jul/2023:01:05:08 +0300] “GET /?8fxFt=NfMbCA4j HTTP/2.0” 502 2869 “-” “Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/108.0
.4521.47 Safari/537.36”
185.220.101.0 - - [11/Jul/2023:01:05:08 +0300] “GET /?s2MoZ=GiHqgdLK HTTP/2.0” 502 2869 “-” “Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/108.0
.4521.47 Safari/537.36”
185.220.101.0 - - [11/Jul/2023:01:05:08 +0300] “GET /?M4OvE=FeQTizaq HTTP/2.0” 502 2869 “-” “Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/108.0
.4521.47 Safari/537.36”
185.220.101.0 - - [11/Jul/2023:01:05:08 +0300] “GET /?Vfrfb=U0NbGitm HTTP/2.0” 502 2869 “-” “Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/108.0
.4521.47 Safari/537.36”
185.220.101.0 - - [11/Jul/2023:01:05:08 +0300] “GET /?ijj9Q=IJlvJSGN HTTP/2.0” 502 2869 “-” “Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/108.0
.4521.47 Safari/537.36”
185.220.101.0 - - [11/Jul/2023:01:05:08 +0300] “GET /?5dxiB=55XXk40J HTTP/2.0” 502 2869 “-” “Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/108.0
.4521.47 Safari/537.36”
185.220.101.0 - - [11/Jul/2023:01:05:08 +0300] “GET /?VT0z9=cjXHFOz0 HTTP/2.0” 502 2869 “-” “Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/108.0
.4521.47 Safari/537.36”
185.220.101.0 - - [11/Jul/2023:01:05:08 +0300] “GET /?xC7r6=YpC769il HTTP/2.0” 502 2869 “-” “Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/108.0
.4521.47 Safari/537.36”
185.220.101.0 - - [11/Jul/2023:01:05:08 +0300] “GET /?y1GZu=Hguq8wAU HTTP/2.0” 502 2869 “-” “Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/108.0
.4521.47 Safari/537.36”

eris · July 10, 2023, 10:08pm

Yes looks like it

This is a Tor Exit Node

saleseo · July 10, 2023, 10:13pm

Is it possible to tunr off /?.. requests or need to kill tor traffic, don’t know how

saleseo · July 10, 2023, 10:36pm

Dumn it… i changed php7.3 to php 7.4 for problem domain and cpu was 2% load…
What is was, fix for bad playload for php 7.3?

saleseo · July 11, 2023, 3:25am

Problem is still here
Could you help to understand, after ddos attacks free space is out of
Where i should look and delete logs or something else, for 3-4 hours 60 gb was filled…

saleseo · July 11, 2023, 5:17am

solved it… add # before log routes
New task, many zero connections in network
Need to close them faster…

nginx 21328 www-data 419u IPv4 27220940 0t0 TCP 5...:443->86.28.76.188:43367 (ESTABLISHED)
nginx 21328 www-data 420u IPv4 27263064 0t0 TCP 5...:443->222.65.178.106:59524 (ESTABLISHED)
nginx 21328 www-data 421u IPv4 27263065 0t0 TCP 5...:443->92.42.110.45:60492 (ESTABLISHED)
nginx 21328 www-data 423u IPv4 27244934 0t0 TCP 5...:443->62.169.91.206:50743 (ESTABLISHED)
nginx 21328 www-data 425u IPv4 27257712 0t0 TCP 5...:443->182.253.175.10:40106 (ESTABLISHED)
nginx 21328 www-data 427u IPv4 27163327 0t0 TCP 5...:443->119.113.194.231:37998 (ESTABLISHED)
nginx 21328 www-data 428u IPv4 27257713 0t0 TCP 5...:443->5.8.35.226:10286 (ESTABLISHED)
nginx 21328 www-data 429u IPv4 27251151 0t0 TCP 5...:443->59.89.60.76:44152 (ESTABLISHED)
nginx 21328 www-data 430u IPv4 27171090 0t0 TCP 5...:443->154.117.156.18:55810 (ESTABLISHED)
nginx 21328 www-data 431u IPv4 27260757 0t0 TCP 5...:443->213.19.205.18:43382 (ESTABLISHED)
nginx 21328 www-data 432u IPv4 27246734 0t0 TCP 5...:443->91.213.33.57:58415 (ESTABLISHED)
nginx 21328 www-data 433u IPv4 27250690 0t0 TCP 5...:443->141.147.161.189:59432 (ESTABLISHED)
nginx 21328 www-data 434u IPv4 27257330 0t0 TCP 5...:443->103.166.39.250:48544 (ESTABLISHED)
nginx 21328 www-data 435u IPv4 27222505 0t0 TCP 5...:443->182.253.182.116:58176 (ESTABLISHED)
nginx 21328 www-data 436u IPv4 27245863 0t0 TCP 5...:443->175.137.116.55:50558 (ESTABLISHED)
nginx 21328 www-data 437u IPv4 27223001 0t0 TCP 5...:443->107.173.159.113:51394 (ESTABLISHED)
nginx 21328 www-data 438u IPv4 27260696 0t0 TCP 5.*..:443->105.214.86.166:44505 (ESTABLISHED)