Any commands I can do to provide some logs? I have no idea where to go, the web panel is working, and all processes are online according to it but none of my websites nor other user websites are working.
I had some problem similiar to this a few weeks back.
I had to disable FastCGI caching, and all my sites went back to working.
How can I do that? Sorry very new to FastCGI, I know it’s related to PHP.
I’ve used v-delete-fastcgi-cache on one of my domains via SSH and well didn’t really fix it.
Also got the new nginx signing key and apt update, rebooted hestia but nothing.
Check apache2 and/or nginx service status.
I have tried that but to no avail. 
root@server:~# sudo systemctl status apache2
â—Ź apache2.service - The Apache HTTP Server
Loaded: loaded (/lib/systemd/system/apache2.service; enabled; preset: enabled)
Active: active (running) since Sat 2024-06-15 00:00:42 PDT; 26min ago
Docs: https://httpd.apache.org/docs/2.4/
Process: 546098 ExecStart=/usr/sbin/apachectl start (code=exited, status=0/SUCCESS)
Main PID: 546122 (apache2)
Tasks: 56 (limit: 7103)
Memory: 15.1M
CPU: 421ms
CGroup: /system.slice/apache2.service
├─546122 /usr/sbin/apache2 -k start
├─546125 /usr/sbin/apache2 -k start
├─546139 /usr/sbin/apache2 -k start
└─546140 /usr/sbin/apache2 -k start
Jun 15 00:00:42 server.uniplex.xyz systemd[1]: Starting apache2.service - The Apache HTTP Server...
Jun 15 00:00:42 server.uniplex.xyz systemd[1]: Started apache2.service - The Apache HTTP Server.
root@server:~#
root@server:~# sudo systemctl status nginx
â—Ź nginx.service - nginx - high performance web server
Loaded: loaded (/lib/systemd/system/nginx.service; enabled; preset: enabled)
Active: active (running) since Sat 2024-06-15 00:00:38 PDT; 27min ago
Docs: https://nginx.org/en/docs/
Process: 545896 ExecStart=/usr/sbin/nginx -c /etc/nginx/nginx.conf (code=exited, status=0/SUCCESS)
Main PID: 545897 (nginx)
Tasks: 3 (limit: 7103)
Memory: 7.4M
CPU: 5.001s
CGroup: /system.slice/nginx.service
├─545897 "nginx: master process /usr/sbin/nginx -c /etc/nginx/nginx.conf"
├─545898 "nginx: worker process"
└─545899 "nginx: cache manager process"
Jun 15 00:00:37 server.uniplex.xyz systemd[1]: Starting nginx.service - nginx - high performance web server...
Jun 15 00:00:38 server.uniplex.xyz systemd[1]: nginx.service: Can't open PID file /run/nginx.pid (yet?) after start: No such >Jun 15 00:00:38 server.uniplex.xyz systemd[1]: Started nginx.service - nginx - high performance web server.
lines 1-17/17 (END)
Found this from Hestia logs, don’t know what to think of it:
root@server:~# tail -f /var/log/hestia/nginx-error.log
2024/06/14 21:19:10 [error] 805#0: OCSP responder timed out (110: Connection timed out) while requesting certificate status, responder: r3.o.lencr.org, peer: 95.101.54.131:80, certificate: "/usr/local/hestia/ssl/certificate.crt"
2024/06/14 23:19:32 [error] 693#0: *158 FastCGI sent in stderr: "PHP message: PHP Warning: Undefined variable $v_redirect_custom in /usr/local/hestia/web/templates/pages/edit_web.php on line 149" while reading response header from upstream, client: 108.64.187.206, server: _, request: "GET /edit/web/?domain=ashbot.work&token=413585597cdad0e61a7c1c402da41a51 HTTP/2.0", upstream: "fastcgi://unix:/run/hestia-php.sock:", host: "server.uniplex.xyz:8083", referrer: "https://server.uniplex.xyz:8083/list/web/"
2024/06/14 23:19:55 [error] 693#0: *158 FastCGI sent in stderr: "PHP message: PHP Warning: Undefined variable $v_redirect_custom in /usr/local/hestia/web/templates/pages/edit_web.php on line 149" while reading response header from upstream, client: 108.64.187.206, server: _, request: "GET /edit/web/?domain=ashbot.work&token=413585597cdad0e61a7c1c402da41a51 HTTP/2.0", upstream: "fastcgi://unix:/run/hestia-php.sock:", host: "server.uniplex.xyz:8083", referrer: "https://server.uniplex.xyz:8083/list/web/"
2024/06/14 23:46:38 [error] 693#0: OCSP responder timed out (110: Connection timed out) while requesting certificate status, responder: r3.o.lencr.org, peer: 2.16.241.15:80, certificate: "/usr/local/hestia/ssl/certificate.crt"
Could you please show the output of these commands?
grep '^nameserver' /etc/resolv.conf
grep -r resolver /etc/nginx/
ping -4 -c 4 r3.o.lencr.org
ping -6 -c 4 r3.o.lencr.org
curl -IL r3.o.lencr.org
root@server:~# grep '^nameserver' /etc/resolv.conf
grep -r resolver /etc/nginx/
ping -4 -c 4 r3.o.lencr.org
ping -6 -c 4 r3.o.lencr.org
curl -IL r3.o.lencr.org
nameserver 1.1.1.1
/etc/nginx/nginx.conf: resolver 1.1.1.1 valid=300s ipv6=off;
/etc/nginx/nginx.conf: resolver_timeout 5s;
PING (2.16.202.121) 56(84) bytes of data.
64 bytes from a2-16-202-121.deploy.static.akamaitechnologies.com (2.16.202.121): icmp_seq=1 ttl=54 time=25.9 ms
64 bytes from a2-16-202-121.deploy.static.akamaitechnologies.com (2.16.202.121): icmp_seq=2 ttl=54 time=25.7 ms
64 bytes from a2-16-202-121.deploy.static.akamaitechnologies.com (2.16.202.121): icmp_seq=3 ttl=54 time=25.7 ms
64 bytes from a2-16-202-121.deploy.static.akamaitechnologies.com (2.16.202.121): icmp_seq=4 ttl=54 time=25.8 ms
--- ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 3005ms
rtt min/avg/max/mdev = 25.677/25.765/25.897/0.087 ms
PING r3.o.lencr.org(g2a02-26f0-0780-0000-0000-0000-5f65-367b.deploy.static.akamaitechnologies.com (2a02:26f0:780::5f65:367b)) 56 data bytes
64 bytes from g2a02-26f0-0780-0000-0000-0000-5f65-367b.deploy.static.akamaitechnologies.com (2a02:26f0:780::5f65:367b): icmp_seq=1 ttl=59 time=1.62 ms
64 bytes from g2a02-26f0-0780-0000-0000-0000-5f65-367b.deploy.static.akamaitechnologies.com (2a02:26f0:780::5f65:367b): icmp_seq=2 ttl=59 time=0.873 ms
64 bytes from g2a02-26f0-0780-0000-0000-0000-5f65-367b.deploy.static.akamaitechnologies.com (2a02:26f0:780::5f65:367b): icmp_seq=3 ttl=59 time=0.799 ms
64 bytes from g2a02-26f0-0780-0000-0000-0000-5f65-367b.deploy.static.akamaitechnologies.com (2a02:26f0:780::5f65:367b): icmp_seq=4 ttl=59 time=7.87 ms
--- r3.o.lencr.org ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 3005ms
rtt min/avg/max/mdev = 0.799/2.788/7.865/2.948 ms
HTTP/1.1 200 OK
Server: nginx
Content-Length: 0
Cache-Control: max-age=20106
Expires: Sat, 15 Jun 2024 14:18:14 GMT
Date: Sat, 15 Jun 2024 08:43:08 GMT
Connection: keep-alive
root@server:~#
Just as a note, the web panel works but none of the websites hosted on Hestia. I have looked into the nginx error log and don’t know what to make of this.
2024/06/11 02:10:05 [error] 3514156#3514156: OCSP responder timed out (110: Connection timed out) while requesting certificat>2024/06/11 06:06:43 [error] 3514156#3514156: OCSP responder timed out (110: Connection timed out) while requesting certificat>2024/06/11 06:07:43 [error] 3514156#3514156: OCSP responder timed out (110: Connection timed out) while requesting certificat>2024/06/11 07:57:08 [error] 1787357#1787357: OCSP responder timed out (110: Connection timed out) while requesting certificat>2024/06/11 07:58:08 [error] 1787357#1787357: OCSP responder timed out (110: Connection timed out) while requesting certificat>2024/06/11 08:14:07 [error] 1787357#1787357: OCSP responder timed out (110: Connection timed out) while requesting certificat>2024/06/11 08:15:07 [error] 1787357#1787357: OCSP responder timed out (110: Connection timed out) while requesting certificat>2024/06/11 08:16:07 [error] 1787357#1787357: OCSP responder timed out (110: Connection timed out) while requesting certificat>2024/06/11 08:56:08 [error] 1787357#1787357: OCSP responder timed out (110: Connection timed out) while requesting certificat>2024/06/11 10:05:08 [error] 1787357#1787357: OCSP responder timed out (110: Connection timed out) while requesting certificat>2024/06/11 14:35:12 [error] 1787357#1787357: OCSP responder timed out (110: Connection timed out) while requesting certificat>2024/06/11 14:36:12 [error] 1787357#1787357: OCSP responder timed out (110: Connection timed out) while requesting certificat>2024/06/11 14:37:12 [error] 1787357#1787357: OCSP responder timed out (110: Connection timed out) while requesting certificat>2024/06/11 17:36:36 [error] 1787357#1787357: OCSP responder timed out (110: Connection timed out) while requesting certificat>2024/06/12 03:07:53 [error] 1787357#1787357: OCSP responder timed out (110: Connection timed out) while requesting certificat>2024/06/13 02:13:06 [error] 2981779#2981779: OCSP responder timed out (110: Connection timed out) while requesting certificat>2024/06/13 11:12:10 [error] 129740#129740: OCSP responder timed out (110: Connection timed out) while requesting certificate >2024/06/13 11:13:10 [error] 129740#129740: OCSP responder timed out (110: Connection timed out) while requesting certificate >2024/06/13 19:04:17 [error] 129740#129740: OCSP responder timed out (110: Connection timed out) while requesting certificate >2024/06/13 22:45:42 [error] 129740#129740: OCSP responder timed out (110: Connection timed out) while requesting certificate >
2024/06/14 22:31:29 [notice] 159720#159720: using inherited sockets from "38;39;40;"
2024/06/14 23:05:11 [notice] 309349#309349: signal process started
My apache2 logs are a mess, here’s some of it.
GNU nano 7.2 /var/log/apache2/error.log
[Sun Jun 09 00:02:17.827602 2024] [ssl:warn] [pid 3148882:tid 140528570165120] AH01909: 45.134.39.71:443:0 server certificate>[Sun Jun 09 00:02:17.829223 2024] [mpm_event:notice] [pid 3148882:tid 140528570165120] AH00489: Apache/2.4.59 (Debian) mod_fc>
[Sun Jun 09 00:02:17.829251 2024] [core:notice] [pid 3148882:tid 140528570165120] AH00094: Command line: '/usr/sbin/apache2'
[Sun Jun 09 06:19:29.819747 2024] [proxy_fcgi:error] [pid 3300670:tid 140528368707264] [client 170.106.115.173:0] AH01071: Go>[Sun Jun 09 06:19:30.182446 2024] [proxy_fcgi:error] [pid 3300670:tid 140528343529152] [client 170.106.115.173:0] AH01071: Go>[Sun Jun 09 06:19:30.370089 2024] [proxy_fcgi:error] [pid 3300670:tid 140528444241600] [client 170.106.115.173:0] AH01071: Go>[Sun Jun 09 07:21:05.107433 2024] [mpm_event:notice] [pid 3148882:tid 140528570165120] AH00493: SIGUSR1 received. Doing grac>[Sun Jun 09 07:21:07.477850 2024] [ssl:warn] [pid 3148882:tid 140528570165120] AH01909: 45.134.39.71:443:0 server certificate>
[Sun Jun 09 07:21:07.478732 2024] [mpm_event:notice] [pid 3148882:tid 140528570165120] AH00489: Apache/2.4.59 (Debian) mod_fc>
[Sun Jun 09 07:21:07.478757 2024] [core:notice] [pid 3148882:tid 140528570165120] AH00094: Command line: '/usr/sbin/apache2'
[Sun Jun 09 08:30:24.501502 2024] [proxy_fcgi:error] [pid 1136561:tid 140528427456192] [client 188.166.24.70:0] AH01071: Got >[Sun Jun 09 08:30:24.577449 2024] [proxy_fcgi:error] [pid 1136561:tid 140528435848896] [client 188.166.24.70:0] AH01071: Got >[Sun Jun 09 08:30:24.671248 2024] [proxy_fcgi:error] [pid 1136561:tid 140528335136448] [client 188.166.24.70:0] AH01071: Got >
[Sun Jun 09 08:30:24.923622 2024] [proxy_fcgi:error] [pid 1136561:tid 140528326743744] [client 188.166.24.70:0] AH01071: Got >[Sun Jun 09 10:30:49.923497 2024] [proxy_fcgi:error] [pid 1136561:tid 140528533382848] [client 102.23.122.251:0] AH01071: Got>[Sun Jun 09 13:02:41.509223 2024] [proxy_fcgi:error] [pid 1136561:tid 140527798298304] [client 123.56.81.27:0] AH01071: Got e>[Sun Jun 09 14:14:44.317069 2024] [proxy_fcgi:error] [pid 1136561:tid 140528541775552] [client 27.222.11.184:0] AH01071: Got >[Sun Jun 09 14:14:46.313911 2024] [proxy_fcgi:error] [pid 1136561:tid 140527823476416] [client 27.222.11.184:0] AH01071: Got >
[Sun Jun 09 14:14:47.040668 2024] [proxy_fcgi:error] [pid 1136561:tid 140528368707264] [client 27.222.11.184:0] AH01071: Got >[Sun Jun 09 20:10:21.865630 2024] [proxy_fcgi:error] [pid 1136561:tid 140528524990144] [client 43.134.36.209:0] AH01071: Got >[Sun Jun 09 20:10:22.683631 2024] [proxy_fcgi:error] [pid 1136561:tid 140527823476416] [client 43.134.36.209:0] AH01071: Got >[Sun Jun 09 20:10:23.184410 2024] [proxy_fcgi:error] [pid 1136561:tid 140528427456192] [client 43.134.36.209:0] AH01071: Got >[Mon Jun 10 05:00:00.473470 2024] [proxy_fcgi:error] [pid 1136561:tid 140528541775552] [client 164.92.248.94:0] AH01071: Got >[Mon Jun 10 05:00:00.485606 2024] [proxy_fcgi:error] [pid 1136561:tid 140528533382848] [client 164.92.248.94:0] AH01071: Got >
[Mon Jun 10 05:00:00.499022 2024] [proxy_fcgi:error] [pid 1136561:tid 140528410670784] [client 164.92.248.94:0] AH01071: Got >[Mon Jun 10 05:00:00.502073 2024] [proxy_fcgi:error] [pid 1136561:tid 140527840261824] [client 164.92.248.94:0] AH01071: Got >[Mon Jun 10 07:21:04.720811 2024] [mpm_event:notice] [pid 3148882:tid 140528570165120] AH00493: SIGUSR1 received. Doing grac>[Mon Jun 10 07:21:05.002927 2024] [ssl:warn] [pid 3148882:tid 140528570165120] AH01909: 45.134.39.71:443:0 server certificate>[Mon Jun 10 07:21:05.004826 2024] [mpm_event:notice] [pid 3148882:tid 140528570165120] AH00489: Apache/2.4.59 (Debian) mod_fc>
[Mon Jun 10 07:21:05.013018 2024] [core:notice] [pid 3148882:tid 140528570165120] AH00094: Command line: '/usr/sbin/apache2'
[Mon Jun 10 18:14:55.467366 2024] [proxy_fcgi:error] [pid 3513599:tid 140527823476416] [client 122.51.11.50:0] AH01071: Got e>[Mon Jun 10 18:14:56.165820 2024] [proxy_fcgi:error] [pid 3513599:tid 140527806691008] [client 122.51.11.50:0] AH01071: Got e>[Mon Jun 10 18:14:56.421723 2024] [proxy_fcgi:error] [pid 3513598:tid 140528343529152] [client 122.51.11.50:0] AH01071: Got e>[Tue Jun 11 05:32:06.743793 2024] [proxy_fcgi:error] [pid 3513598:tid 140528435848896] [client 64.227.136.250:0] AH01071: Got>[Tue Jun 11 05:32:08.793318 2024] [proxy_fcgi:error] [pid 3513598:tid 140528444241600] [client 64.227.136.250:0] AH01071: Got>[Tue Jun 11 05:32:10.798035 2024] [proxy_fcgi:error] [pid 3513598:tid 140528318351040] [client 64.227.136.250:0] AH01071: Got>
[Tue Jun 11 05:32:11.286927 2024] [proxy_fcgi:error] [pid 3513598:tid 140528550168256] [client 64.227.136.250:0] AH01071: Got>[Tue Jun 11 06:01:18.676597 2024] [proxy_fcgi:error] [pid 3513598:tid 140528293172928] [client 103.77.204.90:0] AH01071: Got >
All looks good there.
But most of the errors you shown are from a few days ago, you should check the last logs.
Could you please disable ocsp stapling in one of your sites?
Example with example.net:
Edit conf file /etc/nginx/conf.d/domains/example.net.ssl.conf
And replace:
ssl_stapling on;
ssl_stapling_verify on;
by
ssl_stapling off;
ssl_stapling_verify off;
Save the file and restart nginx.
systemctl restart nginx
Try to access your site.
Now instead of timing out on me. I’m getting the hestia error page after a bit.
Internal Server Error
Oops! Something went wrong.
The server encountered an internal error or misconfiguration and was unable to complete your request.
Error Code: 500
I tried it on another site which has CF proxying and restarted nginx and I still get 504 there, on the above one, it was left unproxied.
Most recent apache2 logs
[Sat Jun 15 00:00:42.744378 2024] [suexec:notice] [pid 546100:tid 140398429251456] AH01232: suEXEC mechanism enabled (wrapper: /usr/lib/apache2/suexec)
[Sat Jun 15 00:00:42.931282 2024] [ssl:warn] [pid 546122:tid 140398429251456] AH01909: 45.134.39.71:443:0 server certificate does NOT include an ID which matches the server name
[Sat Jun 15 00:00:42.945240 2024] [mpm_event:notice] [pid 546122:tid 140398429251456] AH00489: Apache/2.4.59 (Debian) mod_fcgid/2.3.9 OpenSSL/3.0.11 configured -- resuming normal operations
[Sat Jun 15 00:00:42.945315 2024] [core:notice] [pid 546122:tid 140398429251456] AH00094: Command line: '/usr/sbin/apache2'
[Sat Jun 15 01:29:36.233552 2024] [mpm_event:notice] [pid 546122:tid 140398429251456] AH00492: caught SIGWINCH, shutting down gracefully
[Sat Jun 15 01:29:36.482432 2024] [ssl:warn] [pid 933239:tid 139806767368064] AH01909: 45.134.39.71:443:0 server certificate does NOT include an ID which matches the server name
[Sat Jun 15 01:29:36.483476 2024] [suexec:notice] [pid 933239:tid 139806767368064] AH01232: suEXEC mechanism enabled (wrapper: /usr/lib/apache2/suexec)
[Sat Jun 15 01:29:36.579963 2024] [ssl:warn] [pid 933240:tid 139806767368064] AH01909: 45.134.39.71:443:0 server certificate does NOT include an ID which matches the server name
[Sat Jun 15 01:29:36.583838 2024] [mpm_event:notice] [pid 933240:tid 139806767368064] AH00489: Apache/2.4.59 (Debian) mod_fcgid/2.3.9 OpenSSL/3.0.11 configured -- resuming normal operations
[Sat Jun 15 01:29:36.583901 2024] [core:notice] [pid 933240:tid 139806767368064] AH00094: Command line: '/usr/sbin/apache2'
Most recent nginx logs only complaining about errors with the domains that have CF proxy but it cant reqeuest new certificates and i havent disabled ssl stapling on those domains.
I’m trying to access to u....x.xyz and a....t.work directly using your server’s ip and I receive the internal server error.
Regarding the apache logs, I see no problem, those are notice or warning messages, not errors.
You could try to change your nameserver, instead of 1.1.1.1 use 76.76.2.0 in both, your resolv.conf and in nginx conf file, in directive resolver.
Show the output of these commands:
nginx -t
apache2ctl configtest
lsof -Pn +c0 -i:9000 -sTCP:LISTEN
df -h
nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful
Syntax OK
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
php-fpm8.2 935458 root 12u IPv4 1542260 0t0 TCP 127.0.0.1:9000 (LISTEN)
Filesystem Size Used Avail Use% Mounted on
udev 2.9G 0 2.9G 0% /dev
tmpfs 595M 632K 594M 1% /run
/dev/sda1 40G 4.9G 33G 13% /
tmpfs 3.0G 2.2M 2.9G 1% /dev/shm
tmpfs 5.0M 0 5.0M 0% /run/lock
/dev/sda15 124M 12M 113M 10% /boot/efi
tmpfs 595M 0 595M 0% /run/user/0
root@server:~#
I have edited the nameserver to 76.76.2.0 and restarted nginx after doing it on /etc/resolv.conf and /etc/nginx/nginx.conf.
Should I try another system reboot 