Websites unavailable after 1 day

Meister · January 9, 2021, 3:56pm

Hello I have the following problem. The websites all go offline after about a day although Apache and Mysql are running. You can still get into the panel and everything looks fine.
Yesterday the server was restarted and the websites still stayed offline. After 30 minutes without doing anything everything went online again.
The problem occurs now only after I updated Hestiacp from 1.2.4 to 1.3.2.

Another error I found in Hestia SSL at ISSUER: C = US, ST = California, L = San Francisco, O = Hestia Control Panel, OU = IT, CN = server.domain2.com, emailAddress =
Suddenly there is another domain in here. “Server.” is correct but domain2.com is not correct. Why did this change? I do not know since when the change is in the domain also runs on the server but has nothing to do with the panel. If the server crashes because of this I can’t say.

Errorlogs:

2021-01-07 13:34:56 0 [ERROR] InnoDB: (Duplicate key) writing word node to FTS auxiliary index table.
2021-01-07 13:34:56 0 [Note] InnoDB: FTS optimize thread exiting.
2021-01-07 13:34:56 0 [Note] InnoDB: Starting shutdown…
2021-01-07 13:34:56 0 [Note] InnoDB: Dumping buffer pool(s) to /var/lib/mysql/ib_buffer_pool
2021-01-07 13:34:56 0 [Note] InnoDB: Buffer pool(s) dump completed at 210107 13:34:56
2021-01-07 13:34:58 0 [Note] InnoDB: Shutdown completed; log sequence number 124208327823; transaction id 294646515
2021-01-07 13:34:58 0 [Note] InnoDB: Removed temporary tablespace data file: “ibtmp1”
2021-01-07 13:34:58 0 [Note] /usr/sbin/mysqld: Shutdown complete

Apache:

[Sat Jan 09 15:38:18.677721 2021] [mpm_prefork:error] [pid 1128] AH00161: server reached MaxRequestWorkers setting, consider raising the MaxRequestWorkers setting

eris · January 9, 2021, 4:02pm

Sat Jan 09 15:38:18.677721 2021] [mpm_prefork:error] [pid 1128] AH00161: server reached MaxRequestWorkers setting, consider raising the MaxRequestWorkers setting

Good starting point

Meister · January 9, 2021, 4:05pm

that doesn’t help me at all. worker_connections is set to 1024. as I said with HestiaCP 1.2.4 it ran without problems.

eris · January 9, 2021, 4:13pm

We haven’t change any regarding Apache2 / max workers in any Apache2 config

Only: * Updated Apache2 configuration to use Include with IncludeOptional (#1072)

See

github.com

hestiacp/hestiacp/blob/main/CHANGELOG.md

# Changelog
All notable changes to this project will be documented in this file.

## [DEVELOPMENT]
### Features
- Introduced support for PHPmyAdmin Single Sign On

### Bugfixes
- Fixed an issue where user name was duplicated when editing FTP users (#1411)
- Fixed an issue where the iptables service would appear to be in a stopped state when fail2ban is stopped. (#1374)
- Fixed an issue where the default language value was incorrectly set under Server Settings > Configure.
- Fixed an issue with the dark theme where available updates were incorrectly displayed.
- Fixed an issue where local and FTP backup files were not deleted when running `v-delete-user-backup`. (#1421)
- Fixed an issue where IP addresses could not be deleted (#1423)
- Improvements have been made to the API's error handling - thanks **@danielalexis**!

## [1.3.2] - Service Release
### Features
- Added PHP v8.0 support for multiphp environment.

This file has been truncated. show original

But what is the main reason still running in prefork?

Meister · January 9, 2021, 4:24pm

Maybe someone has an idea or can confirm the problem. Until then, I’ll watch the whole thing a bit more whether it continues to occur.

How could one return to the 1.2.4 if necessary? Or is that not possible.

Meister · January 9, 2021, 4:45pm

And one more error I just found:
in error_reporting the following is entered:
E_ALL |E_ALL |E_ALL |E_ALL & ~E_DEPRECATED & ~E_STRICT& ~E_DEPRECATED & ~E_STRICT& ~E_DEPRECATED & ~E_STRICT& ~E_DEPRECATED & ~E_STRICT

I doubt that I have written so many E_ALL myself. Something is stored twice.

Raphael · January 9, 2021, 5:05pm

You can’t downgrade to a lower hestia version, this isnt supported. Also as @eris already pointed out, there were no changes from 1.2.4 to 1.3.2 for apache2. What do you run exactly? Apache2+Nginx?

As @eris also pointed out is that you’re still running on mpm_prefork - fpm has a much better performance and is since 1.1.1 (!) the default for any new hestia installation. Please consider to run the manual migration script, you’ll find the needed informations here: Getting Started | Hestia Control Panel

No idea about your other point about the “new” domain, hestia doesnt change anything automatically .

eris · January 9, 2021, 5:07pm

Also:

Improved Apache2 performance by switching to mpm_event instead mod_prefork by default for new installations.
Added a manual migration script for apache2 mpm_event for existing installations/upgrades ($HESTIA/install/upgrade/manual/migrate_mpm_event.sh).

eris · January 9, 2021, 5:09pm

Also check

https://www.google.com/search?client=safari&rls=en&q=[ERROR]+InnoDB:+(Duplicate+key)+writing+word+node+to+FTS+auxiliary+index+table.&ie=UTF-8&oe=UTF-8

Meister · January 9, 2021, 5:31pm

Unfortunately it runs via prefork although I have HestiaCP installed since 1.1.1. Then I have to change that.

the domain change can only come from the system. i do not even know how to change that, there is no function to change!

thank you both.

eris · January 9, 2021, 6:43pm

Mpm event was enabled in 1.2 of Hestia

Meister · January 10, 2021, 12:39am

so i took a look at the file it says “a2modules=“php5.6 php7.0 php7.1 php7.2 php7.3 php7.4 ruid2 mpm_itk mpm_prefork””

php 8.0 is missing. does it still work?

what command do i use to perform the update? like this?

sudo chmod +x /usr/local/hestia/install/upgrade/manual/migrate_mpm_event.sh

eris · January 10, 2021, 12:51am

Probably you should add php8.0

After that:
sudo ./usr/local/hestia/install/upgrade/manual/migrate_mpm_event.sh

Raphael · January 10, 2021, 2:34pm

@Meister any update? Is the server now running properly?

Meister · January 10, 2021, 4:33pm

at the moment the server runs without problems. but i haven’t switched to Mpm event yet. however i have set the max_execution_time to 120, before it was unlimited.

Raphael · January 10, 2021, 4:46pm

That would explain a lot - unlimited execution time would overload the server on bad written scripts. Glad you found it out, even when “unlimited” isnt hestia’s default value .

Meister · January 10, 2021, 4:49pm

problem is that i have a cronjob that needs a longer execution time. the script is located in /var/www/html

is there a way to increase the max_execution_time for this folder?

eris · January 10, 2021, 4:57pm

Use CLI

Raphael · January 10, 2021, 5:00pm

or increase the value, but dont set it tl unlimited. But cli is the better way because it’s unlimited by default.

Meister · January 10, 2021, 5:33pm

thanks for the tip about cli it just worked. then the default limits can stay in:)