Broken HestiaCP Installation

Community Support
1

Sorry but I really need help, I broke my HestiaCP installation by trying to add a antispam software to my mail configuration.

So far after failing I tried to roll back and now my whole mail configuration is broken:

  • HestiaCP Panel shows all domains and all domains are reachable
  • HesitaCP Panel shows all Mailboxes and sizes but via webmail (roundcube) and all other software the mailboxes are not reachable for login because dovecot cant authenticate and hestiacp does not wirte any passwd on rebuilding
  • Clamav-daemon is not starting up at the moment and spamd also

Is there a way to “reinstall” the mailserver oder mail configuration without loosing all accounts and mailboxes (backup is already done but I dont want to reset all passwords for all mail users if possible).

2

I would suggest to take backup of all users (v-backup-users), wipe the server (aka reinstall the whole os), reinstall hestia and do a restore (v-restore-user). It’s probaly the fastest way.

3

So but if I do so I will loose all my docker software systems like RTMP servers, services I wrote and and will it also backup all websites including all data like the nextcloud files and so on? Is there no way to reset just the mail server?

4

hestia will only backup its user directory, not additional services added later or docker systems. If you’ve setup nextcloud within hestia, either over the web app installer or manualy inside of the public_html directory, it will be completly backuped.

There is no easy way to restore the services, you could try to run trough the installer script and see, what is modified, what works and what not, but this requires deep knowledge which also can’t be transfered over a public forum.

Maybe another solution would be to move each user and services to another server instance to prevent any loss or additional issues.

5

OK thanks so far. What I found so far is dovecot is the only problem in my eyes, it is not able to authenticate: auth: Error: passwd-file /etc/dovecot/users: open failed: No such file

If i create new users in HestiaCP or new mail addresses, the folders and users are created correctly.
I just cant login through any application like roundcube or either Thunderbird.

I guess dovecot is not able to read the passwd file. I checked differences between my other HestiaCP server and found no difference between the config files.

6

In a short answer, YES! I have had such nightmares in earlier days. No more. I did not use docker. But that is only an environment on which everything sits. So long as you have an access inside and are able to read files, you are good to go.

After I had such crashes, I now can restore the whole system without much of problems. The pre-requisite is that you do not play with important system files and are willing to work step by step in a controlled restoration process.

You first need to make a backup (zip) of some directories immediately before you begin the whole process. Would you like to do this and follow my instructions?

7

I just created a v-userbackup and downloading it to a safe space (80GB) so hold on.

I just want to make clear: HestiaCP is not running in docker, I have some docker applications like an RTMP server and N8N and some self written scripts as service like Python3 scripts. So a whole Operating System Reinstall will be a nightmare.

HestiaCP was the first thing on this server which was installed and is up and running since then and managing all my web and mail staff. Nextcloud is installed manually to one of my domains.

Yes i have full access via SSH. I am downloading the backups right now and If a rescue or recovery will not work, I will book a new root server, restore the HestiaCP users there, route DNS to new server and then copy paste the services but it will be also a day of work.

So if I can restore the dovecot, smpt, imap services and bring it live again, it will be very nice!

8

Sorry for spamming I just want to keep you informed and hoping I will find all fixes:

During the backup process I just tired out some things and found 1 error:
Dovecot was not able to connect the mail directories because the rebuild was blocked caused by an error. There was 2 namespace configurations and so the dovecot broke the process.

Actually I am able to login back on roundcube and all mailboxes are having their content.

Now I have to fix the smtp and exim server because trying sending messages will occur an “SMTP server not available” error and incoming mails are not incoming.

9

If the hestia configuration files in each user directory is not screwed up,you will be able to restore everything without problems. here are the steps:

Make a zip of the following directories, each seperate:

/etc
/home
/usr/local/hestia

This is a compulsary task in such a situation because if you have these things in your pocket, then you can restore evry thing manually, of course you should know what you are doing ….

In the /usr/local/hestia/hestia.conf change the version to the following:

VERSION=‘1.0.0’

Then execute followiong set of commands:

cp /usr/local/hestia/install/common/sudo/hestiaweb /etc/sudoers.d/
v-add-sys-dependencies
apt -y remove hestia-nginx hestia-php
apt -y install bubblewrap
apt -y install --reinstall hestia
apt -y install hestia-nginx hestia-php

Mind you, as you are downgrading the hestia version to 1.0.0., it will go through updating the system files and environment and will produce some errors. This is normal.

For example, now you must have defined an admin name. Earlier, the admin user was a system default and you could not define it. So obviously it will generate errors. However, it will do many other things that the whole installation will be repaired.

It will not break your installtion so do not be scared. It will go through lot of things like templates, checking permissions, etc. So rebuilding config files is - in many cases - not sufficient at all. In these cases, only such an aggressive repair remains the only solution.

Some members have executed in the past my above solution and have had a success. You may want to read their post and see if there is something in parallel.

The above solution is not just for email services. It will repair and restore everything. In case if you are curious on the solution I have given above and what it will do, you can have a look at these update scripts residing under the hestia system installation.

To be honest, it is a complete wrong idea to enforce everyone to install hestia on a fresh installation. I do not agree with this basic and fundamental approach that has been pumped in Vesta. This idea has been taken over or carried forward by the hestia team. While I know that having one main installer, it becomes much easier to handle everything.

Things like this what you are facing, and what I have faced lots of times, can happen to every one. As an admin, our tools and instruments must be ready or available to help us out. But the answer “Install everything fresh” is for me a No-Go. Therefore, I have discovered my tricks of repairing it, which works!

Good luck!

10

Oh, you already restored. So you do not need to downgrade and upgrade it anymore. We were writing post in parallel.

11

I honestly wouldn’t recommend running this unless you really know what you’re doing. In practice, it’s a bit like Russian roulette and can easily cause more problems than it solves. I understand that it may have helped in your specific case, but it’s definitely not a “fix-all” solution.

The approach @kparzinger is taking — checking the error logs of the affected services and fixing or restoring the configurations accordingly — is the right way to handle this situation. That’s essentially the point I was trying to make: in cases like this, you need to be able to go beyond the usual steps and actually understand what’s happening under the hood.

12

Thanks for your long post and @Raphael thanks for your understanding.

I guess one of the biggest problems is that I am blind for the main error.
I cant believe its a completely broken server because a lot of things are working:

  • Domains and webcontent is fully reachable
  • I can add new mail accounts
  • I can use all the rebuild commands and so on

But it stuck every time on the last stepts writing maisl (STMP not reachable) and mails are not coming in.

For sure, I have the “new install” solution in my mind and the user backup file is now downloaded.

13

In this case, you no longer need to repair hestia at all because the conf data under /usr/local/hestia is OK (i.e. not currupted) and is able to create configuration under the /etc file. This is already a good news.

In that case, you need to work with the email services only. The first thing to look at is if the dovecot.conf and exim4.conf is having proper configuration such that it is binding to the public IP.

To begin the test, you can have a basic default conf saved (remove trhe current conf) in the directories and see if the services can start. Thereafter, i.e. if these services can start, you can slowly reconstruct the whole conf step by step.

For example, first add only the main section in Exim4.conf and see if the service can start. If yes, add the ACL section, and so on. When you reach the section of Authentication, Routers, add slowly because there could be a bug waiting for you, in case if it did not create a mess earlier.

Execute following and this will help you in tracking and debugging:

exim4 -d+all -bt [email protected]
exim4 -d+route -bt [email protected]
exim4 -bt [email protected]
nc -vz mail.domain.com 25
telnet mail.domain.com 25

One of the typical cause of emails not being sorted or incoming not being accepted is that the config under /etc/exim4/domains is not available or is destroyed or is corrupted. I am not sure if the rebuild will recreate it. So check, if the domains directory is there and services have an access to the configuration of each domain. If this is missing or destroyed, that would be the perfect place to start with.

Check under /var/spool/exim4/input, if there are messages arriving and are waiting to be distributed. If this is the case, let me know and I will give you a soilution for that.

1 Like
14

I reinstalled my os and hestia, now on restoring the backup with w-restore-user …. .tar the error occurs: invalid backup format

15

Reinstalling the OS (which was a bad mistake!) means inivitation to a complete new set of problems.

Try to see, if you can unpack the tar ball manually.

If you can’t unpack the tar ball, that’s a bad news. In that case, I do not know further steps to suggest. If someone has a diffirent solution in this situation, then welcome.

This error may happen due to incorrect file placement, naming issues, permissions, or shell compatibility problems.

HestiaCP’s v-restore-user script requires the backup .tar file to be in the /backup directory. If this is not in this directory, you get this error. (I do not know, if the error message was recently changed in the last years to reflect that the file is missing.). So the best is:

cd /backup
v-restore-user username username.yourfile.tar

3.
The script performs strict format validation. It must have the format like username.YYYY-MM-DD.tar. If you did not change, then the problem lies somewhere else.

Ensure /backup has 755 permissions: chmod 755 /backup and the .tar file is readable by root.

Test if tar is corrupt:

tar -tf /backup/username.yourfile.tar

Confirm it’s a Hestia/Vesta backup by checking contents:

tar -tf /backup/username.yourfile.tar | head

It should show ./hestia or ./vesta, pam/, web/, etc.

Review logs in /var/log/hestia/error.log for details like checksum failures.

1 Like
16

As @Deepak already wrote, the restore command is probaly wrong. Follow the guidelines of the docs, place the backups in /backup and run v-restore-user username backupfile (without full patch, just the backup file itself).