I am running Hestia 1.06 and everything has been pretty nice so far.
Now I noticed CTV security issue and checked connections and noticed unauthorized outbound sshd connections on port 22 to wery suspicious ips.
Tried to stop exim but it will not let me. I am currently taking backups of most crucial things to my external hard drive through ftp because I have yet not had time to make external ftp to server backup, this takes some time.
I changed admin password with no help. However server root password still shows that I am the only root user and that I was the last person to log in. So this seems to be only port 22 related currentlyor logs are spoofed.
I donāt know but first I would like to know a way to stop exim, it bumps all ways on even when stopped from shell which is very suspicious. Secondly I am going to put up our hosting provider L3 firewall and ban some ip ranges. Seems we are getting so hard hammering I suspect software firewall alone is not an option.
Also will there be issues if I manually change the server root password aka: sudo passwd root. Can the malicious intruders possibly grep the new root password If I change it now from logs?
Also the issues seems to be fixed in Hestia 1.1.1 What is your recommended way to secure the server and upgrade to 1.1.1?
Sorry for these simple questions but I am not an sysadmin and donāt know much about changes that hestia has made to ubuntu 18.04 lts or the ubuntu server itself.
Any help to resolve this problem would be greatly appreciated.
EDIT: Should I also try to restrict SSH access to specific user group where is only 2 users aka my boss and me? Could not reach one employee which is on sick leave and have to restrain all his user privileges, there is a possibility that user passwords have leaked from his pc.
What for issue you mean? usualy apt-get update && apt-get upgrade is enough.
Restricting SSH access is always a good idea. We usualy allow only our ipv4 subnet to connect to ssh (for example using iptables), also I would suggest to start using keys for authentification, so you canāt be bruteforced anymore.
Everyone starts at this point, I also got a hacked cpanel server years ago when I did . I hope you appreciate that we canāt help you out to understand from zero what happened with your server, for such support cases we donāt have enough time. If you would like to have some paid assistance, please write us on [email protected] - we currently work on a 24/7 paid support service.
if you see outbound connections to foreign IPs port 22, it could be as simple as a php script sneaked into one of your websites, that now runs bruteforce attempts.
while reinstalling can be a good starting point, this might not help with the issue, if you havenāt figured the cause properly. could be that youād just restore the malware script from your backups in the end.
so, rather make sure to find the real cause of what you are seeing. as a first countermeasure, Iād change the own sshd port to something, just to not confuse it. then block port 22 in and out completely and see if the pattern changes, or if suddenly apache/php processes start to get stuck because they sit and wait for timeout.
as @Raphael said, does not sound like a Hestia issue at all, so sorry to say but rather not expect an upgrade to 1.1 to fix it for you (long term).
I truly am not an sysadmin and I think I made things worse. I did change root password and tried to uodate from gui setting up the cron job near current time. Unfortunately I was not aware of this: Due to a change of the repository infrastructure, please install the new key before you upgrade your existing installations:
Lost connection to hestia gui (error 500) because when trying to upgrade I had not upgraded pubkey
Also I did install hestia originally as a testserver with ip as tld and I think I did make a mistake by suspending that installation domain which is our server ip. Got an email from root@mycompany:
From shell tried to issue:
root@mycompany:~# wget -qO - https://gpg.hestiacp.com/deb_signing.key | sudo apt-key add -
sudo: unable to resolve host mycompany.com: Resource temporarily unavailable
gpg: no valid OpenPGP data found.
Added my (dynamic) ip to firewall allow list before this attempt and still luckily have shell access atleast.
Regarding original problem:
connection logs say something like this from hestia netwok gui:
sshd 18520 root 3u IPv4 1003787 0t0 TCP xx.xx.xx.xx:22->54.39.163.64:55952 (ESTABLISHED)
sshd 18521 sshd 3u IPv4 1003787 0t0 TCP xx.xx.xx.xx:22->54.39.163.64:55952 (ESTABLISHED)
sshd 18594 root 3u IPv4 1003949 0t0 TCP xx.xx.xx.xx:22->79.157.219.48:51671 (ESTABLISHED)
sshd 18595 sshd 3u IPv4 1003949 0t0 TCP xx.xx.xx.xx:22->79.157.219.48:51671 (ESTABLISHED)
sshd 18664 root 3u IPv4 1004100 0t0 TCP xx.xx.xx.xx:22->106.12.88.95:59032 (ESTABLISHED)
sshd 18696 sshd 3u IPv4 1004100 0t0 TCP xx.xx.xx.xx:22->106.12.88.95:59032 (ESTABLISHED)
sshd 19755 root 3u IPv4 484302 0t0 TCP xx.xx.xx.xx:22->82.203.140.64:27493 (ESTABLISHED)
So falso I think you are right, it must exploit of the old web shop system possibly, I did upgrade phpmailer while ago but there might be other loopholes.
And the hammering is pretty hefty imho auth.log:
ost=51.79.70.223 user=root
Apr 6 06:25:19 mycompany sshd[20499]: Failed password for root from 51.79.70.223 port 47470 ssh2
Apr 6 06:25:19 mycompany sshd[20499]: Received disconnect from 51.79.70.223 port 47470:11: Bye Bye [preauth]
Apr 6 06:25:19 mycompany sshd[20499]: Disconnected from authenticating user root 51.79.70.223 port 47470 [preauth]
Apr 6 06:26:01 mycompany CRON[20542]: pam_unix(cron:session): session opened for user admin by (uid=0)
Apr 6 06:26:01 mycompany sudo: pam_unix(sudo:session): session opened for user root by (uid=0)
Apr 6 06:26:01 mycompany sudo: pam_unix(sudo:session): session closed for user root
Apr 6 06:26:01 mycompany CRON[20542]: pam_unix(cron:session): session closed for user admin
Apr 6 06:26:05 mycompany sshd[20573]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=178.128.218.56 user=root
Apr 6 06:26:07 mycompany sshd[20573]: Failed password for root from 178.128.218.56 port 55816 ssh2
Apr 6 06:26:07 mycompany sshd[20573]: Received disconnect from 178.128.218.56 port 55816:11: Bye Bye [preauth]
Apr 6 06:26:07 mycompany sshd[20573]: Disconnected from authenticating user root 178.128.218.56 port 55816 [preauth]
Apr 6 06:26:09 mycompany sshd[20576]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=118.25.151.40 user=root
Apr 6 06:26:11 mycompany sshd[20576]: Failed password for root from 118.25.151.40 port 56930 ssh2
Apr 6 06:26:11 mycompany sshd[20576]: Received disconnect from 118.25.151.40 port 56930:11: Bye Bye [preauth]
Apr 6 06:26:11 mycompany sshd[20576]: Disconnected from authenticating user root 118.25.151.40 port 56930 [preauth]
Apr 6 06:26:14 mycompany sshd[20578]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=134.209.33.62 user=root
Apr 6 06:26:16 mycompany sshd[20578]: Failed password for root from 134.209.33.62 port 50822 ssh2
Apr 6 06:26:16 mycompany sshd[20578]: Received disconnect from 134.209.33.62 port 50822:11: Bye Bye [preauth]
Apr 6 06:26:16 mycompany sshd[20578]: Disconnected from authenticating user root 134.209.33.62 port 50822 [preauth]
Apr 6 06:26:32 mycompany sshd[20580]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=149.56.183.202 user=root
Apr 6 06:26:33 mycompany sshd[20582]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=120.92.93.250 user=root
Apr 6 06:26:34 mycompany sshd[20580]: Failed password for root from 149.56.183.202 port 4680 ssh2
Apr 6 06:26:35 mycompany sshd[20580]: Received disconnect from 149.56.183.202 port 4680:11: Bye Bye [preauth]
Apr 6 06:26:35 mycompany sshd[20580]: Disconnected from authenticating user root 149.56.183.202 port 4680 [preauth]
Apr 6 06:26:35 mycompany sshd[20582]: Failed password for root from 120.92.93.250 port 33070 ssh2
Apr 6 06:26:35 mycompany sshd[20582]: Received disconnect from 120.92.93.250 port 33070:11: Bye Bye [preauth]
Apr 6 06:26:35 mycompany sshd[20582]: Disconnected from authenticating user root 120.92.93.250 port 33070 [preauth]
Apr 6 06:27:03 mycompany sshd[20608]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=35.200.206.240 user=root
Apr 6 06:27:04 mycompany sshd[20608]: Failed password for root from 35.200.206.240 port 38020 ssh2
Apr 6 06:27:07 mycompany sshd[20606]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=175.6.35.52 user=root
Apr 6 06:27:09 mycompany sshd[20606]: Failed password for root from 175.6.35.52 port 49524 ssh2
Apr 6 06:27:52 mycompany sshd[20860]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=195.154.119.48 user=root
Apr 6 06:27:55 mycompany sshd[20860]: Failed password for root from 195.154.119.48 port 43862 ssh2
Apr 6 06:28:00 mycompany sshd[20930]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=124.127.206.4 user=root
Apr 6 06:28:01 mycompany CRON[20932]: pam_unix(cron:session): session opened for user admin by (uid=0)
Apr 6 06:28:01 mycompany sudo: pam_unix(sudo:session): session opened for user root by (uid=0)
Apr 6 06:28:01 mycompany sudo: pam_unix(sudo:session): session closed for user root
Apr 6 06:28:01 mycompany CRON[20932]: pam_unix(cron:session): session closed for user admin
Apr 6 06:28:02 mycompany sshd[20930]: Failed password for root from 124.127.206.4 port 36629 ssh2
Apr 6 06:28:02 mycompany sshd[20930]: Received disconnect from 124.127.206.4 port 36629:11: Bye Bye [preauth]
Apr 6 06:28:02 mycompany sshd[20930]: Disconnected from authenticating user root 124.127.206.4 port 36629 [preauth]
Apr 6 06:28:13 mycompany sshd[20962]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=217.182.70.150 user=root
Apr 6 06:28:15 mycompany sshd[20962]: Failed password for root from 217.182.70.150 port 44174 ssh2
Apr 6 06:28:15 mycompany sshd[20962]: Received disconnect from 217.182.70.150 port 44174:11: Bye Bye [preauth]
So I think I should possibly try to resume that originally installed ip web domain, would this do the trick v-list-web-domains admin:
DOMAIN IP TPL SSL DISK BW SPND DATE
xx.xx.xx.xx xx.xx.xx.xx PHP-56 no 4729 184 yes 2019-09-16 onedomain.com xx.xx.xx.xx default no 5 0 no 2019-10-07 mycompany.com xx.xx.xx.xx PHP-56 yes 28295 16444 no 2019-10-07 somedomain.com xx.xx.xx.xx default no 11 18 no 2019-10-18 otherdomain.com xx.xx.xx.xx default yes 4 0 no 2019-11-11
you are right again falzo, exim does not have nothing to do with this, I would like postfix for security reasons more, exim has some history with incidents. Also I think I must change the default email which exim uses to real one.
I will try to get my boss convinced I need pro help, I will contact you later with my work email.
All the best and Happy easter to all folks and than you for the great job.
If you set this as requirement, stop touching hestia, itās a fork of vesta and we all know, that they had much more critical security issues including compromised repositoryā¦
@viperzero from the logs you posted above, I think you might have just misinterpreted things and maybe overreacted?
thatās not even close to āhammeringā but merely some ground noise, thatās to be expected, when running ssh open to public on a standard port
pay attention for the high port numbers these requests are coming fromā¦
now have a closer look at what you consider an outgoing attack:
your sshd is sending something to a high port on another IP. guess what that could be? right, most likely just an answer to the incoming brute force attempt.
Iād assume if you try to match the ips and timestamp of both logs, you can see the direct relationshipā¦
as said above. just move your ssh-port and get rid of the noise. be aware that this does not make your server more secured, itās just a way to avoid dumb bots a bit better.
on the mailserver thing: be aware that you canāt simply replace exim with postfix, if you want to continue using Hestia.
also as scit already pointed out, if you are concerned about security issue of specific parts of the stack Hestia uses, you probably better not use any control panel
this type of panel is just a bunch of scripts and automations to help managing native server solutions. so it usually can not fix any bugs or issues thatās part of these core components. however, it may add risks of additional flaws inside these scripts.
please donāt take this as offense weāre not here to discourage anyone, we just want to be very clear about what Hestia is and what itās not - all the time.
thank you for clarifying this but I saw from hestia cp under ssh connections 50 ssh connections simultaneously open and possibly overreacted. Main reason I am bit paranoid is the old systems which are still running on the server, not the Hestia cp which defends it well. Now I have to figure out how I get pubkey & gui working again. Iāll take a snapshot from server and try those commands from earlier post.
I do still suspect there might be something fishy going on but as you pointed out it can be some old scripts. I herited a lot of unmaintained code which are partly from 2007
Thank you also for updating the documentation. When I have some time I will put Hestia on my own hobby server and really try to upgrade my server administration skills and learn Hestia CP better.
Yes, pure testing server with shell and fail2ban etc with cron jobs it was originally but my boss wanted an control panel and Hestia truly makes things easier to maintain This was not originally even not meant for production use but we had to move quickly from old hosting provider because of their performance & other issues.
. I hope you appreciate that we canāt help you out to understand from zero what happened with your server, for such support cases we donāt have enough time. If you would like to have some paid assistance, please write us on 
This was not originally even not meant for production use but we had to move quickly from old hosting provider because of their performance & other issues.