Resolving disk space issues on my VPS

Wibol · February 18, 2020, 2:08pm

Today I found my VPS disk full. I could not log in to HestiaCP, IMAP or Roundcube. Not even the user admin via SSH could create temporary files to operate. I downloaded the 3 backups using scp and deleted them to free up space. After that I restarted and finally I could log in.

Then I followed the tutorial Resolving Disk Space issues on a VPS / Dedicated server and the disk occupation has remained at 48% after deleting several 5 GB log files.

Does HestiaCP have any tool to perform this process? I was not able to find it. Would it be possible to add it?

Raphael · February 18, 2020, 2:26pm

Depends on which log files you’ve deleted, usualy the logs should have a few mb, not 5 gb.

Wibol · February 18, 2020, 2:31pm

I’ve deleted these files:

5.3G /var/log/syslog.1
5.3G /var/log/daemon.log
4.1G /var/log/clamav/freshclam.log-20200217_132103

Raphael · February 18, 2020, 2:45pm

Did you checked what the content or some parts if it? They are srsly to big, which could point to a issue with the system, maybe you can check the current files for any abnormal parts.

falzo · February 18, 2020, 3:39pm

to add to that, usually logrotate takes care of compressing and even removing old logfiles after a certain amount of time. seeing syslog.1 in your list makes me guess it works like expected on your system…

BUT, your box might have seen a brute force attack or some other incident which made the log-files grow very quickly in a short amount of time. so you definitely should follow @Raphael s advice and try to investigate… aka READ these logs

Wibol · February 19, 2020, 10:40am

So far I have not seen anything suspicious. I will continue watching them.

Thank you.

Wibol · February 23, 2020, 10:19am

This seems to be related to a ClamAV problem. Hundreds of lines appear in syslog and daemon.log:

root@mail:/home/admin# cat /var/log/syslog.1| grep Malformed
Feb 22 16:25:31 mail freshclam[951]: Sat Feb 22 16:25:31 2020 -> ^[LibClamAV] Can't load daily.hsb: Malformed database
Feb 22 16:25:31 mail freshclam[951]: Sat Feb 22 16:25:31 2020 -> ^[LibClamAV] Can't load /var/lib/clamav/tmp/clamav-0f5c43861d7584667417f38ced9727f5.tmp-daily.cld: Malformed database
Feb 22 16:25:31 mail freshclam[951]: Sat Feb 22 16:25:31 2020 -> !Failed to load new database: Malformed database
Feb 22 17:25:38 mail freshclam[951]: Sat Feb 22 17:25:38 2020 -> ^[LibClamAV] cli_loadhash: Malformed hash string at line 1155587
Feb 22 17:25:38 mail freshclam[951]: Sat Feb 22 17:25:38 2020 -> ^[LibClamAV] Can't load /var/lib/clamav/tmp/clamav-068909abd171768f89c40e1569fdc31e.tmp-daily.cld: Malformed database
Feb 22 17:25:38 mail freshclam[951]: Sat Feb 22 17:25:38 2020 -> !Failed to load new database: Malformed database
Feb 22 18:25:45 mail freshclam[951]: Sat Feb 22 18:25:45 2020 -> ^[LibClamAV] cli_loadhash: Malformed hash string at line 1155587
Feb 22 18:25:45 mail freshclam[951]: Sat Feb 22 18:25:45 2020 -> ^[LibClamAV] Can't load /var/lib/clamav/tmp/clamav-89e3cb2dd47ac723a6ee87051b73b90c.tmp-daily.cld: Malformed database
Feb 22 18:25:45 mail freshclam[951]: Sat Feb 22 18:25:45 2020 -> !Failed to load new database: Malformed database
Feb 22 19:25:53 mail freshclam[951]: Sat Feb 22 19:25:53 2020 -> ^[LibClamAV] Can't load daily.hsb: Malformed database
Feb 22 19:25:53 mail freshclam[951]: Sat Feb 22 19:25:53 2020 -> ^[LibClamAV] Can't load /var/lib/clamav/tmp/clamav-8a80d96c081b2238b974aa425db1a646.tmp-daily.cld: Malformed database
Feb 22 19:25:53 mail freshclam[951]: Sat Feb 22 19:25:53 2020 -> !Failed to load new database: Malformed database
Feb 22 20:26:00 mail freshclam[951]: Sat Feb 22 20:26:00 2020 -> ^[LibClamAV] cli_loadhash: Malformed hash string at line 1158542
Feb 22 20:26:00 mail freshclam[951]: Sat Feb 22 20:26:00 2020 -> ^[LibClamAV] Can't load /var/lib/clamav/tmp/clamav-058dda19d2f23d673eee5bce68aced9e.tmp-daily.cld: Malformed database
Feb 22 20:26:00 mail freshclam[951]: Sat Feb 22 20:26:00 2020 -> !Failed to load new database: Malformed database
Feb 22 21:26:08 mail freshclam[951]: Sat Feb 22 21:26:08 2020 -> ^[LibClamAV] cli_loadhash: Malformed hash string at line 1164532
Feb 22 21:26:08 mail freshclam[951]: Sat Feb 22 21:26:08 2020 -> ^[LibClamAV] Can't load /var/lib/clamav/tmp/clamav-f7def1be8282325638b058ad9491e1aa.tmp-daily.cld: Malformed database
Feb 22 21:26:08 mail freshclam[951]: Sat Feb 22 21:26:08 2020 -> !Failed to load new database: Malformed database
Feb 22 22:26:31 mail freshclam[951]: Sat Feb 22 22:26:31 2020 -> ^[LibClamAV] cli_loadhash: Malformed hash string at line 1192476
Feb 22 22:26:31 mail freshclam[951]: Sat Feb 22 22:26:31 2020 -> ^[LibClamAV] Can't load /var/lib/clamav/tmp/clamav-001d63d4e78457d6483401430c63c605.tmp-daily.cld: Malformed database
Feb 22 22:26:31 mail freshclam[951]: Sat Feb 22 22:26:31 2020 -> !Failed to load new database: Malformed database
Feb 22 23:27:01 mail freshclam[951]: Sat Feb 22 23:27:01 2020 -> ^[LibClamAV] cli_loadhash: Malformed hash string at line 1250779
Feb 22 23:27:01 mail freshclam[951]: Sat Feb 22 23:27:01 2020 -> ^[LibClamAV] Can't load /var/lib/clamav/tmp/clamav-bb32c824af9ee97ccc041edc6e2aa361.tmp-daily.cld: Malformed database
Feb 22 23:27:01 mail freshclam[951]: Sat Feb 22 23:27:01 2020 -> !Failed to load new database: Malformed database
Feb 23 00:27:32 mail freshclam[951]: Sun Feb 23 00:27:32 2020 -> ^[LibClamAV] Can't load daily.hsb: Malformed database
Feb 23 00:27:32 mail freshclam[951]: Sun Feb 23 00:27:32 2020 -> ^[LibClamAV] Can't load /var/lib/clamav/tmp/clamav-b69d10cce261c747dfcd742e91f017bf.tmp-daily.cld: Malformed database
Feb 23 00:27:32 mail freshclam[951]: Sun Feb 23 00:27:32 2020 -> !Failed to load new database: Malformed database
Feb 23 01:28:00 mail freshclam[951]: Sun Feb 23 01:28:00 2020 -> ^[LibClamAV] cli_loadhash: Malformed hash string at line 1250779
Feb 23 01:28:00 mail freshclam[951]: Sun Feb 23 01:28:00 2020 -> ^[LibClamAV] Can't load /var/lib/clamav/tmp/clamav-13f5b884bb2860994a10f53f8427ed70.tmp-daily.cld: Malformed database
Feb 23 01:28:00 mail freshclam[951]: Sun Feb 23 01:28:00 2020 -> !Failed to load new database: Malformed database
Feb 23 02:28:29 mail freshclam[951]: Sun Feb 23 02:28:29 2020 -> ^[LibClamAV] cli_loadhash: Malformed hash string at line 1265691
Feb 23 02:28:29 mail freshclam[951]: Sun Feb 23 02:28:29 2020 -> ^[LibClamAV] Can't load /var/lib/clamav/tmp/clamav-92269043eee7e9e5940f7feea613beff.tmp-daily.cld: Malformed database
Feb 23 02:28:29 mail freshclam[951]: Sun Feb 23 02:28:29 2020 -> !Failed to load new database: Malformed database
Feb 23 03:28:58 mail freshclam[951]: Sun Feb 23 03:28:58 2020 -> ^[LibClamAV] cli_loadhash: Malformed hash string at line 1262755
Feb 23 03:28:58 mail freshclam[951]: Sun Feb 23 03:28:58 2020 -> ^[LibClamAV] Can't load /var/lib/clamav/tmp/clamav-fab41eff4999233ad46615056189ec45.tmp-daily.cld: Malformed database
Feb 23 03:28:58 mail freshclam[951]: Sun Feb 23 03:28:58 2020 -> !Failed to load new database: Malformed database
Feb 23 04:29:27 mail freshclam[951]: Sun Feb 23 04:29:27 2020 -> ^[LibClamAV] cli_loadhash: Malformed hash string at line 1260234
Feb 23 04:29:27 mail freshclam[951]: Sun Feb 23 04:29:27 2020 -> ^[LibClamAV] Can't load /var/lib/clamav/tmp/clamav-415612e07c6b3e9a7372efe70ddfaef4.tmp-daily.cld: Malformed database
Feb 23 04:29:27 mail freshclam[951]: Sun Feb 23 04:29:27 2020 -> !Failed to load new database: Malformed database
Feb 23 05:29:56 mail freshclam[951]: Sun Feb 23 05:29:55 2020 -> ^[LibClamAV] cli_loadhash: Malformed hash string at line 1253975
Feb 23 05:29:56 mail freshclam[951]: Sun Feb 23 05:29:55 2020 -> ^[LibClamAV] Can't load /var/lib/clamav/tmp/clamav-362eef3fb267c9aa6eed7aefce99479e.tmp-daily.cld: Malformed database
Feb 23 05:29:56 mail freshclam[951]: Sun Feb 23 05:29:55 2020 -> !Failed to load new database: Malformed database

Will I cause a disaster if I follow these instructions to reinstall ClamAV and fix it?

Should I do it from the admin or root account?

Raphael · February 23, 2020, 10:31am

It took me just a second to google it, I would follow this here first: https://www.howtoforge.com/community/threads/clamav-database-update-error.76634/

Wibol · February 23, 2020, 11:18am

Great! Everything seems to work now. The databases file extension changed from “* .cld” to “* .cvd”. These were the commands I used:

cd /var/lib/clamav/
ls -l
mv daily.cld daily.cld.old
mv main.cld main.cld.old
ls -l
systemctl restart clamav-freshclam
cat /var/log/clamav/freshclam.log
cat /var/log/clamav/clamav.log

Thank you very much.