Improving HestiaCP's mail subsystem (e.g. SA config, using dovecot for local delivery)?

Community Support Mail
1

I assume (from previous comments by the devs) that email services are not the main focus of HestiaCP development, but have you considered using Dovecot (LMTP or LDA) for local delivery ?

Here is some info:

Use Dovecot’s LMTP for local delivery:
https://wiki2.dovecot.org/LMTP/Exim

Use Dovecot’s LDA for local delivery:
https://wiki.dovecot.org/LDA/Exim

Some tutorial I found while Googling:
https://wiki.meurisse.org/wiki/Exim#Dovecot_Delivery

I would imagine that if we let only Dovecot write in the mail directories, it should work better.

Let’s discuss it here and if others think it is a good idea, I could open an issue (feature request) at Github.

Happy new year, KP

PS: Btw is it possible to create a HestiaCP domain from the command-line (like e.g. Virtualmin can)?

2

For those that are reading the docs yes.

https://docs.hestiacp.com/cli_commands/commands.html#v-add-domain

3

Why would I need more efficiency for a local delivery? What are the advantages? My CPU is always near zero

4

Thank you for your kind remark. I had already found v-add-domain. but I was looking for something like Virtualmin’s create-domain – Virtualmin (check its functionality)

5

To clarify my original post, basically I am trying to start a well-intentioned constructive discussion about improving HestiaCP’s mail subsystem.

E.g. Hestia’s default SpamAssassin configuration can be improved.

6

I haven’t used Exim4 before (in production), so I don’t know about possible advantages and disadvantages of letting Exim handle local delivery.

But I would like to check with others more knowledgeable whether using Dovecot LMTP might be a better option (for e.g. security, performance and features like sieve filter).

Or maybe changing local delivery from Exim to Dovecot would break existing functionality, which would make this whole discussion moot.

Thanks, KP.

7

Emails get delivered locally. For me it’s enough.

I think that we could improve:

  • Spam / ham config - someone is using sieve for that.
  • Sieve repository of handy scripts such as intercept mail…
  • Enable sieve with EXIM so we can intercept outgoing messages.
  • More flexibility with mail per hour quotas. Per domain / per account
  • Spam assassin filters
  • Whitelisting
  • Blacklisting
  • DNS checking
  • alarm / alerts
  • email compression

But I am happy with what I have. The thing is that I always want more.

8

While looking for ways to improve HestiaCP’s mail subsystem, I noticed that SpamAssassin spamd runs as root:

root@myserver:~# ps aux
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root         1  0.0  0.0 170648  6468 ?        Ss    2021   9:53 /sbin/init
root        38  0.0  0.1  83540 38824 ?        Ss    2021   5:35 /lib/systemd/systemd-journald
[...]
root       361  0.0  0.0 102124 10076 ?        Ss    2021  27:28 /usr/bin/perl -T -w /usr/sbin/spamd -d --pidfile=/var/run/spamd.pid --create-prefs --max-children 5 --helper-home-dir
root       404  0.0  0.0 102124  6052 ?        S     2021   0:12 spamd child
root       405  0.0  0.0 102124  6712 ?        S     2021   0:11 spamd child
clamav     407  0.0  3.7 1462456 1220068 ?     Ssl   2021  13:33 /usr/sbin/clamd --foreground=true
Debian-+   660  0.0  0.0  33560  7440 ?        Ss    2021   0:03 /usr/sbin/exim4 -bd -q30m
www-data  2678  0.0  0.0  53500  5660 ?        S    Jan15   0:01 nginx: worker process
www-data  2679  0.0  0.0  53500  4036 ?        S    Jan15   0:00 nginx: worker process
[...]

There exists a system user “debian-spamd” with shell access but it isn’t used by default. The spamd (perl) daemon binds to tcp/783 where Exim4 connects.

root@myserver:~# fgrep spam /etc/passwd
debian-spamd:x:113:117::/var/lib/spamassassin:/bin/sh
root@myserver:~# netstat -ntlp -4|fgrep 783
tcp        0      0 127.0.0.1:783           0.0.0.0:*               LISTEN      361/perl            
root@myserver:~#

root@myserver:~# grep -v ^# /etc/default/spamassassin |grep -v ^$
OPTIONS="--create-prefs --max-children 5 --helper-home-dir"
PIDFILE="/var/run/spamd.pid"
CRON=0
root@myserver:~# 

root@myserver:~# fgrep 783 /etc/exim4/exim4.conf.template 
spamd_address = 127.0.0.1 783
root@myserver:~#

At first thought, it seems that HestiaCP security on Debian 11 hosts can be improved by having spamd run as an unprivileged user (e.g. debian-spamd). For this we would need to have spamd bind to a higher port (e.g. tcp/1783)

By changing /etc/default/spamassassin accordingly e.g.
OPTIONS="-u debian-spamd -p 1783 --create-prefs --max-children 5 --helper-home-dir"

I will also have to check if there are differences with previous supported Debian releases (9 and 10) and how SA is configured under Ubuntu.

https://spamassassin.apache.org/full/3.4.x/doc/spamd.html

9

As with everything else, improvements are always welcome and needed. The main problem is time. There is not enough time to improve everything at once. So we always need to make compromises and prioritize what we need to do first.

There are some improvements that get higher priority than others. That does not mean that your proposal is not good. It might be that it gets lower priority because of other more important things.

Let me bring an example. We all know the IPv4 shortage is real and will get even worse in the future. Maybe some hosts will stop offering IPv4 VPS or charge a much higher price (like Hetzner who raised some IPv4 prices the start of 2022 and I’m sure they’re not the only ones). So, it makes more sense to give priority to IPv6 implementation than improving mail subsystem.

TLDR: Every suggestion is welcome. Just don’t be disappointed if it doesn’t get any immediate attention or priority. But feel free to work on it. So when the time is right we will be many steps ahead :slight_smile:

2 Likes
10
  • Spam / ham config - someone is using sieve for that.
  • Sieve repository of handy scripts such as intercept mail…
  • Enable sieve with EXIM so we can intercept outgoing messages.
  • More flexibility with mail per hour quotas. Per domain / per account

It is mostly done it mainly need some changes allowing to store for backups and some minor changes and it should be working

  • Spam assassin filters
  • Whitelisting
  • Blacklisting
  • DNS checking
  • alarm / alerts
  • email compression
11

Sure, as I wrote above “… I am trying to start a well-intentioned constructive discussion about improving HestiaCP’s mail subsystem”.

So let us submit some ideas/suggestions, even if they won’t be on the Roadmap for the near future.

12

Some more notes:

  1. To regularly update SpamAssassin rules regularly with cron (using sa-update) define CRON=1 in /etc/default/spamassassin

  2. To reject spam with very high score during SMTP-time it takes 3 lines in an Exim ACL:

deny message = Rejected by spam filter (score=$spam_score, req=8.0)
spam = nobody
condition = ${if >={$spam_score_int}{80}{1}{0}}
13

Another note:

  1. PER-USER SpamAssassin filtering by invoking spamc (e.g. train the filter every time a user moves a mail from Inbox to Spam).
2 Likes
14

This should be a must.

15

Issue what should be de default behaviour. Marking something as spam maybe yes. Deleting it on default probally not…

1 Like
16

@eris your take on it looks nice and complete!

Another option would be to use Dovecot system-wide sieve filters to move X-Spam-Status: Yes mails to the Junk/Spam folder (instead of changing the Subject to *** SPAM ***). The only downside of this approach would be that users using POP3 instead of IMAP wouldn’t get those mails dumped in Junk.

I don’t see a big problem, with rejecting mails with super-high spam score (>10) as long as you do it properly during SMTP-time (and not with a DSN after accepting it) . Score >10 is either unquestionably spam or a misconfigured mailserver (e.g. SPF).

17

It’s not trivial. There are many different ways to invoke spamc to learn spam/ham. Based on a quick Google search, most online tutorials use postfix and invoke spamc from master.cf

I found a guide on how to invoke spamc with Dovecot Sieve filters here:
https://doc.dovecot.org/configuration_manual/howto/antispam_with_sieve/
https://docs.iredmail.org/dovecot.imapsieve.html

1 Like
18

I made a git repo of configs, scripts and installer for spam ham learning by user moving email to from junk folder. The installer isn’t perfect. Need a few.manual changes after. But most of the work is done if someone wants to check it out make some improvements go ahead. It’s a good startingroung.

2 Likes