What about implementing dovecot sieve?

@Machiel92 - Yes. Altered setup shell script to install Rspamd instead of SA.

@anon95304325 - Seems like I will have to find some time every day and come up with the tuto :smiley:

@salnz & @Raphael - This is a way to go. However those are resource intense solutions, unlike Hestia w/ Rspamd. Not all people here wanna pay for additional VPS.

Note: I don’t use ClamAV as it’s not needed IMO unless you use unpatched browser and/or opening malicious attachments/parts of messages.

1 Like

@martineliascz You can run Mailcow on cheap 1 - 2gb vps from as low as $5 p/m just disable clamav which I agree is not needed just too resource hungry
.
If on a tight budget you can use Mail-in-a-box which will run on 512mb vps.

I am looking to replace Exim4 on a new Hestia install this week with unsupported Postfix and Rspamd as prefer it over Exim4

2 Likes

Great! I was forced to visit my hometown earlier than I planned - obviously due to coronavirus :smiley:

@salnz I’m staying home for 2 weeks. Let me first upgrade Debian, Hestia and we can come up with tutorial together as I used some of your tips & tricks from several posts AFAIK.

I already sent e-mail to @anon95304325 - waiting for his reply.

Saw your mail in the ticketing system and has added it to Kristan, probaly he missed that :slight_smile:.

1 Like

@salnz, I noticed that you included the dovecot-lmtp package when you described getting sieve running on Ubuntu. Did you, then, change the delivery transport for “localuser” and “localuser_spam” in exim4.conf.template? The way it’s configured now, exim delivers new email directly into the user’s mail directory with the “appendfile” driver. I was wondering if Hestia can work with exim configured to use LMTP to deliver the message to Dovecot.

Thanks
–Dan

@fluidmindorg yes I did install it and used LMTP, have also configured Dovecot with LDA
It is also possible to configure LMTP to be listening on TCP and there is a link at the bottom with details

The guide I used as a base was from DA which used Exim4

[How-To] Exim + Dovecot + Sieve + Roundcube + ManageSieve | DirectAdmin Forums

Have some spare time due to COVID-19 lock down here in New Zealand, every non essential business must close for 4 weeks and everyone to remain home unless going to the supermarket, doctors, chemist so will be able to complete a full test install and will document everything, I will be using Ubuntu as the notes I have some items got errors in Debian 10

Thanks @salnz. Sounds good. Tell ya what: you work it out on Ubuntu and I’ll work out any errors on Debian 10 :slight_smile:.

The main reason I want to use LMTP is that I’d like to set Dovecot to use the Maildir “LAYOUT=index” storage format for mailbox folder names. As I discussed in a separate thread a few days ago, the “LAYOUT=fs” format allows for periods in folder names, and enabling the listescape plugin allows for other characters (like a forward slash). But the LAYOUT=index format alleviates all issues that might arise from limitations of the underlying filesystem by saving mail folders in filesystem directories named with the GUIDs of the mail folders instead of the actual names of those folders. The problem is that the way Hestia currently has Exim configured (to place the incoming message directly into the INBOX folder) can’t work with that, because Exim has no way of knowing the GUID of any given user’s INBOX.

There might be another reason to use LMTP: As the Dovecot docs say, the Dovecot delivery agent (LDA or LMTP) keeps the Dovecot index files up to date. I’m not sure if that happens when you have Exim place messages into the Maildir folders directly (that is, bypass Dovecot’s delivery agent).

(Regarding LDA, the impression I get from various discussion threads is that LDA is the old Dovecot delivery agent and that LMTP is now preferred. In fact, Dovecot’s own documentation on LDA says just that: “Nowadays you should probably use the LMTP server instead, because it’s somewhat easier to configure and gives better performance.”)

1 Like

Alrighty! I got LMTP delivery working from Exim to Dovecot. Config changes below.

I presume, @salnz, that the idea is for sieve to take care of checking the X-SPAM headers and putting a spam message into the Junk folder instead of INBOX, and so we would no longer need the “localuser_spam” router and “local_spam_delivery” transport in the Exim config?

Configuring Exim to deliver messages to Dovecot through LMTP on Debian 10:

(1) Install dovecot’s LMTP component:

apt install dovecot-lmtpd

(2) Edit /etc/dovecot/conf.d/20-lmtp.conf

Add the following two lines at the top (below the first comment)

# Uncomment to enable LMTP
protocols = $protocols lmtp

Uncomment the following line and set it to “yes”:

lmtp_rcpt_check_quota = yes

Down in the “protocol” section, uncomment and modify the mail_plugins line as follows:

mail_plugins = $mail_plugins sieve quota

(3) Edit /etc/dovecot/conf.d/15-lda.conf, uncomment and set the following directives to “yes” (LMTP uses the configuration of LDA in Dovecot):

lda_mailbox_autocreate = yes
lda_mailbox_autosubscribe = yes

(4) Edit /etc/exim4/exim4.conf.template

Under the “localuser” router, change “transport” to the following:

transport = dovecot_lmtp

Add the following new transport in the transports section:

dovecot_lmtp:
  driver = lmtp
  socket = /var/run/dovecot/lmtp
  batch_max = 200
  rcpt_include_affixes
  delivery_date_add
  envelope_to_add
  return_path_add

(5) Restart both Dovecot and Exim (Dovecot first, so that it can create the lmtp Unix socket):

systemctl restart dovecot
systemctl restart exim4

P.S.: Forgive me if I ask any lame questions or have wrong assumptions about Exim. I’ve been using Postfix for over 20 years, so this is the first time I’ve had to deal with Exim’s configuration.

4 Likes

@fluidmindorg That’s true, I have “localuser_spam” commented out as well and let Dovecot sieve and Rspamd take care of that itself.

Rspamd - I am working on tutorial, but only 20% done. Current pandemic situation is worse than I thought.

1 Like

Okay, I got sieve working with Roundcube and with a default filter that checks for either SpamAssassin’s or rspamd’s headers.

I’m pretty sure this is everything I did to make it work. Had to improvise a lot from those old tutorials because they were pretty outdated, but it seems simpler than they had done.

(1) Edit /etc/dovecot/conf.d/20-managesieve.conf

Uncomment the following line to enable the sieve protocol:

protocols = $protocols sieve

If any of the following lines are commented, uncomment them:

service managesieve-login {
  inet_listener sieve {
    port = 4190
  }

  service_count = 1
  process_min_avail = 0
}

service managesieve {
  process_limit = 1024
}

protocol sieve {
  managesieve_max_line_length = 65536
  managesieve_implementation_string = Dovecot Pigeonhole
}

(2) Create global sieve directory along with empty files where rules can be placed that will be executed before or after the user’s rules, respectively.

mkdir /etc/dovecot/sieve
touch /etc/dovecot/sieve/before.sieve
touch /etc/dovecot/sieve/after.sieve

(3) Edit /etc/dovecot/sieve/default.sieve and paste the following:

require ["fileinto", "regex", "date", "relational", "vacation", "imap4flags", "envelope", "subaddress", "copy", "reject"];

# rule:[Spam Filter]
if anyof (header :contains "X-Spam-Flag" "YES", header :contains "X-Spam" "Yes") {                                                                                    
  fileinto "Junk";
  stop;
} 

(4) Edit /etc/dovecot/conf.d/90-sieve.conf

Uncomment the “sieve” line and set its value to the following:

sieve = file:~/mail/%d/%n/sieve/;active=~/mail/%d/%n/sieve/managed.sieve

(That will result in a “sieve” directory being automatically created in the user’s mail directory the first time the RoundCube “Filters” tab in Settings is clicked, and the “default.sieve” copied to it as the first rule.)

Uncomment the “sieve_before” and “sieve_after” lines and set them to the following:

sieve_before = /etc/dovecot/sieve/before.sieve
sieve_after = /etc/dovecot/sieve/after.sieve

(5) Edit /etc/roundcube/config.inc.php

Go to the $rcmail_config[‘plugins’] line and add ‘managedsieve’ to the list of plugins:

$rcmail_config['plugins'] = array('jqueryui','password','managesieve');

Then add the following lines below that one:

// Dovecot managedsieve TCP port
$rcmail_config['managesieve_port'] = 4190;                                               
                              
// Default contents of filters script (eg. default spam filter)
$rcmail_config['managesieve_default'] = '/etc/dovecot/sieve/default.sieve';
2 Likes

@fluidmindorg if you are using rspamd you can easily setup dovecot to send emails into the spam folder to train them as spam, and out of the spam folder to train them as ham for rspamd

Works well and use in my setups

@martineliascz might be useful for you as well

see this link below which has install script and rules you can change to your requirements

What does it do?

  1. Implement the imap_sieve rules as mention in the wiki link above and provide also the scripts to call rspamc.You need to set the password for rspamc in /etc/dovecot/rspamd-controller.password .This means we can move mails into the spam folder to train them as spam, and out of the spam folder to train them as ham.
  2. Configure spamtest extension. The included config defaults to the score based rspamd headers but examples are provided for other options.
  3. Global rule using the spamtest extension to sort all mails that are 100% spam into the spam folder. The rational behind the global rule is that we want all all spam mails in the spam folder. If the user moves them out of there afterwards, we learn them as ham. Anything that wasn’t detected as 100% spam yet will be trained as spam if we move mails in. so this supports the first point

GitHub - darix/dovecot-sieve-antispam-rspamd: Dovecot Antispam with Sieve (and rspamd)

1 Like

Cool! Is there any benefit between this solution and this one? learning-ham/spam is working fine for me using the latter one. Btw how well is learning ham/spam working for you? I have to learn like 100+ e-mails in order for Bayesian to do it’s work well.

Sweet! Thanks @salnz. I have to say, that would really awesome to be able to just click the Junk button in the Mac’s Mail.app and have that trigger the server to make the spam filter learn. On Zimbra I have to go to Zimbra’s webmail app to have it do that.

In order for this to work, do I assume correctly that Exim config needs to be changed in some way to stop using SpamAssassin and start using rspamd instead?

@martineliascz the link I posted sends the junk/non junk mail moved between the user inbox and junk folders to rspamd for learning just like you would manually do it inside the rspamd panel.

It is like the roundcube integration in some of the guides but for imap

eg for example if you get a email from hestia forum and it goes to junk folder when you move it to the inbox rspamd will learn it as not junk.

If you get one of those spam emails about mobile app developer that usually go to the inbox you move it to junk and rspamd will learn it as junk.

My spam learning is working well, there are only at least 1 - 5 items that slip through to inbox and junk folder per week on external mail server,

I am about to start again to install postfix to replace exim4 on my dev hestia 1.1.2 vps as ran into a couple of bugs trying to install postfix admin using Debian 10 and trying again with Ubuntu

1 Like

@fluidmindorg you would need to install and configure rspamd instead of spamassassin. but you could continue with your current setup with spamassassin and modify this imap method.

Report spam: Move email from Inbox to Junk
Report ham: Move email from Junk to any other folder (except Trash)

Link

Auto learn spam/ham with Dovecot `imap_sieve` plugin

Cool. Both tutorials seems very similar. The behavior of moving e-mails between inbox/spam & learning connected therewith is working well for me. I thought there are some benefits between the two tutorials (links) I posted.

Got Rspamd installed. Working on a way now to integrate it a little better into the Hestia web interface.

Now for the big question, @Raphael : with these rather severe changes (replacing SpamAssassin, changing the mechanism Exim uses to deliver mail, changing how Dovecot stores the mail, adding Sieve to Dovecot and Roundcube, etc.), how much more difficult am I making it on myself for the future when I need to do a Hestia 1.2 upgrade, and 1.3 upgrade, etc.?

Wha’d y’all think of this placement for a link to Rspamd from the Hestia web interface?

Edit /usr/local/hestia/web/templates/admin/list_services.html and add the following lines between the “Graphs” and “IP” lines:

      <?php if ($_SESSION['ANTISPAM_SYSTEM'] == 'rspamd') { ?>
      <a target="_blank" href="/rspamd/" class="ui-button cancel" title="<?=__('Rspamd')?>"><i class="fas fa-envelope-square status-icon purple"></i> <?=__('Rspamd')?></a>
      <?php } ?>

(And, of course, change ANTISPAM_SYSTEM to ‘rspamd’ in hestia.conf.)

Usualy, we do not change any config files, so you’ll be good to go with your changes.

1 Like

@fluidmindorg if you want to have it visible all the time you could probably replace the records location as it is the same as users but in the server area it looks good there.

When I added rspamd link I had to write protect the main theme template file to prevent it being removed on updates

1 Like