How to configure HestiaCP / Exim 4 for SMTP Relay (SendGrid, Mailgun, Amazon SES, etc)

Hi Guys,

I would like to use Sendgrid to relay all emails from the server and have followed the below

Sadly the configuration file update-exim4.conf.conf doesn’t seem to be included on loading the service and mail is still being delivered directly from the server.

I’ve been through a number of posts on the VestaCP forums which seem to refer to a different implementation of Exim.

Does anyone have this working? Thanks for your time.

2 Likes

I’ve figured this out for mailjet. You probably want to be editing exim4.conf.template

Under ‘begin authenticators’ I added

mailjet_login:
  driver = plaintext
  public_name = LOGIN
  client_send =:apiuser:apikey

Under ‘begin routers’ I added

 send_via_mailjet:
   driver = manualroute
   domains = ! +local_domains
   transport = mailjet_smtp
   route_list = * in-v3.mailjet.com

// This sends all non-local domains through mailjet. You can add specific domains if you want. The route_list is the name of the mailjet server I’m sending through.

Under ‘begin transports’ I have

mailjet_smtp:
  driver = smtp
  port = 587
  hosts_require_auth = *
  hosts_require_tls = *

That was pretty much it. Chang the port as neccessary. Put your real apiuser and apikey in between the colons

5 Likes

Thank you for your help.

I carried out the following to get it working with sendgrid

Under ‘begin authenticators’ I added

sendgrid_login:
  driver = plaintext
  public_name = LOGIN
  client_send = : apikey : key

Note: username should actually be ‘apikey’, not your sendgrid username

Under ‘begin routers’ I added

send_via_sendgrid: 
  driver = manualroute
  domains = ! +local_domains
  transport = sendgrid_smtp
  route_list = "* smtp.sendgrid.net::587 byname"
  host_find_failed = defer
  no_more

Under ‘begin transports’ I added

sendgrid_smtp: 
  driver = smtp
  hosts = smtp.sendgrid.net
  hosts_require_auth = <; $host_address
  hosts_require_tls = <; $host_address

Under router configuration for alias and catchall I also changed:

redirect_router = dnslookup

to:

redirect_router = send_via_sendgrid
4 Likes

hey guys, thanks a lot for sharing, I am sure this is helpful for other community members as well!

going to add infos about using amazon SES here as well:

Under ‘begin authenticators’ I added

ses_login:
driver = plaintext
public_name = LOGIN
client_send = : user : password

Under ‘begin routers’ I added

send_via_ses:
driver = manualroute
domains = ! +local_domains
transport = ses_smtp
route_list = * email-smtp…amazonaws.com;

Under ‘begin transports’ I have

ses_smtp:
driver = smtp
port = 587
hosts_require_auth = *
hosts_require_tls = *

as one can see it’s pretty much the same, external aws guide here: https://docs.aws.amazon.com/ses/latest/DeveloperGuide/send-email-exim.html

keep in mind that the domains usually need to be verified within your mail relay service first, also the simple configuration above is used for all domains, so if you want to have it more specific - change as needed (and maybe report back) :wink:

3 Likes

So the way to limit to certain domains is under the ‘routers’ stanza.
Where you have this line saying “applies to all non local recipient domains”
domains = ! +local_domains
You can add a 'senders = ’ line to specify which sending addresses this gets applied to eg.

domains = ! +local_domains
senders = *@domain1.com : *@otherdomain.net : *@alt2domain.org
1 Like

1.4 will come with build in support for mail relay on user level and admin level

1 Like

Just wanna share 1 more workaround which can be used for multiple domains, and will allow to use different SMTP by domain (MailJet, MailGun, SendPulse, SendGrid etc.).

  1. Edit exim4.conf.template
1.1 Add after begin authenticators
# Smart Host Sending
sendbysmarthosts:
  driver = plaintext
  public_name = LOGIN
  hide client_send = : ${extract{user}{${lookup{$sender_address_domain}lsearch{/etc/exim4/exim_smarthosts}}}} : ${extract{pass}{${lookup{$sender_address_domain}lsearch{/etc/exim4/exim_smarthosts}}}}
1.2 Add after begin routers:
# Smart Host Sending
sendbysmarthostsrouter:
  driver = manualroute
  domains = ! +local_domains
# senders can be used to limit email addresses, for example: senders = [email protected] : *@domain2.com
# senders = [email protected]
  condition =  "${if eq{${lookup{$sender_address_domain}partial-lsearch{/etc/exim4/exim_smarthosts}{$value}}}{}{false}{true}}"
  ignore_target_hosts = 0.0.0.0 : 127.0.0.0/8
# headers_add = "${perl{mailtrapheaders}}"
  transport = sendbysmarthoststransport
  route_list = "* ${extract{smtp}{${lookup{$sender_address_domain}lsearch{/etc/exim4/exim_smarthosts}}}}::${extract{port}{${lookup{$sender_address_domain}lsearch{/etc/exim4/exim_smarthosts}}}} byname"
# host_find_failed = defer
# no_more
1.3 Add after begin transports
# Smart Host Sending
sendbysmarthoststransport:
  driver = smtp
  port = ${extract{port}{${lookup{$sender_address_domain}lsearch{/etc/exim4/exim_smarthosts}}}}
  hosts_require_auth = $host_address
  hosts_require_tls = $host_address
  1. Create file /etc/exim4/exim_smarthosts (root:root, 644), with next content:
exim_smarthosts
# "domain" - domain name used in index of the line
# "user" - user (or ID) used in smarthost's SMTP
# "pass" - password (or API secret, or API key, etc.) used for the SMTP user in smarthost
# "smtp" - SMTP server address of smarthost
# "port" - port used in Smarthost (586,25 or another)

# sending by MailJet
# $domain: domain=$domain user=$MAILJET_smtp_user pass=$MAILJET_api_password smtp=in-v3.mailjet.com port=587
domain1.com: domain=domain1.com user=xxxxxx pass=xxxxxx smtp=in-v3.mailjet.com port=2525

# sending by MailGun
# $domain: domain=$domain [email protected]$domain pass=$MailGun_default_password smtp=smtp.eu.mailgun.org port=587
domain1.com: domain=domain1.com [email protected] pass=xxxxxx smtp=smtp.eu.mailgun.org port=587
domain2.com: domain=domain2.com [email protected] pass=xxxxxx smtp=smtp.eu.mailgun.org port=587

# sending by SendPulse
# $domain: domain=$domain user=$SENDPULSE_smtp_user pass=$SENDPULSE_api_password smtp=smtp-pulse.com port=587

# sending by SendGrid
# $domain: domain=$domain user=$SENDGRID_smtp_user pass=$SENDGRID_api_password smtp=smtp.sendgrid.net port=587
  1. Restart Exim: sudo service exim4 restart

Good to know:

  • domains must be verified on SMTP relay account
  • domains not existing in /etc/exim4/exim_smarthosts will use local SMTP
  • in case there are 2 configs for 1 domain in /etc/exim4/exim_smarthosts only upper will be used: In example above emails from domain1.com will be sent by MailJet, and domain2.com by MailGun
2 Likes

Hello falzo.
Thanks for the information, it was very helpfull when setting up my AWS server.
Im reaching to you, to ask you about the configuration for SMTP relay for Forwarding emails.
Internal Forwarding works, but when trying to Forward to an external email (gmail) I have the 550 error (because AWS blocks SMTP) my guess is that Hestia is trying to Forward with regular SMTP and not with the relay (I have my domains validated by SES and working for sending)
Do you have any ideas of the configuration I need to set in order to send Forward emails using SES?

Thanks for taking the time of reading this, please be free to reply any thoughts you have on the subject.

Regards, koode.mx

I’d assume that SES picks up the original senders domain and obviously does not relay that, because that is not validated (and nothing you can actually do)

to be honest I never really looked into that so far as it did not concern my use cases. there is something like Sender Rewriting Scheme - Wikipedia - maybe that helps, but I haven’t done this yet in exim.

another option probably would be to check if you can exclude forwards from using SES and have exim send those directly…

1 Like

Hey falzo. We recently started using Mailjet to relay our emails from Hestia server. Our droplet’s IP address was blacklisted and all our emails were going to spam. The current configuration from pluto makes Mailjet work.

However, we have the same forwarding problem as @koodemx.
I had to change router configuration for alias to make forward mails use the relay. As @lebeef suggested:

redirect_router = send_via_mailjet

As you said, the original sender address pops up on Mailjet, waiting for confirming the mail address.

I did not take a look at any specific solutions but if I use the server’s SMTP, the forwarded mail will go to spam. If I relay it to Mailjet, it won’t accept it.

2 Likes

yeah forwarded mail are a big problem nowadays anyway. the ‘why?’ is answered quickly - a very high percentage of the forwarded mail will be just spam. and while people might think ‘I don’t care because gmail as final destination has a good spamfilter and will take care’ they simply oversee that they become part of the problem. in the eyes of the final receiving provider yourserver/ip is going to be the origin of exact that spam mail and with that your IPs reputation will die and sonn end up on a blacklist.

worst case you combine forwarding with a catch-all because of FOMO. heaven for spammers and you’ll bear the costs.

so, best way is to try and avoid forwarding at all costs.for regular big services you could setup pop-collection instead - which will also have the mails filtered again spam but never influence your IPs reputation.
I understand that this sometimes not feasible or at least hard to explain to clients - still the best option to go with.

the second option would be to split between SES and regular sending just for forwarded mail, but that probably won’t get you far as the problem stays the same and the relation between good and bad mails send from your host probably even gets worse.

last option would be getting into SRS, sender rerwriting scheme as written above. with that for all forwarded mails the header gets rewritten so that the mailfrom domain then is yours and can be validated with SES or mailjet and the likes so the mail can pass there.
however chances here are as well, that some of them bounce because spam will still be spam. those bounces might hurt your reputation within the relay services you use, so you need to heavily monitor that because you might run into trouble there… kind of knockon effect.

TL;DR; do not forward mail

2 Likes

Hello @arinc9.
Mailjet forwarding won’t work because Exim is trying to forward the email keeping the FROM header (the FROM email is not going to be part of your valid sender addresses (ie: gmail)).
I encountered this problem when changing the router for aliases and forwarders on Exim.
Since then Ive been looking for a solution, and I am stuck on trying to rewrite the FROM Header address before sending the email.
I found an experimental SRS support for Exim here (Sender Rewriting Scheme (SRS) Experimental) but you need to install an extra library.
Also I found the Exim documentation for Address rewriting but Im confused and I dont have experience on setting the rules on the config files (My best guess is that you need to set a wildcard to ALL FROM Addresses and rewrite them to you new verified email address) but I dont really know how to do this.
@falzo Thank you so much for taking the time to reply, I understand your concern about the SPAM, but in my case, I only want the forwarded email to reach my Gsuite email account for internal use, I dont worry about spam since my Hestia emails accounts are not public.

Please be free to reply any thoughts you have on the subject.

Regards, koode.mx

1 Like

as said above, if it’s just a single or few accounts simply have your mail address(es) be regular accounts in hestia with no forwarding and use the option to collect the mails from gsuite itself via pop3 directly (and eventually delete them afterwards). that’s what I do and recommend to my clients and saw the least problems with…

I haven’t looked into SRS yet, but it’s on my todo as well…

1 Like

Hello @falzo
Thanks for the reply.
Im making some test with information I found online.
This is my aliases exim config

aliases:
driver = redirect
headers_remove = From
headers_add = From: [email protected]_domain
#headers_add = X-redirected: yes
data = ${extract{1}{:}{${lookup{[email protected]$domain}lsearch{/etc/exim4/domains/$domain/aliases}}}}
require_files = /etc/exim4/domains/$domain/aliases
redirect_router = send_via_ses
#redirect_router = dnslookup
pipe_transport = address_pipe
unseen

Im trying to remove the FROM header of the original email, and rewrite to the original TO address.

But this configuration is not working though.

Thanks in advance for your comments.

I eventually backed down on the idea of using relay services for forwarded mails.
I wouldn’t want forwarded mails to consume the daily email quota from Mailjet which is 200.
Also as @falzo said, forwarded spam mails will get us in trouble with the relay service.

I set up that we forward mail using the Hestia server’s SMTP instead of relaying it to Mailjet.
It will go to spam on the recipient but it will reach the other side, at the very least.

Current configuration from @pluto works this way, so it’s pretty much ready to go.

1 Like

to add to that: your recipients probably are aware of these forwarded mails coming in, so they should simply check their spam folder regularly and more importantly always mark non-spam mails as such. ideally on the webmail thing or native app - whatever they are using. maybe even add the contacts to their trusted contact list etc. in the long run this might help with the reputation.

1 Like