Common debugging tasks (spam, php spam scripts)

This may be out of the scope of Hestia Devs…but is just an incomplete idea.

Most of us have to deal with similar problems. For example, been blocked in a spam list.
How good will be to have a command like this

v-list-top-mail-senders # to see 10 top sending mail accounts in the server in the last 24hs
v-list-top-mail-senders user # to see 10 top sending mail accounts from user in the server in the last 24hs
v-list-top-mail-senders user domain.com # to see 10 top sending mail accounts from user’s specific domain in the server in the last 24hs
v-list-top-mail-senders 100 # to see 100 top sending mail accounts in the serve in the last 24hs

This can give you a fast view of what is going on.
The oder thing will be to know which are the top PHP scripts sending email in the last 24hs.

v-list-top-php-mail-senders # to see 10 top mail sending php scripts in the last 24hs
v-list-top-php-mail-senders user # to see 10 top mail sending php scripts in the last 24hs
v-list-top-php-mail-senders user domain.com 50 # to see 50 top mail sending php scripts in the last 24hs

I know it fills strange !! But having those commands will do my life much simpler.
They can be use to cron reports on the web interface and have an idea of what was going on lately.

I know there are chains of commands that do that, but the idea behind Hestia is to simplify admins’ life.

Lets discuss

interesting idea! though I agree that this might be out of scope for quite a while…

there are existing scripts like pflogsumm (that’s for postfix, never tried to make it work with exims logfiles) which you can use to generate daily statistics - maybe that could be a starting point to build something that we can integrate later :wink:

for the second wish, getting a list of php-scripts sending out mails, I think is nothing feasible at all. this is simply due to the fact that there is no logging for the initating script, at least as far as I am aware.
and there are a whole lot of different php mailing classes around, so you would need to know what your clients are using and somehow implement some logging/tracking of the script. and even if you would manage to do so this still won’t be accurate as users could use smtp auth within php-scripts instead etc.

1 Like

Hi, I make a small ssh script to see that info. But only the emails sent by autenticated users, and only to an external smtp. I have completely disable php script emails. Only allow by authentication via smtp.
Maybe you can modify to feet your needs:

#!/bin/bash -l
CURRENT_DIR=`dirname $0`
if [[ $1 ]]; then
  EXIM_LOG=$1
else
  EXIM_LOG=/var/log/exim4/mainlog
fi

OUT_MAIL=`cat $EXIM_LOG | grep "=>" | grep "remote_smtp"`
OUT_MAIL_USERS=`echo "$OUT_MAIL" | sed -n 's/.*F=\([-_+=<>@a-zA-Z0-9.]*\)\(.*\)/\1/p' | sort -u`
OUT_MAIL_DOMAINS=`echo "$OUT_MAIL_USERS" | sed -n 's/.*@\([-_+=a-zA-Z0-9.]*\)\(.*\)/\1/p' | sort -u`

OUT_MAIL_COUNT=0
OUT_USER_COUNT=0
OUT_DOMAIN_COUNT=0

OUT_MAIL_USERS_RESULTS=()
OUT_MAIL_DOMAIN_RESULTS=()

while read -r USER; do
    let OUT_USER_COUNT++
    USER_OUT_MAIL=`echo "$OUT_MAIL" | grep "$USER"`
    #MAIL_TO=$(echo "$line" | awk -F " " '{print $5}')
    USER_MAIL_COUNT=0
    while read -r MAIL; do
      let USER_MAIL_COUNT++
      let OUT_MAIL_COUNT++
    done <<< "$USER_OUT_MAIL"
    OUT_MAIL_USERS_RESULTS+=("$USER_MAIL_COUNT - $USER")
done <<< "$OUT_MAIL_USERS"

while read -r DOMAIN; do
    let OUT_DOMAIN_COUNT++
    DOMAIN_OUT_MAIL=`echo "$OUT_MAIL" | grep "$DOMAIN"`          
    DOMAIN_MAIL_COUNT=0
    while read -r MAIL; do
      let DOMAIN_MAIL_COUNT++
    done <<< "$DOMAIN_OUT_MAIL"
    OUT_MAIL_DOMAIN_RESULTS+=("$DOMAIN_MAIL_COUNT - $DOMAIN")     
done <<< "$OUT_MAIL_DOMAINS" 

echo "----------------------------------------------------------"
echo "Total outgoing domains: $OUT_DOMAIN_COUNT"
echo "Total outgoing emails addresses: $OUT_USER_COUNT"
echo "Total outgoing email messages: $OUT_MAIL_COUNT"
echo "----------------------------------------------------------"
echo "----------------------------------------------------------"
echo "----- Outgoing emails per domain:"
echo "----------------------------------------------------------"
printf '%s\n' "${OUT_MAIL_DOMAIN_RESULTS[@]}" | sort -n -r
echo "----------------------------------------------------------"
echo "----- Outgoing emails per email address:"
echo "----------------------------------------------------------"
printf '%s\n' "${OUT_MAIL_USERS_RESULTS[@]}" | sort -n -r
echo "----------------------------------------------------------"

Good!
But is not working.
Check my output

root@gato:~# ./mailchk.bash
----------------------------------------------------------
Total outgoing domains: 1
Total outgoing emails addresses: 1
Total outgoing email messages: 132
----------------------------------------------------------
----------------------------------------------------------
----- Outgoing emails per domain:
----------------------------------------------------------
132 -
----------------------------------------------------------
----- Outgoing emails per email address:
----------------------------------------------------------
132 -
----------------------------------------------------------
root@gato:~#

Check if you have this line in your exim config:

log_selector = +tls_sni +address_rewrite +all_parents +arguments +connection_reject +delay_delivery +delivery_size +dnslist_defer +incoming_interface +incoming_port +lost_incoming_connection +queue_run +received_sender +received_recipients +retry_defer +sender_on_delivery +skip_delivery +smtp_confirmation +smtp_connection +smtp_protocol_error +smtp_syntax_error +subject +tls_cipher +tls_peerdn

I don’t have that line at /etc/exim4/exim4.conf.template

  1. is that the correct file name?
  2. Why should I put that line?
  3. Where should put it?

Thanks !

Ramiro, have you tried Isoqlog on Hestia or vesta?
You can install it with apt.
Do you have any chance to try it?
I have posted this in a new topic.

Thanks

I think the log_selector = that @ramirojoaquin proposed will add more things to be logged in the log file, which later on are accounted for by the script.

One simple command I use to track the number of email messages is the following:
grep '<=' /var/log/exim4/mainlog | awk '{print $5}' | grep \@ | sort | uniq -c | sort -nrk1
But mind you! This is not perfect, nor does it give lots of information. I only use it to get a rough estimation.

You can try with eximstats https://manpages.ubuntu.com/manpages/bionic/man8/eximstats.8.html

You can get a html report or a txt report, that you can send to your own email