Dovecot gz trailer has wrong CRC value

After installing dovecot I broke my server and moved the maildir to a different one. After the move I enabled gzip and (I think this was my mistake) moved directories and files from mozilla Thunderbird.
Now I have the following problem. Some messages are broken and dovecot can’t read them, resulting in a folder loading problem with the mail client.
That is, I delete the index files in the folder sent, for example, I launch Mozilla, it starts to download the folder from the server. At some point an error appears in the dovecot.log with a reference to a message file and the download of messages stops.

read /path/to/message failed. gz trailer has wrong CRC value at 8192 (read reason=mail stream)

I tried to move the damaged emails, but the problem is that dovecot stops scanning the folder after the first error, and to see the next one, I have to move the email, delete the index files and start creating them again.
How can I fix the problem or find all the broken emails to move at once?

I tried to read broken emails with gunzip -c and they open, but at the end of the email there is probably an image in base64 followed by a warning of CRC.

Content-Type: multipart/mixed; boundary="------------egJdRmaVhjH2gk4jyDoim1xN"
Message-ID: <$$>
Date: Mon, 3 Oct 2022 16:06:17 +0300
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101
 Thunderbird/91.13.1
Subject: =?UTF-8?B?UmU6INGC0LDQsdC70LjRhtCwINC6INC30LDQv9C+0LvQvdC10L3QuNGO?=
 =?UTF-8?B?INGB0LXQvdGC0Y/QsdGA0YwgMjAyMg==?=
Content-Language: ru
To: Marina Nikitina <M*ru>
References: <[email protected]>
From: =?UTF-8?B?0JrQsNGI0YLQsNC90L7QstCwINCQ0L3QsNGB0YLQsNGB0LjRjyDQrtGA0Yw=?=
 =?UTF-8?B?0LXQstC90LA=?= <*>
In-Reply-To: <*>

This is a multi-part message in MIME format.
--------------egJdRmaVhjH2gk4jyDoim1xN
Content-Type: multipart/alternative;
 boundary="------------88BmI04frXCThbnk1baIR4WJ"

--------------88BmI04frXCThbnk1baIR4WJ
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
gzip: /home/$$/.Sent/cur/1664802400.M298227P1667779.<mydomain>,S=6509045:2,S: invalid compressed data--format violated

In general, I got out of the situation as follows.
One person helped write a script that:

  1. Runs through folders recursively, determines whether the file is compressed.
  2. If the file is compressed, decompress it and check if there are any errors. If there are no errors, skip.
  3. If the file was with an error, it unpacks the file into a temporary folder “as is”, after which it compresses it again and moves it back to the message folder.
    This helped to rid dovecot of errors and the mail client now working fine. It is possible that some of the messages are not readable, but this does not play a big role for me. In any case, it was not possible to do something else with them.
    Perhaps the next script will be useful to someone in the future.
#/bin/sh
MAIL_DIR='/home/user/mail/domain/'
BACKUP_DIR='/home/user/broken'
TMP_FILE='/tmp/dovecot_move_broken.tmp'

[ -e "$BACKUP_DIR" ] || mkdir -p "$BACKUP_DIR" || exit 1

find "$MAIL_DIR" -iname '[0-9]*.M*.*' -printf '%s:%p\n'| while IFS="" read -r p; do
    size=${p%%:*}
    file=${p#*:/home/}
    path=${file%/*}
    file=${file##*/}

    # пропускаем файлы размер которых равен указанному в флагах
    [ "${file%,S=$size,*}" != "$file" ] && continue

    # пропускаем файлы распакованные без ошибок
    gunzip -c "/home/$path/$file" 2>/dev/null >"$TMP_FILE" && rm "$TMP_FILE" && continue

    # пропускаем если файл не был сжат (пустой выходной файл)
    [ -s "$TMP_FILE" ] || continue

    echo "$path/$file"

    dir="$BACKUP_DIR/$path"
    [ -e "$dir" ] || mkdir -p "$dir"

    # копируем файл в бэкап, затем записываем в него заново сжатые данные
    cp -p "/home/$path/$file" "$BACKUP_DIR/$path/" && cat "$TMP_FILE" | gzip >"/home/$path/$file"

    rm "$TMP_FILE"
done
1 Like

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.