Using zstd instead of gzip for backups

Don’t you want to use zstd for backups? The resulting size may be similar (gzip), but it takes much less time.

p.s. hestia is nice cp :wink:

6 Likes

+1, this is a good suggestion. I saw proxmox lately switching to zstd as well.
gzip esp. with a high level of compression isn’t so effective and interesting anymore…

1 Like

Yep, looks impressive. (I’ve started using it in Proxmox.)

I agree very good compression protocol, it was designed by developers at Facebook. I would like this as well added much better option @Hexaris can you please open up a github feature request for this! Any yes I agree Hestia is a nice Control Panel.

Request created, for comparison:

  • gzip
    INFO: 100% (64.0 GiB of 64.0 GiB) in 37m 12s, read: 25.4 MiB/s, write: 25.2 MiB/s
    INFO: backup is sparse: 11.29 GiB (17%) total zero data
    INFO: transferred 64.00 GiB in 2232 seconds (29.4 MiB/s)
    INFO: archive file size: 48.80GB
    INFO: Backup job finished successfully
  • zstd
    INFO: 100% (64.0 GiB of 64.0 GiB) in 27m 41s, read: 21.9 MiB/s, write: 21.7 MiB/s
    INFO: backup is sparse: 11.29 GiB (17%) total zero data
    INFO: transferred 64.00 GiB in 1661 seconds (39.5 MiB/s)
    INFO: archive file size: 48.57GB
    INFO: Backup job finished successfully
2 Likes

I am to lazy to keep both updated :slight_smile:

2 Likes

A different data set - a small Proxmox VM
gzip
INFO: backup is sparse: 3.42 GiB (57%) total zero data
INFO: transferred 6.00 GiB in 197 seconds (31.2 MiB/s)
INFO: archive file size: 816MB
INFO: Finished Backup of VM 122 (00:03:18)
zst
INFO: backup is sparse: 3.42 GiB (57%) total zero data
INFO: transferred 6.00 GiB in 90 seconds (68.3 MiB/s)
INFO: archive file size: 793MB
INFO: Finished Backup of VM 122 (00:01:32)
INFO: Backup finished at 2020-09-16 22:45:47
INFO: Backup job finished successfully

It’d be interesting to compare zst with bz2 though - a well established and commonly installed gzip replacement/alternative.

That’s enough to put me off right away! :-o

Im wondering what the System Load Diff between the two

Think zstd is a clear winner…

3 Likes

Did some testing on my “main” server

Gzip “Level 9”

2020-09-18 05:30:03 Remote: sftp://****.your-storagebox.de//{user}.2020-09-18_05-30-03.tar
2020-09-18 05:30:05 Rotated sftp backup: 2020-09-09_05-29-21
2020-09-18 05:30:07 Uploading {user}.2020-09-18_05-30-03.tar
2020-09-18 05:41:09 Size: 23885 MB
2020-09-18 05:41:09 Runtime: 19 minutes

Zstd “Level 9”
2020-09-18 20:35:47 Remote:sftp://****.your-storagebox.de//{user}.2020-09-18_20-35-47.tar
2020-09-18 20:35:49 Rotated sftp backup: 2020-09-10_05-29-31
2020-09-18 20:35:50 Uploading {user}.2020-09-18_20-35-47.tar
2020-09-18 20:47:54 Size: 23980 MB
2020-09-18 20:47:54 Runtime: 6 minutes

New files have been added since midnight and mainly videos

It this is true then I will be very happy…

2 Likes
1 Like