Borg incremental backups

Following on from a closed topic, I took a look at the referenced repository and did a simple search and replace to reflect the changes between VestaCP and HestiaCP. Following the instructions to step 4 (I skipped myduper) and everythign has been working well. I also performed 2 restores which worked too.
Note, I haven’t done testing beyond this however having read through the entire project I don’t believe issues are likely. For me this was enough to disable the built in backup by disabling the cron job in HestiaCP.

I prefer Borg backups because they allow me to backup more regularly, even hourly and restore any backup without the drive space expense of having 24 additional copies on the filesystem every day - Borg only saves the changes and also de-duplicates.

If this is interesting to anyone else the repo is below, I wish I knew how to integrate it into HestiaCP as an option. https://github.com/andyg2/hestiacp-borg-incremental-backups

2 Likes

I’d love this BORG integration, I think @falzo is big fan of Borg too so he can chime in with ideas.
I spent some time earlier trying to setup Borg and Restic but missed on a few logics and then did not really get time to look into it because of the sheer ‘manual’ work it all needs.

I do want to have some kind of reliable ‘incremental’ and ‘differential’ backup mechanism, it helps a lot on bigger sites and on NVMe Servers where storage is premium.

Feel free to connect with me personally, we can try to figure out a few things.

2 Likes

I have read the documentation thoroughly. I think that this feature should substitute Hestia’s backup system in the future and at least it should be supported.

Incremental backups are a must. For now, I have it sorted out with expensive wordpress plugins that make me invest a lot of time everytime I want to restore if the WordPress install is not operative.

I suggest you ask in GitHub for an Hestia integration.

1 Like

I am the original author of vesta-borg backups scripts. Let me know if i can do something to help. It would be great to see this integrated into hestia core.

4 Likes

As far as I can see - which isn’t very far by the way, I’ve only done a search and replace then fell off my chair when it all worked!!! - I thought the existing backup options can possibly be reused:

  • The compression (gzip, zstd), maybe add Borg here?
  • The quantity of backups, maybe Borg could interpret this as the time between full-backups? /usr/local/hestia/data/users/[USERNAME]/user.conf > BACKUPS=‘1’
  • The location/protocol, I’m thinking these could remain the same. These are here: /usr/local/hestia/conf/hestia.conf > BACKUP_MODE/BACKUP_SYSTEM

Unfortunately shell code is not something I’m good at though so my 2 pence worth is just a hopeful suggestion.

2 Likes

I think that if we want it integrated we need to decide:
Who will merge the scripts?
Who will maintain this Borg feature?
Will we create also an UI in the future?

And what does the development team think about this.

2 Likes

Before 1.5.0 Will never happen it is scheduled for next week after that we need to decide what our priorities are:

  • IpV6 support
  • New Web UI
  • Improvement regarding Backup
  • Improvement regarding active sync and so on…

It is a useful addition but time…

What I want to know @eris is if you can help @ramirojoaquin or someone else create a pull request with the minimum viable Borg implementation.

And if the development team is willing to support this addition or it has to be someone else.

I can’t promise anything…

You can’t promise that you will help. I get it. So we will have to create a pull request ourselves.

I was not asking for a promise. I was asking if the development team is willing to support this addition or if it would be unsupported and we have to support it ourselves.

You have already said that you have other priorities.

If we make an acceptable pull request will you maintain this feature, will you partially maintain this addition or you most likely not maintain it.

Did some testing as it is still interesting as it would save me backing up large video files every time…

Bugs I found so far:

  1. hestiacp-borg-incremental-backups/restore-web.sh at master · andyg2/hestiacp-borg-incremental-backups · GitHub
    Does check if /home/{user}/web/{domain} exists how ever if you delete the domain for what reason you have to create a new domain + restore it but then the original settings are still lost
  2. hestiacp-borg-incremental-backups/restore-mail.sh at 5e4f948acfe363cefb306824a200ba002517d2ee · andyg2/hestiacp-borg-incremental-backups · GitHub
    Has the same issue for mail also it is not possible to restore one account
  3. No option to restore DNS domain
  4. v-restore-user did not rebuild web and mail domains and dns domains and so on haven’t tested fully why
  5. To allow for use in Web UI at least Listing active backups should be possible
    borg list /backup/borg/home/user seem to be working how ever it would be nice to allow restoring a specific domain that is “active” in the backup but not on the Hestia server.
  6. All settings are currently more or less hardcode. We should probally add support to make it more flexible

Even the “Hestia” backup is far from perfect and I totally agree with it allows restoring only a domain or even only one mail account.

So is it useful? Yes, probally
Does it need a lot of work? Yes, for sure.

4 Likes

This will be nice

But I not understand why make entery backup system from 0

You just can integrate backup system to actually v-backup-user

Im working in similar idea with duplicity, but you can integrate same system with borg, rdiff and any orther script.

My idea: GitHub - Skamasle/Hestia-Vesta-Diff-Backup: Differential bakcup integration for Hestiacp and Vestacp

Just use v-backup-user, and leave hestia panel make backup of all, databases, users etc

So, why break your head, focus just in a problem, and problem big resource, big space and big time used to make backup in large sites, or large mail servers.

So leave hestiacp make a backup of hestia account, databases and just integrate a borg, duplicity or any orther to make diferencial backup of web files and emails.

In anyway you need make mysql databases each day, also mail passwords and any orther information ( information with very few mb and information than not need incremental backup ) and hestia panel just do it well

Just integrate diff backup inside v-backup-user and v-restore-user

1 Like

Some development:



and

What to do:

  • Update packages to allow user to enable backups
  • Server settings
  • Restore of files working via UI
  • Bulk restore files + Options in: “Selection what you want to restore”
  • Testing
  • More Testing
  • Fine tuning / tweaking

Please note Browsing files and opening snapshots might be a bit slow…

1 Like