Cloudflare R2 for Backups

yes, it is

access_key_id and secret_access_key are my cloudflare login details right?

You need to enter that in Rclone yes…

Hestia should not need that “info”

is there a way to shell test rclone to make sure it can reach r2 server etc and auth correctly?

Create a test file

rclone copy ./test.txt diurnal:diurnal 

2022/09/29 21:04:06 ERROR : Attempt 1/3 failed with 1 errors and: BadRequest: Bad Request
status code: 400, request id: , host id:

rclone.conf:

[diurnal]
type = s3
provider = Cloudflare
access_key_id = email_address
secret_access_key = my_password
endpoint = https://aplha-numeric.r2.cloudflarestorage.com/diurnal
acl = private
region = auto
bucket_acl = private
upload_cutoff = 5Gi

I think you need to create seperate tokens

okay that fixed it, successfully upload file now. thought it may be wrong credentials but wrangler itself used login details though.

5gb cutoff is gonna be messy with mail accounts!

that is:
upload_cutoff = 5Gi

is required in rclone.conf (in rclone config / advanced ) otherwise the upload file size limit is a lot less, and if i try give it a higher value than this it fails.

just had a look at my main email account and its way over this max limit and i guess its gonna catch out people with rich content too.

definitely needs some /backup/ file segmentation if the main file is above 5GB before trying to upload.

Unless we able to implement some kind of chunking and limit the file size I don’t think there is a easy method to fix it.

Archive the archive into part1 part2 etc suffixed ofcourse?! easy to do with RAR

Hey @eris

I see rclone support is coming on 1.7. I am already using rclone as I sent my backups to backblaze and bunny every night with a cronjob.

Would I be affected by the HestiaCP 1.7 update? Would I have to uninstall rclone first then update to 1.7?

Just trying to prepare I know you don’t have an ETA for it yet.

If you keep using a “other” cronjob your setup will not effected…

i havent touched anything but days later the backups went from rclone back to sftp and failed

Did the server upgrade to a new version?

It seems to be a bug with rclone make sure to update to the last version.

I am currently running some tests and it looks to be working just fine

Did not work unless I installed a beta version

rclone-v1.58.0-beta.5838.074234119-linux-amd64.zip

Even then I get:
2022/10/10 12:18:13 ERROR : jaap.2022-10-10_12-12-59.tar: Failed to copy: multipart upload failed to upload part: SignatureDoesNotMatch: The request signature we calculated does not match the signature you provided. Check your secret access key and signing method.
status code: 403, request id: , host id:
2022/10/10 12:18:13 ERROR : Attempt 1/3 failed with 1 errors and: multipart upload failed to upload part: SignatureDoesNotMatch: The request signature we calculated does not match the signature you provided. Check your secret access key and signing method.
status code: 403, request id: , host id:
2022/10/10 12:19:07 INFO :

It might be an other branch where It fixed in…

2022/10/10 12:28:07 INFO : jaap.2022-10-10_12-12-59.tar: Copied (new)
2022/10/10 12:28:07 ERROR : Attempt 2/3 succeeded
2022/10/10 12:28:07 INFO :
Transferred: 25.068 GiB / 25.068 GiB, 100%, 31.836 MiB/s, ETA 0s
Transferred: 1 / 1, 100%
Elapsed time: 15m0.5s

2022-10-10 12:28:15 Local: /backup/jaap.2022-10-10_12-12-59.tar
2022-10-10 12:28:16 Size: 6145 MB
2022-10-10 12:28:16 Runtime: 1 minute

It is quite random. But if it is with a back up of larger then 6GB we should notify the user maybe warn the user maybe. Or see if it happens more often…