New to Proxmox Backup Server (PBS) 4.0 is a ‘tech preview’ feature for using S3 buckets as a datastore. While many people associate S3 buckets with Amazon, it is more or less a standard that many cloud providers support. This opens up many possibilities for direct PBS to cloud backups, which is great news.
However, I cannot emphasize enough this new feature IS ‘tech preview’ and I’ve seen issues with it. Namely, backup job verifications often fail on bad chunks of data. So I would in NO WAY (as of PBS 4.0.14) rely on S3 backups as being reliable and bulletproof. However, I imagine the feature will mature and the bugs will be worked out over time.
As such, this blog post is more a FYI on how to configure S3 storage and is fun to do out of curiosity. But until PBS works out the bugs, I would not rely on it. My preferred S3 compatible cloud storage is Backblaze B2. Unlike Wasabi, there’s no minimum billing retention period. And unlike other providers, there’s no egress fees (subject to plan limits).Â
Backblaze B2 is priced at $6/TB/month which is VERY reasonable. You are billed only for the amount of storage you use. Buckets can also be deleted and the billing for that storage amount immediately stops (unlike Wasabi).Â
Create Backblaze S3 Bucket
- Login to your Backblaze account and under B2 Cloud Storage, click on Buckets.
- At the top of the screen click on Create a Bucket.
- Give the bucket a globally unique name. Leave the rest of the default values, unless you wish otherwise.
- Click Create a Bucket.
Note: Proxmox Backup Server appears to be VERY picky about bucket names that it considers acceptable. Backblaze lets me use upper case, lower case, numbers, and some special characters. However, PBS was rejecting some of them:
parameter verification errors (400)
backend: schema validation failed: Bucket name does not match the regex pattern
I suggest using all lowercase letters and digits. No upper case and no special characters.Â
5. In the left pane under B2 Cloud Storage, click Application Keys.
6. Click on Add a New Application Key.
7. Choose a name for the key, select your PBS bucket, then tick the box to allow listing of all buckets.
8. Click Create New Key.
8. Safely store the keyID and applicationKey. The applicationKey will NEVER be shown again.Â
9. Go back to your bucket list in Backblaze and find your new bucket. Look for the Endpoint FQDN and copy that to your clipboard.
Add Proxmox Backup Server S3 Endpoint
- Login to your Proxmox Backup Server (PBS) 4.0 (or later).
- On the left click on S3 Endpoints.
- Click on Add. Use the following parameters:
- S3 Endpoint ID: Anything name you wish
- Endpoint: Paste your BB S3 endpoint FQDN
- Port: Leave default
- Region: Leave blank
- Access Key: Paste your keyID.
- Secret Key: Paste your applicationKey
- Path Style: Check the box
- Provider Quirks: Select Skip If-None-Match header
4. Click Add.
Note: If you get an error such as the one below, double check your endpoint doesn’t have any leading or trailing spaces in the FQDN.
parameter verification errors (400)
endpoint: value does not match the regex pattern
Add Proxmox Backup Server S3 Datastore
- In the left pane under Datastore click Add Datastore.
- Use the following parameters:
- Name: Any datastore name you wish
- Datastore Type: S3 (tech preview)
- Local Cache: /mnt/backblazecache (any path you wish)
- S3 Endpoint ID: Your configured endpoint name
- GC Schedule: daily
- Prune Schedule: daily
- Bucket: Your Backblaze B2 bucket
3. Click on Prune Options.
4. Configure reasonable retention periods for your Backblaze B2 bucket. Keep costs in mind when considering how long to keep your data.
5. Click Add. Wait for the datastore to be successfully created.
Add Verify Job (Optional)
I strongly suggest creating a verify job to check the integrity of your S3 bucket. During the tech preview I’ve seen verify jobs failing due to bad chunks.
- Click on the Backblaze datastore in the left pane.
- Click on the Verify Jobs tab.
- Click Add.
- Use all the defaults but I suggest changing the schedule to run a few hours after your backups are expected to complete. This will use internet bandwidth, so be mindful. I chose 06:00.
- Click Add.
6. Your Proxmox Backup Server datastore is now ready to use. Jump over to your first Proxmox VE server, add a Proxmox Backup Server datastore, and point it to your new Backblaze datastore on your PBS. The configure backup jobs as you see fit.
For more details on these steps, check out the second half of my article: How To: Proxmox Backup Server 4 (VM) Installation and start with the section “Add PBS Datastore to Proxmox VE“.
Summary
Brand new to Proxmox Backup Server 4.0 is the ability to use S3 compatible storage buckets. In this blog post I walk you through how to configure a Backblaze B2 bucket for use with Proxmox Backup Server 4.0. We then mount it as a backup datastore to Proxmox VE, and schedule backup jobs against it.Â
This feature is in ‘tech preview’ and I’ve seen backup verification jobs fail due to bad chunks. So I would NOT rely on this as a backup destination until it comes out of ‘tech preview’ and the bugs are resolved.
