S3 versioning is not a backup: how to actually back up a bucket

Ask an engineer how their S3 data is protected and you'll usually get one of three answers: versioning is on, replication is configured, or "it's S3, it's eleven nines."

All three are true statements. None of them is a backup.

Eleven nines of durability is a promise about disks failing. It is not a promise about your credentials leaking, an intern running aws s3 rm --recursive, a Terraform apply rewriting a lifecycle rule, or AWS closing an account over an unpaid invoice. Durability protects your data from hardware. Nothing in that number protects your data from you, or from anyone holding your keys.

Here's the test that cuts through it: if someone gained admin access to your AWS account for ten minutes, what would still exist afterward? Anything living inside that account is in the blast radius. That's the whole argument.

What each AWS feature actually protects against

Object Lock in compliance mode is the one genuinely strong control here, and it's worth enabling on buckets that hold anything you'd cry about. It requires versioning, and it's far easier to set up at bucket creation than to retrofit. But note the right-hand column: it protects objects, not the account they sit in.

Three specific ways versioned buckets lose data anyway:

• aws s3 rm writes a delete marker. The old version survives — until a lifecycle rule expires noncurrent versions. Most teams add exactly that rule to control cost, and it quietly becomes a data-destruction schedule.
• Delete markers are not replicated by default. So the "backup" bucket in another region diverges from the source in the one direction you'd want it to follow, and teams discover this while trying to recover.
• Version deletes are permanent. delete-object --version-id skips the marker entirely. Anyone with s3:DeleteObjectVersion can erase history.

The fix: one copy that isn't in AWS

The 3-2-1 rule predates the cloud but survives it intact: three copies, two media, one off-site. In cloud terms, "off-site" doesn't mean another region. It means outside the account, ideally outside the provider, reachable with credentials that AWS doesn't issue.

The pipeline is four steps and takes about ten minutes.

1. Mirror the bucket locally

mkdir -p ./s3-mirror

aws s3 sync s3://your-bucket ./s3-mirror \
  --delete \
  --exact-timestamps

The --delete flag deserves a paragraph, because it looks like exactly the wrong choice. It deletes local files that no longer exist in the bucket — which sounds like propagating the disaster straight into your backup.

It isn't, and the reason matters. Each upload in the next step creates an immutable point-in-time snapshot. Yesterday's snapshot still contains every file, whatever today's mirror looks like. So the mirror should track the bucket exactly, and history lives in the snapshots rather than in an ever-growing pile of local files. Without --delete, your mirror only grows, you pay to store deleted objects forever, and you can never answer "what did the bucket look like on the 3rd?"

--exact-timestamps avoids re-downloading files whose size matches but whose timestamps drifted.

Two things to know before the first run:

• Data transfer out of S3 is billed per GB. The first sync pulls the whole bucket; after that you only pay for changes. Check current AWS rates before syncing a large bucket, and consider running the job on an EC2 instance in the same region if egress is the concern.
• Sync from a read-only IAM user. The job needs s3:GetObject and s3:ListBucket, nothing more. A backup process with delete permissions is a backup process that can participate in the incident.

2. Ship it somewhere AWS can't reach

A mirror on one machine is a second copy, not an off-site one. Push it to storage with separate credentials, so a compromised AWS account doesn't reach it.

Sign in at Backup Data, claim the free 5 GB workspace, mint an API key scoped backup:write, backup:read, snapshots:read, then:

npm install @lighthouse-web3/baas-js-sdk
export LH_API_KEY="lh_xxxxxxxxxxxxxxxxxxxxxxxx"
export LH_WORKSPACE_ID="your-workspace-uuid"

import { BackupClient } from "@lighthouse-web3/baas-js-sdk";

const client = new BackupClient({
  apiKey: process.env.LH_API_KEY,
  workspaceId: process.env.LH_WORKSPACE_ID,
});

const snapshot = await client.backup(["./s3-mirror"], {
  description: "nightly s3 mirror",
  tags: { type: "s3", bucket: "your-bucket", env: "prod" },
});

console.log(`snapshotId=${snapshot.snapshotId} totalSize=${snapshot.totalSize}`);

(A Go SDK offers the same surface.)

Uploads are content-addressed and chunked with FastCDC, so the second night doesn't re-upload the bucket — only the objects that changed, and only the changed regions within them. A 40 GB bucket with 2% daily churn ships roughly 800 MB a night, not 40 GB.

If the bucket holds anything sensitive, encrypt client-side. SSE-S3 and SSE-KMS protect data at rest in AWS, using keys AWS holds. Once the objects leave the bucket, that protection ends:

import { generateKeyfile } from "@lighthouse-web3/baas-js-sdk";

// once:
generateKeyfile("/secure/lh.keyfile", process.env.LH_KEYFILE_PASSPHRASE);

// then on every backup:
await client.backup(["./s3-mirror"], {
  description: "nightly-encrypted",
  encryption: {
    keyfilePath: "/secure/lh.keyfile",
    passphrase: process.env.LH_KEYFILE_PASSPHRASE,
  },
});

Data is AES-GCM-encrypted on your machine; the server only ever stores ciphertext. You hold the keys. Lose the keyfile and passphrase and the data is unrecoverable — there is no server-side reset. Store it like a root credential, and specifically not in the AWS account you're backing up away from.

3. Retention, so storage stays bounded

const before = new Date(Date.now() - 30 * 24 * 60 * 60 * 1000).toISOString();

// sourceId identifies the machine and path a backup came from; every snapshot
// carries it. Retention is currently applied across the whole workspace, so
// keepLatest counts every snapshot in it, not just this source's.
const policy = { sourceId: snapshot.sourceId, keepLatest: 14, before };

// Preview first:
const preview = await client.pruneSnapshots({ ...policy, dryRun: true });
console.log("would delete:", preview.snapshotIds);

await client.pruneSnapshots({ ...policy, dryRun: false });

A snapshot goes only when it's both outside the newest 14 and older than 30 days. Chunks still referenced by surviving snapshots are kept automatically.

4. Prove it restores

This is the step that separates a backup from a folder of files. Restore into a scratch directory and compare it against the live bucket:

import { mkdtempSync } from "node:fs";
import { tmpdir } from "node:os";
import { join } from "node:path";

const target = mkdtempSync(join(tmpdir(), "s3-restore-drill-"));
await client.restore(snapshot.snapshotId, target, {
  onProgress: (e) => console.log(`[${e.phase}] ${e.current}/${e.total}`),
});

# Should print nothing if the restore matches the bucket:
aws s3 sync "$TARGET/s3-mirror" s3://your-bucket --dryrun --delete

Every restore verifies chunk integrity against expected checksums before reassembly, so a silently corrupted backup fails loudly here rather than during an incident.

Then time a real recovery. Pushing a restored mirror back into a bucket is:

aws s3 sync ./restored-mirror s3://your-bucket-recovered

Whatever the stopwatch says is your actual RTO. Write it down. If it's four hours and your team believes "we'd be back in twenty minutes," that gap is the most valuable thing this exercise produces.

The drill checklist:

• ✅ Restore into a fresh, empty directory — never over the live mirror
• ✅ Leave checksum verification on
• ✅ Diff against the live bucket with --dryrun
• ✅ Spot-check a few objects byte-for-byte, not just filenames
• ✅ Record wall-clock restore time, keep a log across drills

5. Schedule it

# /etc/cron.d/s3-backup (nightly at 02:30)
30 2 * * * backup /opt/backup/s3-backup.sh >> /var/log/s3-backup.log 2>&1

Where s3-backup.sh is the aws s3 sync from step 1 followed by node upload.mjs. Add the restore drill monthly — step 4 is not a one-time ceremony.

What you end up with

Keep the AWS controls. They're good at what they do. Just stop counting them as the backup, because every one of them shares a fate with the account.

Start with the free tier: backupdata.io gives you 5 GB free with no card, and the 10-minute quickstart gets you to a first verified snapshot today.