Add incremental backup support

[dudanogueira]

Exposes Weaviate's file-based incremental backups (server v1.37.0+) in the client:

const base = await client.backup.create({
  backupId: 'base-backup',
  backend: 'filesystem',
  waitForCompletion: true,
});

await client.backup.create({
  backupId: 'incremental-backup',
  backend: 'filesystem',
  incrementalBaseBackupId: base.id, // or any existing backup ID string
  waitForCompletion: true,
});

Unchanged files are not copied — they are restored from the base. Deleting a base backup therefore breaks every incremental built on top of it.

Changes

• incrementalBaseBackupId on client.backup.create() and collection.backup.create(), plus .withIncrementalBaseBackupId() on the v2 builder.
• Gated client-side by DbVersionSupport.supportsIncrementalBackups() — throws WeaviateUnsupportedFeatureError below 1.37.0, matching the Python client. The gate applies to the v2 builder too: Weaviate below 1.37.0 silently ignores the field and writes a full backup, which is worse than an error.
• Validates that the base ID differs from the backup being created (the server otherwise returns a bare 500). The comparison is case-insensitive, and the ID is lowercased before it goes on the wire, because Weaviate treats backup IDs as case-insensitive.
• The base backup ID is read back on getCreateStatus() and list().
• parseStatus() now also surfaces size, which was declared on BackupStatusReturn but never populated.

When incrementalBaseBackupId comes back as undefined

This is a read-back with a narrower floor than the feature itself, so it is undefined in more cases than you might expect:

• the backup is not incremental;
• the caller is not an RBAC root user (Weaviate gates the field on isRequestFromRootUser);
• the server is older than v1.37.6 — the create-side field landed much earlier, but the status/list read-back was added in 7a53058, whose earliest tag is v1.37.6;
• the call is create() without waitForCompletion, since BackupCreateResponse carries no such field at all.

All four are documented on the type.

Notes for reviewers

• backup(connection) now takes dbVersionSupport as a second argument — this applies to the v3 factory, backupCollection, and the v2 factory in src/backup/index.ts. All three are internal; only types are re-exported publicly. BackupCreator's new constructor argument is optional, so the public class stays source-compatible.
• list() no longer returns the raw payload verbatim: incremental_base_backup_id is mapped to incrementalBaseBackupId. Everything else is passed through unchanged.
• ci/docker-compose-rbac.yml gains backup-filesystem. It is the only CI instance with a root user (AUTHORIZATION_ADMIN_USERS populates the RBAC root-user list), and ci/docker-compose-backup.yml is anonymous — so the root-only read-back cannot be asserted anywhere else.

Testing

Mock tests cover the request payload, lowercasing, case-insensitive self-reference rejection, the collection-scoped path, the v2 builder, the version gate (1.36 vs 1.37) and response parsing including size.

Integration tests cover two instances:

• the anonymous backup instance, which creates an incremental on top of a base, restores it, and asserts incrementalBaseBackupId is undefined — the honest contract for a non-root caller;
• the RBAC instance, gated on 1.37.6, which asserts the base backup ID round-trips through create, getCreateStatus and list as a root user.

Verified against Weaviate 1.38.0 and 1.38.2.

🤖 Generated with Claude Code

https://claude.ai/code/session_01Gc9nJTnKeYs41fidZAYWf6

[orca-security-eu]

Orca Security Scan Summary

Status	Check	Issues by priority
Passed	Infrastructure as Code	0   0   0   0	View in Orca
Passed	SAST	0   0   0   0	View in Orca
Passed	Secrets	0   0   0   0	View in Orca
Passed	Vulnerabilities	0   0   0   0	View in Orca

[g-despot]

Tested this locally against 1.35.0, 1.36.10 and 1.38.9.

The feature works. I confirmed the incremental genuinely skips base segments on disk (1.7 MB vs 35 MB uncompressed, with the objects and property_text segments absent), and that restoring the incremental fails without the base present and succeeds with it.

Two things I'd like sorted before merge:

1. The 1.37.0 floor is too high. 1.36.10 accepts incremental_base_backup_id and produces a real incremental (345 KB vs 7.5 MB base, baseBackupId persisted in backup_config.json), but the client throws WeaviateUnsupportedFeatureError against it. A tag scan puts create at 1.35.13 and 1.36.3. Below that I checked 1.35.0, where the server silently ignores the field and you get a full backup, so the gate is worth keeping, just lower.

2. The read-back is a separate feature with its own floor. incremental_base_backup_id only comes back on status/list from ~1.37.6, and only for root users. On 1.37.0 to 1.37.5 we'd be promising a field the server never sends.

Rest inline. Nice work otherwise, the design is clean.

[g-despot]

1.37.0 is too high. I ran 1.36.10 and the server accepted the field and produced a genuine incremental. Tag scan says create landed in 1.35.13 and 1.36.3. supportsSingleGrouped just above has the pattern for backported ranges.

Worth splitting this into two checks, since the read-back has a different floor (~1.37.6).

[g-despot]

Actually maybe we can ignore this one, as I see that Python also uses 1.37.0

[dudanogueira]

Taking your follow-up — keeping the floor at 1.37.0 so we stay in step with Python (backup/executor.py gates on is_lower_than(1, 37, 0)).

Your tag scan checks out on my side too: the incremental-create merge bb6b4be first appears in v1.34.18 / v1.35.13 / v1.36.3. So the lower floor is real, it just feels like a coordinated change across clients rather than TS diverging on its own. Happy to open a follow-up issue for that if you want it tracked.

[g-despot]

This path skips both the version gate and the lowercasing that client.ts does, and validate() at :95 compares raw, so backupId: 'B1' with base 'b1' gets through. weaviateV2 is public API.

Lowercasing here would fix the validate case for free.

[dudanogueira]

Fixed both halves.

withIncrementalBaseBackupId now lowercases, and validateIncrementalBaseBackupId compares case-insensitively. That second part matters for the exact case you named: lowercasing the base alone would not catch backupId: 'B1' + base 'b1', because the primary ID keeps its casing on the v2 path.

I also added the version gate rather than leaving it to v3. BackupCreator takes an optional DbVersionSupport and do() checks it before posting; weaviateV2.client() passes it through. Worth having given you confirmed 1.35.0 silently produces a full backup — a silent full backup is exactly the failure the gate exists to prevent. Mock coverage under with the v2 builder and should throw from the v2 builder too.

The v3 path now delegates the lowercasing to the builder instead of doing its own.

[g-despot]

While you're in this object: size is declared on BackupStatusReturn but never set here, so it's always undefined. The server does return it and Python surfaces it. One line?

[dudanogueira]

Done — size: 'size' in res ? res.size : undefined. The in guard is needed because parseStatus also handles restore responses, which have no size field. Mock asserts it now.

[g-despot]

It's also undefined for non-root users, on servers below ~1.37.6, and always when waitForCompletion is false, since BackupCreateResponse doesn't carry the field at all. Worth spelling out. This is exactly the field where a wrong comment sends someone hunting the wrong bug.

[dudanogueira]

Rewritten to spell out all four cases: not incremental, non-root caller, server older than 1.37.6, and the return of create() without waitForCompletion.

Confirmed the 1.37.6 number from the server repo rather than guessing — 7a53058 ("Add root only baseBackupId return") first appears in tag v1.37.6. Good catch, thanks.

[g-despot]

Can we get this down to one line? The rest of the file is one-liners and this block is repeated near-verbatim in collection.ts and backupCreator.ts.

Worth stealing the sentence from the Python docstring though: unchanged files are restored from the base, so deleting a base backup breaks every incremental built on it. That's the part people actually need.

[dudanogueira]

One line now, and I stole the Python sentence — unchanged files are restored from the base, so deleting a base backup breaks every incremental built on it.

Also removed the duplicates you were pointing at: BackupCollectionCreateArgs is now BackupCollectionArgs & Pick<BackupCreateArgs, 'incrementalBaseBackupId'>, and the two interface JSDoc blocks in client.ts / collection.ts are one line each.

[g-despot]

This never runs. The server only returns incremental_base_backup_id to RBAC root users, and docker-compose-backup.yml is anonymous, so the guard is always false. (Also tests-with-auth is skipped for fork PRs.)

I did verify it works with a root user on 1.38.9, all three surfaces round-trip correctly. Either wire up a root case or drop the if so the coverage is honest.

[dudanogueira]

Did both, since neither alone is honest on its own.

The anonymous suite now asserts the field is undefined — that is the real contract there, and it would have caught a regression that started leaking the ID to non-root callers.

For the root case: AUTHORIZATION_ADMIN_USERS populates the RBAC root-user list server-side (usecases/config/environment.go falls back to it when AUTHORIZATION_RBAC_ROOT_USERS is unset), so admin-user on ci/docker-compose-rbac.yml is already a root caller. I added backup-filesystem + BACKUP_FILESYSTEM_PATH to that compose and a new requireAtLeast(1, 37, 6) suite on 8091 with ApiKey('admin-key') that asserts the round-trip across create, getCreateStatus and list.

compose_up_all brings that file up in every job and it needs no secrets, so it runs on fork PRs too — it just will not execute anywhere below 1.37.6, and CI's WEAVIATE_137 is 1.37.5, so in practice only the 1.38 legs exercise it.

Verified locally against 1.38.2: all three surfaces round-trip.

[dudanogueira]

Pushed the review fixes in f493925. Summary of what changed per thread is inline; the PR description is rewritten too.

CI

Two legs are red, and I want to be upfront that I looked into whether I caused them. I do not believe I did:

• tests-without-auth (22.x, 1.38) — collection.query > boost parameter > 'timeDecay', an ordering assertion. Unrelated to backups.
• tests-without-auth (24.x, 1.37) — two test/users/integration.test.ts role assign/revoke 404s, plus a deleteMany "Stringified UUID is invalid".

The two users failures are a known flake: the identical pair failed on main on 2026-08-13 (run 31690118238, on the 1.34 leg), with no backup code in play.

The one leg that actually exercises the new root-user suite is tests-without-auth (24.x, 1.38), because requireAtLeast(1, 37, 6) skips it everywhere else and CI's WEAVIATE_137 is 1.37.5. That leg passed.

The only way my change could plausibly touch the users/roles suites is the new backup-filesystem module on ci/docker-compose-rbac.yml, so I ran both suites locally against that exact modified compose on 1.38.2: 39/39 pass.

I do not have rerun rights on this repo — if someone with admin could kick the two failed jobs, that should close it out.