Kubo blocks datastore defaults to sync:false — power loss can corrupt the repo and crash-loop the daemon

[Rinse12]

Incident

A production node lost power while kubo was writing its MFS root block. The flatfs block file was created but left 0 bytes (torn write). On every subsequent start, kubo failed to construct its node:

failed to initialize MFS root from <CID> stored at /local/filesroot:
proto: required field ipfs.boxo.ipld.unixfs.pb.Data.Type not set.
If corrupted, use 'ipfs files chroot' to reset (see --help)

…which made bitsocial daemon exit with status 1, so the service crash-looped indefinitely (systemd restart counter reached ~19k over 2 days) until manual intervention.

Root cause

Kubo's default ipfs init profile ships the blocks flatfs mount with "sync": false (kubo 0.42.0):

"Datastore": { "Spec": { "mounts": [
  { "mountpoint": "/blocks", "type": "flatfs", "sync": false, ... }
] } }

With sync: false, flatfs does not fsync block writes, so an unclean shutdown (power loss, hard reset, hypervisor kill) can leave zero-length or truncated block files. This time it hit the MFS root, which is recoverable with ipfs files chroot --confirm (nothing bitsocial-critical lives in MFS). But the same torn write can hit any block — e.g. community CIDs/blocks that are less trivially recoverable.

bitsocial-cli runs ipfs init and then mergeCliDefaultsIntoIpfsConfig() (src/ipfs/startIpfs.ts), which currently only adjusts Addresses, AutoTLS, and Gateway — it never touches Datastore.

Proposal

Set Datastore.Spec.mounts[/blocks].sync = true as a bitsocial-cli default:

• Apply it in mergeCliDefaultsIntoIpfsConfig() for fresh repos.
• Also ensure it on every daemon start for pre-existing repos, mirroring how ensureIpnsPubsubEnabled() handles Ipns.UsePubsub (single-key write, preserve everything else). Note sync is not part of datastore_spec identity, so changing it does not invalidate the existing repo.

Trade-off: fsync on every block write costs write throughput. Kubo disabled it by default for performance, but a long-running daemon that owns communities arguably should prioritize durability. Could also be made opt-out via a CLI flag if the perf hit matters for some deployments.

Recovery procedure (for reference, until fixed)

1. Stop the daemon
2. Remove the corrupt zero-length block file (path derivable from the CID's multihash, base32, sharded next-to-last/2)
3. IPFS_PATH=<repo> ipfs files chroot --confirm (kubo ≥0.42, resets MFS root to empty dir)
4. Start the daemon