bug(cloud): sync push transmits observation content in cleartext — edge WAFs block memories containing SQL identifiers (e.g. information_schema)

[smithst18]

Summary

Cloud sync transmits observation content in cleartext inside the HTTP push
body. When a memory legitimately contains a common SQL identifier (e.g.
information_schema, UNION SELECT, pg_*), a managed edge WAF in front of
engram cloud serve (OWASP CRS SQL-injection ruleset, as used by Cloudflare /
Render / most PaaS edges) classifies the request as an injection attempt and
returns an edge 403 "Blocked" HTML page before the request ever reaches
the Engram container.

Because a chunk is deterministic (same content → same chunk id), the block is
permanent: every retry rebuilds the identical body and is rejected again.
The whole project's cloud sync is wedged by a single technical note.

Why it's the transport, not the app

The 403 body is the edge provider's branded HTML block page, not Engram's
JSON (engram cloud serve returns JSON such as forbidden: project is not allowed / 401). So auth, the token, and ENGRAM_CLOUD_ALLOWED_PROJECTS are
all fine — the request is killed at the edge, upstream of the app, purely on the
content of the request body.

Observed against the official image ghcr.io/gentleman-programming/engram:latest
deployed on a managed PaaS (Render) whose edge runs a managed WAF.

Repro (synthetic)

1. Deploy engram cloud serve behind any edge with the OWASP CRS SQLi ruleset
   enabled (Cloudflare "Managed Rules", Render, etc.).
2. On a client, save an observation whose content contains a normal technical
   reference to a SQL catalog, e.g.:

   "the post-migration guard queries information_schema to check the column
   exists before the UPDATE"

3. Enroll the project and engram sync --cloud --project <p>.

Expected: the memory syncs.
Actual: `engram: write chunk: cloud: push chunk : status 403:

… <title>Blocked</title> …` (the edge's HTML block page), and it

recurs on every retry because the chunk is content-identical.

Note: a body containing the token only once may pass, while several occurrences
in one chunk cross the WAF's anomaly-score threshold and block — so it surfaces
intermittently and looks project-specific, but the root cause is the cleartext
transport.

Impact

• Any project whose memories discuss databases, migrations, ORM internals, or
  security work (i.e. exactly the technical notes Engram is meant to store) can
  become permanently un-syncable to a WAF-fronted cloud.
• No client-side recovery exists through the supported CLI: the poisoned
  mutations sit in the outbound queue (sync_mutations) and the chunk rebuilds
  identically; engram cloud upgrade repair doesn't touch content, and there's
  no supported "rewrite/rebuild outbound queue" path. The only unblock is
  editing the stored content to avoid the keyword — degrading real data to dodge
  a firewall.

Proposed fix

Transport chunk payloads as an opaque body so no memory content is legible
to content-inspection middleboxes:

• gzip + base64 the chunk (or send binary with Content-Encoding: gzip), and/or
• send the chunk as an opaque octet-stream rather than a JSON object whose string
  fields carry raw memory text.

This removes the entire class of edge-WAF false positives (SQL keywords, path
traversal strings, <script> in notes about XSS, etc.) permanently and for every
deployment, without asking operators to tune WAF rules they often can't reach
(e.g. managed PaaS free tiers expose no WAF config).

Environment

• Engram: official image ghcr.io/gentleman-programming/engram:latest; CLI
  client 1.20.0.
• Edge: managed PaaS with a managed WAF in front of the container.
• OS/arch: Linux x86_64 client.