Skip to content

feat(loops): filter and order data-row loops by a cell value - #348

Open
mostafasadeghidev wants to merge 3 commits into
CoreBunch:mainfrom
mostafasadeghidev:feat/loop-cell-filter
Open

feat(loops): filter and order data-row loops by a cell value#348
mostafasadeghidev wants to merge 3 commits into
CoreBunch:mainfrom
mostafasadeghidev:feat/loop-cell-filter

Conversation

@mostafasadeghidev

@mostafasadeghidev mostafasadeghidev commented Aug 6, 2026

Copy link
Copy Markdown

What

Lets a base.loop on the data.rows source pick which rows it lists, and which cell it sorts by:

  • Filter — one condition on a row's own cell: is, is not, is checked, is unchecked, is set, is empty.
  • Order by a field — the table's own fields join the Order by list as <Label> (field), alongside the existing row columns.

Both are configured in the loop's properties panel and take effect in the canvas preview, in published pages, and in the "load more" endpoint.

Why

A loop could pick a table, an order, and a limit — but not a subset. So a page meant to show the four team members flagged for the About page showed the four most recently updated rows instead, and a list meant to follow the source CMS's own publish date followed our updated_at column. The only workarounds were reordering rows by hand or duplicating a table, and both drift the moment someone edits an item.

This is deliberately one condition rather than a query builder: it covers the real cases (a "featured" checkbox, a category, a "show on homepage" flag) without inventing an AND/OR grammar the editor can't express and future maintainers would have to keep sound in two dialects.

How

New pure module src/core/loops/cellFilter.ts owns the whole contract — parsing, the closed operator set, the SQL fragments, and a TypeScript predicate that mirrors them for callers holding rows instead of a query.

  • The field name is a bound parameter, never SQL text. Postgres reads cells_json #>> array[$n], SQLite json_extract(cells_json, '$.' || ?). A hostile field id cannot reach the statement (covered by a test).
  • The JSON read appears exactly once per fragment. coalesce(…, '') folds the missing-field case into the comparison instead of needing a second is null branch — repeating the expression would repeat its placeholder while the caller binds the field once.
  • SQLite casts the read to text. json_extract returns INTEGER 1/0 for JSON booleans, and SQLite compares by storage class first, so 1 = '1' is false without the cast. The checked/unchecked operators accept both spellings.
  • Ordering rides on orderBy as cell:<fieldId> rather than a new prop, so every caller that already threads orderBy — publisher, canvas preview endpoint, imported data-order-by — gets it for free. Values compare as text: ISO dates sort chronologically, which is the case this exists for.
  • COUNT applies the same filter on both the post-type and data-kind paths, so pagination never advertises rows the page query drops.

Touched: loops/cellFilter.ts (new), loops/sources/dataRows.ts, LoopPropertiesView.tsx, useLoopPreviewItems.ts, server/handlers/cms/data/tables.ts, core/persistence/cmsData.ts.

User impact

Additive. A loop with no cell field configured behaves exactly as before, and a half-configured filter (field picked, operator not yet) keeps listing everything rather than silently emptying the list.

Verification

bun test src/__tests__/loops/     # 75 pass
bun run build
bun run lint

Two new files: src/__tests__/loops/cellFilter.test.ts (parsing, parameter binding, dialect shapes, SQL↔TS parity) and src/__tests__/loops/dataRowsCellFilter.test.ts (behaviour against a real migrated SQLite database, both table kinds, filtered counts, pagination).

Also exercised end-to-end on a real site migrated from another CMS: an About page list that needed 4 of 30 rows, and an index page that had to follow the source CMS's publish date rather than ours.

Note: the two EBUSY failures in src/__tests__/loops/dataRowsFetch.test.ts are a pre-existing Windows temp-file teardown issue in createTestDb, unrelated to this change.

mostafasadeghidev and others added 2 commits August 6, 2026 10:19
A loop could pick a table and an order but not WHICH rows, so a section
that should list the three featured articles listed the three newest
ones. Migrated Webflow sites hit this immediately: their lists are
curated by a boolean field the loop had no way to read.

- New `@core/loops/cellFilter`: parse one condition out of the loop's
  filter bag and render it as SQL. Six operators (is / is not / checked /
  unchecked / has any value / empty), a closed set — never interpolated.
- Both query paths apply it (post-type version join and data-kind direct
  read) and so do their COUNT queries, or pagination advertises rows the
  page query drops.
- The canvas preview endpoint takes the same condition, so the editor
  shows what the published page will emit.
- Properties panel renders a field picker from the selected table, and
  hides the value box for the operators that ignore it.

Dialect notes, both learned the hard way and now pinned by tests:
SQLite binds `?` by position in the TEXT, so the condition's parameters
sit between tableId and limit/offset; and its json_extract returns
INTEGER 1/0 for booleans, which never equals '1' across storage classes
— hence the cast. The JSON read appears exactly once per fragment so the
field name binds once, with coalesce folding in rows that lack the field.

The field NAME binds as a parameter like the value, so no part of a
filter reaches the statement text.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Ordering was limited to the row's own SQL columns, so a list could not be
sorted by a real date, title, or rank that lives in cells_json. A migrated
site could only approximate 'newest first' with import arrival order,
which drifts the moment anything is re-imported.

- `orderBy` now also accepts `cell:<fieldId>`; `parseCellOrder` reads it
  and `cellOrderSql` renders the expression with the field name bound as
  a parameter, so nothing reaches the SQL text.
- Both query paths order by it, values compare as TEXT in both dialects
  (ISO dates sort chronologically; documented that numbers sort
  lexicographically — one predictable rule beats two engine-specific
  ones), and coalesce gives rows lacking the field a defined position.
- The Loop panel lists the selected table's fields as order options.

Riding on `orderBy` rather than a new prop means every caller that already
threads it — publisher, canvas preview, imported data-order-by attributes
— supports this without further plumbing.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@mostafasadeghidev mostafasadeghidev changed the title feat(loops): filter data-row loops by a cell value feat(loops): filter and order data-row loops by a cell value Aug 6, 2026
"has any value" described the same condition in more words. The
operator id was already `isSet`; the label now says so too.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant