feat(loops): filter and order data-row loops by a cell value - #348
Open
mostafasadeghidev wants to merge 3 commits into
Open
feat(loops): filter and order data-row loops by a cell value#348mostafasadeghidev wants to merge 3 commits into
mostafasadeghidev wants to merge 3 commits into
Conversation
A loop could pick a table and an order but not WHICH rows, so a section that should list the three featured articles listed the three newest ones. Migrated Webflow sites hit this immediately: their lists are curated by a boolean field the loop had no way to read. - New `@core/loops/cellFilter`: parse one condition out of the loop's filter bag and render it as SQL. Six operators (is / is not / checked / unchecked / has any value / empty), a closed set — never interpolated. - Both query paths apply it (post-type version join and data-kind direct read) and so do their COUNT queries, or pagination advertises rows the page query drops. - The canvas preview endpoint takes the same condition, so the editor shows what the published page will emit. - Properties panel renders a field picker from the selected table, and hides the value box for the operators that ignore it. Dialect notes, both learned the hard way and now pinned by tests: SQLite binds `?` by position in the TEXT, so the condition's parameters sit between tableId and limit/offset; and its json_extract returns INTEGER 1/0 for booleans, which never equals '1' across storage classes — hence the cast. The JSON read appears exactly once per fragment so the field name binds once, with coalesce folding in rows that lack the field. The field NAME binds as a parameter like the value, so no part of a filter reaches the statement text. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Ordering was limited to the row's own SQL columns, so a list could not be sorted by a real date, title, or rank that lives in cells_json. A migrated site could only approximate 'newest first' with import arrival order, which drifts the moment anything is re-imported. - `orderBy` now also accepts `cell:<fieldId>`; `parseCellOrder` reads it and `cellOrderSql` renders the expression with the field name bound as a parameter, so nothing reaches the SQL text. - Both query paths order by it, values compare as TEXT in both dialects (ISO dates sort chronologically; documented that numbers sort lexicographically — one predictable rule beats two engine-specific ones), and coalesce gives rows lacking the field a defined position. - The Loop panel lists the selected table's fields as order options. Riding on `orderBy` rather than a new prop means every caller that already threads it — publisher, canvas preview, imported data-order-by attributes — supports this without further plumbing. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
"has any value" described the same condition in more words. The operator id was already `isSet`; the label now says so too. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Lets a
base.loopon thedata.rowssource pick which rows it lists, and which cell it sorts by:is,is not,is checked,is unchecked,is set,is empty.<Label> (field), alongside the existing row columns.Both are configured in the loop's properties panel and take effect in the canvas preview, in published pages, and in the "load more" endpoint.
Why
A loop could pick a table, an order, and a limit — but not a subset. So a page meant to show the four team members flagged for the About page showed the four most recently updated rows instead, and a list meant to follow the source CMS's own publish date followed our
updated_atcolumn. The only workarounds were reordering rows by hand or duplicating a table, and both drift the moment someone edits an item.This is deliberately one condition rather than a query builder: it covers the real cases (a "featured" checkbox, a category, a "show on homepage" flag) without inventing an AND/OR grammar the editor can't express and future maintainers would have to keep sound in two dialects.
How
New pure module
src/core/loops/cellFilter.tsowns the whole contract — parsing, the closed operator set, the SQL fragments, and a TypeScript predicate that mirrors them for callers holding rows instead of a query.cells_json #>> array[$n], SQLitejson_extract(cells_json, '$.' || ?). A hostile field id cannot reach the statement (covered by a test).coalesce(…, '')folds the missing-field case into the comparison instead of needing a secondis nullbranch — repeating the expression would repeat its placeholder while the caller binds the field once.json_extractreturns INTEGER1/0for JSON booleans, and SQLite compares by storage class first, so1 = '1'is false without the cast. The checked/unchecked operators accept both spellings.orderByascell:<fieldId>rather than a new prop, so every caller that already threadsorderBy— publisher, canvas preview endpoint, importeddata-order-by— gets it for free. Values compare as text: ISO dates sort chronologically, which is the case this exists for.COUNTapplies the same filter on both the post-type and data-kind paths, so pagination never advertises rows the page query drops.Touched:
loops/cellFilter.ts(new),loops/sources/dataRows.ts,LoopPropertiesView.tsx,useLoopPreviewItems.ts,server/handlers/cms/data/tables.ts,core/persistence/cmsData.ts.User impact
Additive. A loop with no cell field configured behaves exactly as before, and a half-configured filter (field picked, operator not yet) keeps listing everything rather than silently emptying the list.
Verification
Two new files:
src/__tests__/loops/cellFilter.test.ts(parsing, parameter binding, dialect shapes, SQL↔TS parity) andsrc/__tests__/loops/dataRowsCellFilter.test.ts(behaviour against a real migrated SQLite database, both table kinds, filtered counts, pagination).Also exercised end-to-end on a real site migrated from another CMS: an About page list that needed 4 of 30 rows, and an index page that had to follow the source CMS's publish date rather than ours.
Note: the two
EBUSYfailures insrc/__tests__/loops/dataRowsFetch.test.tsare a pre-existing Windows temp-file teardown issue increateTestDb, unrelated to this change.