Carry Subject schema, label and Main Subject flag on the subject-page index - #1335
Draft
alistair3149 wants to merge 4 commits into
Draft
Carry Subject schema, label and Main Subject flag on the subject-page index#1335alistair3149 wants to merge 4 commits into
alistair3149 wants to merge 4 commits into
Conversation
… index Naming a Subject needs three fields the subject slot alone holds: its Schema, its label, and whether it is its page's Main Subject. A surface listing Subjects therefore reads and parses a hosting page per Subject, and gets every Subject on that page whether it wanted them or not. On a depth-3 relation walk over the demo wiki those reads are 78% of the marginal cost. The index writer already parses the slot JSON, so it records the three fields alongside the mapping it was writing anyway. They are read raw, like the ids, so a Subject too broken to deserialize is still indexed under the name it claims. Existing wikis take the columns through a schema patch, and the backfill runs again on a new update key. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017cqUCA4KUn2Y1VumDkQyqm
update.php runs its post-update scripts outside doUpdates(), so the backfill runs even when --noschema or --schema skipped the schema change. Asking whether the table exists was the right question while the table was the only schema object here: a skipped CREATE TABLE leaves nothing to write to. A skipped ALTER leaves the table in place without its columns, and the backfill then dies on the first page with "Unknown column 'nwsp_schema' in 'SELECT'". Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017cqUCA4KUn2Y1VumDkQyqm
The backfill is the reason the update key was bumped, and nothing asserted it wrote anything but the mapping. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017cqUCA4KUn2Y1VumDkQyqm
SubjectLabel::fromText treats whitespace as absence and SchemaName refuses it, so recording " " as a label contradicted both, and contradicted the column's own comment promising null for a Subject that names none. Also drops the raw subject-id reader, which the header reader left without a caller. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017cqUCA4KUn2Y1VumDkQyqm
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #1333
neowiki_subject_pagegains three columns —nwsp_schemaVARBINARY(255) NULL,nwsp_labelBLOB NULL andnwsp_is_mainTINYINT NOT NULL — read from the raw slot JSON the index writer already parses, at the callsite that already writes the table. No new index: they ride a row the caller already looks up by key. Naming
a Subject takes exactly these three plus the page name, which reads already join.
A prerequisite, not a win on its own: nothing reads the columns yet, and
#1334 is filed but unassigned and unscheduled. Nine of the
ten
SubjectDisplayNamecall sites already hold both the Subject and its page'sPageSubjects; one wouldsave a lookup. The 78% figure quoted on the issue measures a walk that loads whole Subjects for their
statements, which these columns cannot spare it. They pay off for a reader that needs names without
statements — #1324's endpoint over #1334's relation index. Landing the migration together with #1334 instead
of ahead of it is a reasonable call to make on this PR.
Decisions worth a look:
label or Schema name stores as null, matching
SubjectLabel::fromTextandSchemaName.renaming a Subject, or changing which one is Main, now updates the index where nothing happened before.
Ordinary edits therefore write more often than they did, and
ClearDefaultSubjectLabelsrewrites every rowit touches. Not measured against ADR 29's interactive-save budget.
update.phpruns post-update scripts outsidedoUpdates(), so the backfill runs even when--noschemaskipped the ALTER.Tests cover null-on-blank, the five-column comparison through a rename, and the columns on both the hook and
the rebuild paths. The guard's negative branch has no test: reaching it needs DDL inside a test transaction.
It was exercised by hand instead, as below.
Known gaps, not fixed here:
exactly that. Whether the amendment belongs here, with Answering "what points at this Subject" #1334, or the columns belong elsewhere is a call for
review.
every pre-existing row. Latent while nothing reads them, wrong display names once something does. Repair is
php maintenance/run.php extensions/NeoWiki/maintenance/RebuildSubjectPageIndex.php --force.$wgSQLMode = '') truncatessilently, and the truncated row then fails the fast-path comparison on every later save; dev and CI
(
STRICT_ALL_TABLES) throw inside the revision transaction instead.Manual upgrade check
php maintenance/update.php --quick --noschema. It reports...skipping schema change, thenThe NeoWiki subject -> page index is not up to date with the schema yet; nothing to rebuild., and does not record the backfill as done.php maintenance/update.php --quick. It adds the field, then reportsDone. Indexed N pages holding Subjects.SELECT COUNT(*), SUM(nwsp_schema IS NOT NULL), SUM(nwsp_is_main) FROM neowiki_subject_page;— every rownames a schema, and the Main count is one per page that has a Main Subject, which need not be every page.