Epic: JSON Schema Compatibility Checker

[jsenko]

Overview

JSON Schema compatibility checker in the data-models library, replacing the everit-based checker in Apicurio Registry. Determines whether a schema update is backward/forward compatible — critical for schema registry governance.

Package: io.apitomy.datamodels.jsonschema.compat (checker), io.apitomy.datamodels.jsonschema.ref (reference resolution).

Related: Generator improvements epic #1041.

---

Phase 1: Compatibility Checker (Draft 4/6/7) — ✅ Complete

Core checker implementation targeting Draft 4/6/7 parity with the existing everit-based checker.

ID	Description	Dependencies	Status
C1	Core Framework — DiffType enum (153 values), Difference, DiffContext, DiffUtil, JsonSchemaCompatibilityChecker entry point.		✅ Done
C2	SchemaAccessor — Bridges the Document/JSchema interface gap so diff code can access properties uniformly regardless of root vs nested schema.		✅ Done
C3	SchemaDiffVisitor — Top-level dispatch by type property value to type-specific diff classes.	C1	✅ Done
C4	ObjectSchemaDiff — required, properties, additionalProperties, patternProperties, propertyNames, dependencies, minProperties, maxProperties.	C3	✅ Done
C5	ArraySchemaDiff — items (single + tuple), additionalItems, minItems, maxItems, uniqueItems, contains.	C3	✅ Done
C6	StringSchemaDiff — minLength, maxLength, pattern, format, contentEncoding, contentMediaType.	C3	✅ Done
C7	NumberSchemaDiff — minimum, maximum, exclusiveMin/Max (cross-version draft-4 boolean ↔ draft-6+ number), multipleOf, integer requirement.	C3	✅ Done
C8	Composition Keywords — allOf/anyOf/oneOf/not with reordering support and keyword switching.	C3	✅ Done
C9	Conditional Schemas — if/then/else (draft-7+).	C3	✅ Done
C10	Enum/Const — Comparison with const-enum equivalency.	C3	✅ Done
C11	Boolean Schema Handling — true/false semantics in sub-comparisons.	C3	✅ Done
C12	Unsupported Feature Tracking — DiffContext.addUnsupported() for multi-type, modern versions, etc.	C1	✅ Done
C13	ModelTypeDetector — Fallback to Draft 7 for schemas without $schema.		✅ Done
C14	Test Data — 145 test cases ported from Registry. 144/145 passing (1 skipped: boolean schema as root — needed G1).		✅ Done

Phase 2: Registry Integration — ✅ Complete

Integrate as a configurable alternative to the existing everit-based checker in Apicurio Registry.

ID	Description	Dependencies	Status
C15	Data-models migration — apicurio-data-models → apitomy-data-models renaming.		✅ Done (Registry PR #8130)
C16	ApitomyJsonSchemaCompatibilityChecker — Adapter implementing Registry's CompatibilityChecker interface.	C1	✅ Done (Registry PR #8202)
C17	RegistryResourceResolver — External $ref resolution from Registry's pre-resolved Map<String, TypedContent> storage.	C1	✅ Done (Registry PR #8202)
C18	Config switch — Experimental feature flag apicurio.compat.json-schema.use-apitomy to toggle between checkers.	C16	✅ Done (Registry PR #8202)
C19	Parameterized tests — Run both checkers against the same test suite.	C16	✅ Done (Registry PR #8202)
C20	Ref resolution refactoring — FragmentResolver + ResourceResolver pattern. ResolvedRef removed, use Node.isAttached() directly.		✅ Done (PR #1057)
C21	Data-models 3.1.1 release — Published with checker included.	C20	✅ Done

Phase 3: Checker Updates + Generator Foundations — ✅ Complete

Update the checker to use the improved generated code from #1041, and complete remaining generator foundations needed for checker correctness.

ID	Description	Dependencies	Status
C22	Union-as-root / JSON Schema spec rewrite — Eliminate Document entity. Root is JsonSchema = boolean|FullSchema type alias. Eliminates Document/JSchema duplication that forced SchemaAccessor.	G1, G2 (#1041)	✅ Done (PR #1097)
C23	Update checker for new entity names — Hand-written compat and ref code updated to new class names (JSchema → FullSchema, old prefixes → new).	C22	✅ Done (PR #1097)
C24	Remove SchemaAccessor — Deleted SchemaAccessor, replaced with direct JFullSchema usage. Moved getTypeString/getTypeList/get$ref to DiffUtil. Cleaned up duplicate instanceof checks in AnchorFragmentResolver.	C23	✅ Done (#1104)
C25	$ref / $id resolution — Fixed NodeImpl.root() bug (checked isRoot() before _parent, so all entities with _modelType returned themselves as root). All 6 test cases enabled and passing: JSON Pointer, anchor via $id, recursive with cycle detection.		✅ Done (#1105)
C26	Traverser map/list-of-union support — Added isUnionList/isUnionMap to CreateTraversersStage. Previously some union collections were missed (e.g., properties, allOf, definitions). Now all union list/map properties generate traversal calls.	G12a (#1041)	✅ Done (#1106)
C26b	Traverser Any entry point — Traverser.traverse(), AbstractTraverser, ReverseTraverser, VisitorUtil.visitTree() now accept Any instead of Node. Handles both entity and union root values.	G12b (#1041)	✅ Done (#1107)
C27	Non-determinism fix — Generator fix (G6.3 in #1041). HashSet→ordered collections in SpecificationIndex. Deferred — not reproducible locally (HashMap ordering is consistent within a JVM version). The blocking bug in CreateUnionTypeValuesStage is no longer relevant (stage is dead code). Fix is mechanical (~20 HashSet→LinkedHashSet, HashMap→LinkedHashMap) when a reproduction is found.	G6.3 (#1041)	⌛ Deferred
C28	dependencies typing — Added Dependency = FullSchema|[string] type alias (drafts 4-7), changed dependencies to {Dependency}. Fixed ApplyUnionInterfacesToTypesStage fallback for primitive list union values, fixed CreateClonersStage for primitive list variants in union maps. Checker uses typed isStringList()/isFullSchema() access.		✅ Done (#1109)
C29	Product types evaluation — Evaluated. Product types would eliminate type-switch dispatch, enable multi-type schemas, and provide proper visitor pattern for components. Components are entities (not nodes) that share the parent's JSON object. Design prototype on branch prototype-product-types, detailed design in #1110. Implementation deferred until after generator cleanup (Phase 4).	G5 (#1041)	✅ Evaluated (#1110)

Phase 4: Generator Code Cleanup — ✅ Complete

Refactoring pass to improve generator code readability and maintainability before adding more complex features. Stages: 8269→4027 lines (-51%). Method classes + code blocks: 54 files. AbstractJavaStage: 735→354 lines. JsonUtil: 961→432 lines. Pipeline: 69→68 stages.

ID	Description	Dependencies	Status
C30a	Set resolvedType on star/regex properties — Already done: CreatePropertyAndTypeModelsStage calls resolveType() for ALL properties including star/regex. The tracking doc claim was outdated.		✅ Already done
C30a	Set resolvedType on star/regex properties — Already done: CreatePropertyAndTypeModelsStage calls resolveType() for ALL properties including star/regex. The tracking doc claim was outdated.		✅ Already done
C30b	Migrate AbstractStage helpers to resolvedType — All property type helpers (isEntity, isEntityList, isEntityMap, isPrimitive, isPrimitiveList, isPrimitiveMap) now check resolvedType first.	C30a	✅ Done (#1112)
C30c	Migrate AbstractCreateMethodsStage to resolvedType — Simplified getter/setter/add/remove/insert to use resolvedType directly. 494→345 lines (-30%). Set resolvedType on synthetic regex properties. createFactoryMethod/createUnionFactoryMethods still use PropertyType.	C30b	✅ Done (#1113)
C30d	Migrate reader/writer/cloner dispatch to resolvedType — All three stages route ALL property types through handleViaResolvedType. PropertyType fallback branches removed from writeTo(). Internal handler methods still use PropertyType for value type determination.	C30c	✅ Done (#1114)
C30e	Delete PropertyType — Removed PropertyType class, CreatePropertyModelsStage, TypeParserTest, JavaType inner class (265 lines), UnionPropertyType inner class (45 lines). Migrated all 15 stage files. Net -754 lines.	C30d	✅ Done (#1115)
C31	Dead code cleanup — Old union stages already deleted in earlier PRs. Remaining: deleted CreateParentTraitsStage (commented out), resolved stale TODOs, fixed SpecificationIndex HashSet→LinkedHashSet.		✅ Done (#1111)
C32	Method generation refactoring — Extracted 11 method classes (GetterMethod, SetterMethod, AddMethod, InsertMethod, RemoveMethod, ClearMethod, FactoryMethod) + CanAddImports, CodeBlock, ImplMethodContext, ParentAttachmentBlock abstractions. Stage: 593→307 lines.	C30	✅ Done (#1117)
C33	Extract shared property helpers — Re-scoped from original G7.2. Extracted PrimitiveTypeHelper (type determination) and EntityResolver (entity lookup with null-check+warn). Eliminated ~80 lines of duplicated logic across reader/writer/cloner blocks.	C30	✅ Done (#1120)
C33b	Refactor CodeGenContext ownership — Made CodeGenContext a concrete class owning entity resolution, FQN construction, naming, and field logic directly. Merged ImplMethodContext. Stages create instance, no longer implement interface.	C33, C32d	✅ Done (#1124)
C33c	Method naming + PropertyCodeGen — Phase 1 (#1126): Method interface, 12 method/naming classes, removed 29 naming overloads. Phase 2 (#1127): PropertyCodeGen wrapper (code blocks take 1-2 params instead of 3-4), getGetterName()/getSetterName() on PropertyCodeGen replacing static calls in 19 code blocks.	C33b	✅ Done (#1126, #1127)
C34	Simplify JsonUtil — 88→40 methods (-55%). Replaced 56 type-specific consume/get/set with generic get+check+remove (reader) and setProperty+toJsonNode (writer). Added validation: allMatch/allValuesMatch for defensive reading. Type mismatch = extra property (preserved).		✅ Done (#1121)
C32b	Apply code block pattern to reader/writer/cloner — 21 code blocks extracted + CodeGenContext interface. Reader: 1421→1249, Writer: 1152→983, Cloner: 789→400. Total -730 lines (-22%). Star/regex/legacy union handlers remain in inner classes (follow-up cleanup).	C32	✅ Done (#1118)
C32c	Delete legacy union handlers from reader/writer — Deleted dead handleUnionProperty, handleUnionListProperty, handleUnionMapProperty. Reader: 1249→777, Writer: 983→712. Total -743 lines.	C32b	✅ Done (#1119)
C32d	✅ Stage structure reorganization — (1) Star/regex code blocks (#1128). (2) ReaderMethod/WriterMethod.writeTo() (#1129). (2b) Method.writeTo unification (#1130). (2c) EmptyClone/Accept/UnionIs/As/MappedNode (#1131). (2d) Factory method classes (#1132). (3) AbstractIOStage (#1133). (5) Merge stages + polish (#1134, #1135).	C32c	✅ Done
C34b	BodyBuilder improvements — Added appendBlock (text blocks), ifElse/ifTrue (conditionals), forEach (loop with scoped context + isFirst), addContext(Map). Applied to 11 code blocks. Template files not needed — 44% of code gen is interleaved Java logic.		✅ Done (#1125)
C34c	(Optional) appendBlock indent stripping — Deferred. Flush-left convention works, no need currently.	C34b	⌛ Deferred
C34d	Remove AbstractStage type predicates — Replace isEntity(property), isPrimitive(property), etc. with inline property.getResolvedType().isEntityType() across all stages. Removes indirection layer.		⌛ Deferred
C34e	Instance-based Logger — Refactor Logger to instance pattern: private final Logger log = Logger.getLog(getClass()) per stage, then log.warn(msg). Eliminates AbstractStage logging delegation wrappers (105 call sites).		⌛ Deferred
C35	Generator style unification + CLAUDE.md — CLAUDE.md with architecture, BodyBuilder, JSweet compat, generated code style. Converted all forEach→for in generated code (2387→1).		✅ Done (#1136)

Phase 5: Modern Versions (2019-09 / 2020-12)

Add modern JSON Schema version support to the checker. Paired visitor generated before this phase to simplify the implementation.

ID	Description	Dependencies	Status
C35b	✅ Generated code cleanup — (1) writeRoot returns null for boolean schema roots — checks isObjectNode(result) instead of handling JsonNode directly. (2) Union writer casts (java.util.List<?>) union.asStringList() unnecessarily — asStringList() already returns List<String>. (3) ParentPropertyKind enum should hold its string value directly instead of using kindToString() switch. (4) Review other generated code for similar cast/style issues. writeRoot→JsonNode, DataModelUtil.setParent unified, JsonType enum, conditional ModelType, union map validation, insert javadoc. (#1137, #1138)		✅ Done
C35c	Generated PropertyName enum — Generate an enum of all field names per spec version. Replaces string literals in parent tracking, reader/writer property names, pairing strategies, and visitor dispatch. Can carry metadata (property type, collection kind). Cross-cutting — design after paired traverser to see all use sites.		⌛ Deferred
C36	✅ Paired visitor / diff visitor — (G13 in #1041). Generate PairedVisitor<T> base class where each visit method receives both current and corresponding node from another tree. Generated DiffTraverser/DiffVisitor per spec version with typed per-field methods, union dispatch, auto-recursion, PairingKey, boolean return for recursion control, post-visit callbacks. (#1139, #1141, #1150)	G5 (#1041)	✅ Done
C36b	DiffTraverser traversal context — Dual TraversalContext (original/updated paths), PairingKey interface, pushListIndex/pushMapKey. (#1141)	C36	✅ Done
C37	✅ $defs / dependentSchemas / dependentRequired — 2019-09 keywords. D4-d7 dependencies split into dependentSchemas+dependentRequired during conversion. $defs handled by auto-recursion. flagModernVersions removed. 8 new test cases including cross-version d7↔2019-09. (#1181)		✅ Done
C38	✅ prefixItems / unevaluatedItems / unevaluatedProperties — D4-d7 tuple items normalized to prefixItems during conversion. unevaluatedItems/unevaluatedProperties with schema comparison. 5 new test cases including cross-version d7↔2020-12. (#1182)		✅ Done
C39	✅ $dynamicRef / $recursiveRef — For now, left as-is by the dereferencer and compared as strings by the CC (same as cyclic $ref). Full dynamic scope resolution deferred to Phase 5b.	C25	✅ Done (string comparison)
C41	✅ Modern version test data — 21 test cases for 2019-09/2020-12 features: $defs, dependentSchemas/dependentRequired, prefixItems, unevaluatedProperties, $recursiveRef, $dynamicRef, if/then/else, composition, enum/const. Cross-version d7↔2019-09/2020-12 tests for definitions, dependencies, tuple items. (#1181, #1182)	C37, C38, C39	✅ Done
C41b	✅ Multi-valued type support — Compare type arrays as sets. Types added = backward compatible, types removed = incompatible. integer/number normalization. 6 test cases. (#1182)		✅ Done
C41c	✅ Canonical compound schema for cross-version comparison — Create a synthetic compound JSON Schema spec that merges all properties from all draft versions into a single entity. Where a property changed type across drafts (e.g., exclusiveMinimum: Boolean in d4, Number in d6+), the compound spec uses either a union type (leveraging is/as methods) or a custom entity/property not in any real spec. Generate converters (G26) from each draft version to the compound type. The diff classes then only compare compound-to-compound — all version-specific instanceof checks move into the converters. This also makes modern schema support (C37-C41) much simpler since new versions just need a new converter, not changes to every diff class. Similarly handles renamed properties (definitions → $defs) and properties added in later versions (if/then/else, $anchor).	G26 (#1041)	✅ Done (#1142, #1143)
C41d	CC: Boolean return for recursion control — Field diff methods return boolean to control auto-recursion. Eliminated suppressAutoRecursionCount hack.	C36	✅ Done
C41e	CC: Post-visit callbacks — afterVisitEntity/afterDiffField methods generated.	C36	✅ Done
C41f	CC: Scoped DiffContext — pushScope/pushIsolatedScope/popScopeIsCompatible. Replaced isolated() pattern. (#1153)	C41e	✅ Done
C41g	CC: Normalize min/max to RangeValue — New RangeValue entity (value+exclusive). Converters normalize d4 boolean+minimum to RangeValue. 70-line comparison reduced to 20. (#1152)	C41c	✅ Done
C41h	CC: Fix \ resolution for compound schemas — Updated AnchorFragmentResolver to handle JCFullSchema. Consolidated detectModelType. Re-enabled \ test. 144/145 CC tests passing. (#1156)	C41c	✅ Done
C41i	CC: Use traverser context for paths — Replace DiffContext.sub() path tracking with traverser's TraversalContext. Only create isolated DiffContext for scoped compatibility checks.	C36b	⌛ Deferred
C41j	CC: Use CollectionDiff in diffProperties — Use the traverser's pairing result (added/removed/matched) instead of custom key matching. Apply additionalProperties logic on top.	C36	⌛ Deferred
C41l	CC: Original path mapping via node attributes — During compound conversion, store original field names as node attributes. DiffContext.sub() checks for the attribute and uses original name if present. Fixes path accuracy for renamed fields (id/$id, definitions/$defs) and RangeValue. General solution — always check attribute existence. Should be considered holistically with C42 (human-readable DiffType descriptions) and C41i (traverser context for paths).	C41c	⌛ Deferred
C41k	✅ CC: Ref resolution + unresolvable ref strategy — UnresolvableRefStrategy enum (COLLECT/FAIL/IGNORE) configurable via builder. MapResourceResolver for in-memory schema resolution. Common JsonSchemaCompatibilityException base exception. Enhanced test infrastructure with config, externalRefs, expected fields. (#1157, #1158)	C41c	✅ Done
C41m	✅ CC: Upfront tree dereferencing — JsonSchemaRefDereferencer with builder pattern, resolves all non-cyclic $ref nodes upfront. CC pipeline: parse → dereference → convert → diff. Ancestry-based cycle detection leaves only back-edges as $ref. DereferenceResult with cyclicRefs map and unresolvedRefs. UnresolvableRefStrategy (COLLECT/FAIL) moved to ref package. Exception hierarchy: JsonSchemaProcessingException → ReferenceResolutionException/DereferenceException/JsonSchemaCompatibilityException. Max depth safety net. (#1162)	C41k	✅ Done
C41n	CC: Use generated traverser in dereferencer — Replace manual dereferenceChildren property enumeration with the generated traverser once G13c (boolean return + post-visit callbacks on regular traverser) is implemented. Eliminates maintenance burden of keeping the property list in sync with the schema spec.	C41m, G13c (#1041)	🔲

Phase 5b: Dynamic Reference Resolution & Strictness — 🔲 Planned

Proper handling of $recursiveRef/$dynamicRef and configurable strictness for cases where the CC can't fully resolve schemas.

ID	Description	Dependencies	Status
C40	Extended ref resolver — Base URI tracking, URL resolver for external references.	C25	🔲
C66	Dynamic scope resolution in dereferencer — Resolve $recursiveRef (walk up to outermost $recursiveAnchor: true) and $dynamicRef (find nearest $dynamicAnchor by name in dynamic scope). Requires tracking schema composition context during dereferencing. Report resolved targets in cyclicRefs.	C41m	🔲
C67	CC: Dynamic ref comparison — After proper resolution, compare the resolved targets structurally instead of comparing ref strings. Cross-schema override scenarios (extending schema captures the recursive hook) handled correctly.	C66	🔲
C68	Configurable strictness — CompatibilityStrictness enum (STRICT/LENIENT) on the CC builder. STRICT (default): current conservative behavior. LENIENT: treat unresolvable/dynamic refs as compatible unless provably incompatible. Applies to: unresolved $ref, $dynamicRef/$recursiveRef, multi-valued type (C41b).		🔲
C69	Split test data by feature group — Split compatibility-test-data.json into per-feature files (string, number, array, object, composition, conditional, ref, modern). Cross-version tests stay with their feature group. Update test runner to load multiple files.		🔲

Phase 6: Checker API Improvements

Improve the checker's public API for consumers.

ID	Description	Dependencies	Status
C42a	✅ CC API design review — Builder pattern, CompatibilityCheckResult/FullCompatibilityCheckResult types, JsonSchemaCompatibilityException base exception, UnresolvableRefStrategy, MapResourceResolver. (#1157, #1158)		✅ Done
C42	Human-readable DiffType descriptions — Currently raw enum names (e.g., SUBSCHEMA_TYPE_CHANGED). Add user-friendly descriptions.		🔲
C43	YAML input support — Accept TypedContent / content type parameter. Parse YAML schemas.		🔲
C44	Accept pre-parsed documents — Allow callers who already have a parsed model to skip re-parsing.		🔲
C45	Backward/forward tagging — Tag each Difference with its direction (backward-incompatible, forward-incompatible, or both) for FULL compatibility mode.		🔲
C46	Structured result type — Machine-readable result with codes, paths, severity. Replace flat List<Difference>.		🔲
C47	✅ Unresolvable reference strategy — Implemented as part of C41k. UnresolvableRefStrategy enum + builder config. (#1158)	C25	✅ Done

Phase 7: Replace Registry JSON Schema Utilities

Replace remaining everit/Vert.x-based JSON Schema utilities in the Registry with data-models equivalents.

ID	Description	Dependencies	Status
C48	Smarter ModelTypeDetector — Check for schema keywords before defaulting to Draft 7. Infer version from version-specific keywords ($defs → 2019-09+, prefixItems → 2020-12, etc.).		🔲
C49	Replace ContentAccepter — Use ModelTypeDetector-based detection instead of everit.	C48	🔲
C50	Full JSON Schema dereferencer — Inline all $ref in a document, producing a self-contained output. JSON Schema equivalent of the OpenAPI dereferencer.	C25	🔲
C51	Replace ContentDereferencer — Use data-models dereferencer instead of Vert.x JSON Schema.	C50	🔲
C52	Replace ContentExtractor — Extract title/description via data-models model access.		🔲
C53	Replace StructuredContentExtractor — Extract property names, $defs keys via data-models model access.		🔲
C54	Replace ReferenceFinder — Use JsonRef parsing + tree traversal instead of everit.		🔲
C55	(Optional) Simplify Registry checker type hierarchy — Remove redundant *CompatibilityDifference wrapper classes, use SimpleCompatibilityDifference directly. Clarify error handling contract. See Apicurio/apicurio-registry#8184.		🔲

Phase 8: JSON Schema Validation

Replace everit/jsonsKema-based JSON Schema validation with a data-models approach.

ID	Description	Dependencies	Status
C56	Evaluate validation approach — Options: implement meta-schema validation in data-models, use jsonsKema (covers all versions, actively maintained), or keep jsonsKema and drop everit only.		🔲
C57	Implementation — Based on evaluation outcome.	C56	🔲

Phase 9: Cleanup

Remove legacy dependencies.

ID	Description	Dependencies	Status
C58	Remove everit-json-schema — Remove dependency from Registry.	C49, C51, C57	🔲
C59	Remove Vert.x JSON Schema — Remove dependency from Registry.	C51	🔲
C60	Consolidate JSON Schema handling — All JSON Schema operations use data-models exclusively.	C58, C59	🔲

Other Follow-ups

ID	Description	Dependencies	Status
C61	JUnit 4 → JUnit 5 — Upgrade test framework in data-models repo. Registry already uses JUnit 5.		✅ Done
C62	Deduplicate test case IDs — compatibility-test-data.json has many duplicate IDs (e.g., multiple "String: minLength"), making failure messages ambiguous. Ported from Registry as-is.		🔲
C63	Cycle detection improvement — Replace identity-hash-based pair cycle detection with $ref-path-based tracking to avoid theoretical hash collisions.	C25	🔲
C64	Investigate const test data — Array wrapper "const": ["3.14..."] — likely upstream test data issue where scalar was intended.		🔲
C65	TS barrel export collision — GenerateCoreTs duplicate export * for same-named types across packages. Tracked in #1101.		🔲

[jsenko]

Phase 1 PR: #1043