M6: finish all plans — audit remediation, STEP tokenizer fix, docs site#12
Merged
Conversation
…rness Eliminate three genuine stubs with real implementations (regression test: ll_gen/tests/test_log_prob_scorer.py, 7 tests green): - rl_trainer._get_log_probs() was `raise NotImplementedError`. Now delegates to a new BaseNeuralGenerator.score_token_sequence() that decodes policy logits (decode_command_logits() on VAE + VQ-VAE) and gathers the differentiable log-probability of a given token sequence. Documented as an EVALUATION score (a forward pass distinct from the sampled trajectory) — not the RL gradient (that stays on proposal.log_probs from generate_for_training), so the historical biased-gradient hazard is not reintroduced. Diffusion (no command-token decoder) returns (None, 0.0). - Wired into production: evaluate_validity scores every proposal from its own latent (deterministic reconstruction-likelihood) and reports a new GenerationMetrics.mean_sequence_log_prob. Scorer forces model.eval() (save/restore) so the own-latent score is reproducible. - neural_diffusion._create_placeholder_face_grids() returned zero grids → _latent_to_face_grids() surfaces the model's real latent (StructuredDiffusion has no separate geometry decoder). - segmentation.py: reworded a misleading "for now" comment (DBSCAN border absorption was already complete — not a stub). ll_gen 1322 passed / 10 skipped; ll_clouds 74 passed. Touched files ruff+black clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…lean Lint (ruff): cleared 737 findings in ll_ocadr. Autofixed quotes/imports/ f-strings/set-literals; removed 17 verified-dead imports + 3 dead locals (each inspected, none wired); kept the 80-byte STL header read (side effect); gave ll_ocadr a package-local ruff config matching its M-series siblings (ll_gen, ll_clouds: E,W,F,I,N,UP,B) instead of the broad root select that flags the repo's mandated lazy-import pattern; scoped N806/N812 (B/N/C tensor dims, `F` for functional) to the encoder files; modernized annotations (UP), added zip(strict=False). Format (black): formatted ll_gen + ll_ocadr to the repo style. Types (mypy): bumped python_version 3.9->3.10 (mypy 2.1 dropped 3.9) in root + ll_gen + ll_clouds. Real fixes: with_error_context typed `self: T -> T` (resolves ~15 subclass-attr errors), dict[str, any]->dict[str, Any] bug, ~15 collection annotations, a list-vs-set bug in CodeProposal.extract_modules, max(key=...) lambda, float() casts at numpy boundaries. Scoped per-module overrides disable only the unsatisfiable dynamic-boundary codes for the torch/OCC/numpy modules — consistent with the repo's "start lenient" config. mypy ll_gen/ll_gen ll_ocadr/vllm ll_clouds/ll_clouds -> 0 errors. Suites green: ll_gen 1322 / ll_ocadr 23 / ll_clouds 74. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ncile status - SPEC-1 status flipped to Done: M1–M6 ✅; Track-B items (VQ-VAE/diffusion proof-of-life, true-DDPO, dimension-conditioning) recorded as deferred by design (NG1/R2). Reconciles the stale `M2 in review · M3 ⬜` line with STATUS.md and the M3 plan. - OQ1 closed (palapav/DeepCAD-DSL, 2000/200) and OQ3 closed (217 MB checkpoint gitignored + reproduced via proof_of_life), per the M3 run log. - STATUS.md M6 row: ◐ → ✅ with the ruff/black/mypy-clean evidence. - finish-all-plans Track A checklist marked complete. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…essing it Follow-up to the M6 mypy pass: the pipeline layer was over-broadly covered by the dynamic-boundary mypy override (assignment/return-value/override/misc were disabled across orchestration logic, not just torch internals). Tighten it: - ll_gen.pipeline.* now uses a narrower override (only the dynamic-attribute / untyped-library-call codes); the bug-catching structural codes stay ON. - orchestrator: typed the lazy-init generator/conditioner attributes (NeuralVAEGenerator | None etc. via TYPE_CHECKING) — the real fix for the `x = None` then `x = Generator(...)` pattern, which also resolves the generate() return-type mismatches. - verification: typed the lazy CLIP attributes + assert-not-None after init. The broad override now applies only to the genuine library-internal modules (model forward passes, OCC tessellation, numpy struct parsing). mypy still exits 0; ll_gen suite 1322 passed. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Remediates the load-bearing deceptions from the 2026-06-09 audit (docs/2026-06-09-partial-deceptive-code-audit.md), with regression tests and full-suite verification (0 regressions in changed modules). HIGH: - brep graph builders: real shared-edge face adjacency (MapShapesAndAncestors) + real normals/curvature/dihedral/signed-convexity, replacing index-order placeholders; wire `cadling hub build --type brep_graphs` to BRepGraphBuilder. - diffusion RL: real DDPO sampler (StructuredDiffusion.sample_with_log_prob) with a param-connected trajectory log-prob — RL now trains the denoisers. - diffusion latent->geometry: real per-primitive token sets + trainable GeometryCodec (UV-Net encoder + decoder), so sample() emits usable B-Rep grids. - hole depth: measured from the cylinder's OCC axial extent, not diameter*2. MEDIUM: - ll_gen metrics: coverage/MMD/JSD are None (not fake 0.0) without a reference; real tessellated surface points instead of bbox corners. - ll_gen VLM verifier: fails closed (verified flag) instead of silent matches=True. - chamfer distance measured from face UV-extent; surface_area estimate off the canonical key; "normalized cuts" docstring corrected; symmetry constraint uses a real centroid-reflection test; UV trim mask via BRepTopAdaptor_FClass2d. - threaded VLM Stage-1: real surface-type classification + measured hole/pocket params + geometric recession (removed toy heuristic, broken extract_from_face, dead branch). - graph edge reconstruction builds real OCC edges between adjacent-face centroids. - geotoken UV quantizer warns loudly when synthesizing approximate xyz. ll_ocadr: implement the two empty encoder files as a full rendered-image modality — real CLIPVisionSDPA + SAMVaryViTSDPA encoders + VisionTower, wired into LatticelabsOCADRForCausalLM (pixel_values -> image tokens spliced alongside 3D mesh tokens, guarded by config.use_vision). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Addresses the LOW-severity audit items that were real bugs/deceptions (the honestly-labeled approximations are left as-is, being disclosed, not deceptive): - generation_metrics._is_valid_shape fails closed (False + warning) when pythonocc cannot validate a TopoDS_Shape, instead of "assume valid" which inflated validity_rate. - Deterministic hashing (new cadling/lib/hashing.py, BLAKE2b) replaces the PYTHONHASHSEED-salted builtin hash() at every site that writes token ids / feature values into data: stepnet_integration entity-type feature, chunker/tokenizer/tokenizer.py, and sdg/qa/sequence_annotator.py (4 sites) — so serialized training data is reproducible across runs/machines. - brep_graph_builder edge convexity is now a real signed centroid-plane test (convex/concave/tangent) instead of angle-magnitude (which cannot sign a 90deg edge). - ll_clouds icp inlier_rmse is computed over inlier correspondences only, matching its docstring. - uv_net._sample_face_placeholder is called with a loud warning so synthetic grids no longer enter the SurfaceCNN silently. - Corrected misleading comments above working code (geometry_extractors "Placeholder classes", feature_recognition "For now, classify as generic", graph_utils dihedral-angle unsigned-by-design). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…hangs Two assembly test files hung the full cadling suite because non-shape inputs (notably unittest.mock.Mock parts, whose attribute access auto-returns truthy children) passed the `is not None` / `if not shape` guards and were handed to pythonocc operations — BRepExtrema_DistShapeShape, BRepBndLib.Add, TopExp_Explorer, BRepAlgoAPI_Common — which hang on garbage input. Validate inputs are genuine, non-null TopoDS_Shape objects before any OCC call: - assembly_analysis.detect_mating_surfaces and the AABB pre-pass in __call__. - assembly_hierarchy_pipeline._load_component_shape (the single shape loader for mate detection and overlap checks) now returns a shape only when it is a real, non-null TopoDS_Shape, else None. This is a real robustness fix (passing non-shapes to OCC is a bug regardless of tests). The full cadling suite now runs to completion (no hangs); the formerly hanging suites pass (32 + 29 tests). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The basic STEP tokenizer (the backend's fallback when OCC yields no
shape, parsing_method="basic") silently returned zero entities for
every file. parse_step_file normalized whitespace — correctly collapsing
newlines, since STEP is whitespace-insensitive outside string literals —
and then split on '\n' and matched sections with startswith('DATA;').
After normalization there are no newlines, so no section ever matched
and _parse_entities received an empty list while the backend reported
success with zero CAD items.
- Split normalized content into STEP statements (';'-terminated) via a
new string-literal-aware _split_statements helper; a ';' inside a
quoted value (e.g. FILE_DESCRIPTION(...,'2;1')) no longer terminates a
statement. Sections are detected from statement keywords, not '\n'.
- Fix _collect_multiline_entity string tracking: a doubled '' is an
escaped quote ONLY when already inside a string; the empty-string
literal '' (ubiquitous in CARTESIAN_POINT('',(...))) was mistaken for
an escape, leaving in_string stuck open so the terminating ';' was
never seen and consecutive entities were glued together. Now uses
lookahead, matching _split_statements; also fixes a latent bug for
genuine multiline files with empty-string literals.
- Align the _parse_single_param numeric contract: numeric tokens are
coerced to int/float — the canonical contract every consumer relies on
(feature extraction, coordinate parsing, and ll_stepnet tokenization
all treat numeric params as numbers on their primary path; the string
branches are defensive fallbacks). Correct the stale test that
asserted "kept as strings" and document the contract on the method.
Tokenizer + STEP integration suites: 11 failed -> 0 (91 passed). Full
cadling unit+integration suite: 45 -> 26 failures (1485 -> 1497 passed);
all 26 remaining failures pre-date and are unrelated to this change.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- site/: Astro Starlight documentation site — per-package overview/installation/usage (cadling, geotoken, ll_stepnet, ll_gen, ll_ocadr, ll_clouds), concepts, guides, tutorials, and roadmap; scripts/gen_api.py regenerates the API reference from docstrings. node_modules, dist, and build are git-ignored (site/.gitignore, and the root .gitignore now un-ignores /site while ignoring site/build outputs). - docs/specs/SPEC-2-documentation-site.md and docs/2026-06-10-spec2-accuracy-review.md: the site spec and its accuracy review. - .github/workflows/docs.yml: documentation build/deploy CI. - README.md tweaks; remove the superseded top-level Review.md and STATUS.md and geotoken/GeotokenReview.md. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… module ll_brepnet is the roadmap item described in site/src/content/docs/roadmap/ll_brepnet.md. This commits the initial package skeleton only: the directory layout and empty module files (models/, dataloaders/, pipelines/, eval/, tests/) plus empty pyproject.toml, requirements.txt, and environment.yaml. All 13 files are 0-byte placeholders — there is no implementation yet; they establish the package structure for future work. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
![sourcery-ai[bot]](https://avatars.githubusercontent.com/in/48477?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Sorry @LayerDynamics, your pull request is larger than the review limit of 150000 diff characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this is
The M6 milestone branch: it finishes the SPEC-1 plans and remediates the partial/deceptive code found in the 2026-06-09 monorepo audit, so the toolkit's marquee claim — "process CAD geometry into genuinely trainable data and generate CAD back" — is actually true end-to-end rather than faked at key seams. Plus a new public documentation site and a scaffold for the planned BRepNet package.
10 commits, 189 files (+17,786 / −2,586), 47 test files touched.
Highlights
Audit remediation — partial & deceptive code made real
Full details in
docs/2026-06-09-partial-deceptive-code-audit.md. The load-bearing fixes:ll_gengenerate_for_trainingreturned the log-prob of a freshtorch.randnleaf. Rewired to real DDPO viaStructuredDiffusion.sample_with_log_prob(stochastic DDIM, per-step Gaussian log-prob, sampled action detached so∇log πflows through the denoiser-predicted mean). Verified: one train step updates all denoiser params. Added a real trainableGeometryCodec(UV-Net Conv encoder + mirrored decoder + masked-MSE) so latents actually decode to B-Rep geometry.min(i+5,…)) with zeroed features. All now delegate to a real shared-edge adjacency helper (cadling/lib/topology/brep_face_graph.py, OCCMapShapesAndAncestors+ real normals/curvature + signed convexity).cadling hub build --type brep_graphsis now wired through (was unreachable).matches:Trueon every error) → fails closed with averifiedflag.0.0) →Nonewhen undefined, real tessellated points otherwise.ll_ocadrencoder files → a full optional rendered-image modality (CLIPVisionSDPA+SAMVaryViTSDPA+ dual-branch tower, spliced into the LLM atimage_token_id).stable_hash(BLAKE2b) replacing PYTHONHASHSEED-saltedhash()at data-writing sites; signed convexity; fail-closed shape validation; ICP inlier RMSE; corrected misleading docstrings.STEP tokenizer correctness
The basic STEP parser silently returned zero entities for every file (newline-split after whitespace normalization + an
''empty-string literal bug in the multiline collector that glued entities together). Fixed both with a string-literal-aware statement splitter and lookahead-based quote tracking; aligned the_parse_single_paramnumeric contract (coerce toint/float, the contract every consumer relies on) and corrected the stale test that asserted otherwise.Test-hang fixes
Two assembly tests hung forever:
Mockshapes are truthy andis not None, so they passed guards into OCC C++ calls. Fixed withisinstance(x, TopoDS_Shape) and not x.IsNull()validation inassembly_analysis.pyandassembly_hierarchy_pipeline.py. The full cadling suite now runs to completion.Docs site + roadmap scaffold
site/: Astro Starlight documentation site (per-package overview/install/usage, concepts, guides, tutorials, roadmap);scripts/gen_api.pyregenerates API reference from docstrings.node_modules/dist/buildgit-ignored. CI in.github/workflows/docs.yml.ll_brepnet/: empty package scaffold only (13 zero-byte files) for the planned BRepNet roadmap item — directory layout + module/config placeholders, no implementation yet.Earlier M6 work on this branch
feat(ll_gen): real teacher-forcing sequence scorer wired into the eval harness.style/refactor: M6 quality gate (ruff/black/mypy clean) and type-checking the ll_gen orchestration layer instead of suppressing it.docs: SPEC-1 closeout (M6 done, OQ1/OQ3 resolved).Verification
Reviewer notes
ll_brepnet/is intentionally empty (scaffold). Do not expect implementation there.🤖 Generated with Claude Code