Skip to content

[Perf] Reduce cold compile and cached restart latency - #3

Merged
alanray-tech merged 4 commits into
dev/compile-time-optfrom
perf/compile-restart-speed
Oct 1, 2026
Merged

alanray-tech merged 4 commits into
dev/compile-time-optfrom
perf/compile-restart-speed

Conversation

@alanray-tech

Copy link
Copy Markdown
Owner

Summary

  • Store cached LLVM modules as bitcode, with backward-compatible textual IR loading.
  • Use bitcode fingerprints for PTX caching and partition PTX by external optimization level.
  • Cache immutable Python function-source preprocessing during repeated inlining.
  • Skip dataclass AST flattening when a function has no struct locals.

Design notes

  • Targets dev/compile-time-opt in this fork, so the existing graph/checkpoint stack stays outside this PR.
  • Cache validation and specialization semantics are unchanged.
  • Two speculative CSE optimizations were measured and rejected; they are not present in this branch.

Measured results

  • Cold first step: 107.60 s -> 95.19 s (11.5% lower).
  • Python materialization: 31.01 s -> 23.16 s (25.3% lower).
  • Cached first step: 20.19 s -> 2.30 s (8.77x lower).
  • Large LLVM artifact load: 16.83 s -> 0.84 s.

Test plan

  • Offline-cache CPU/CUDA round trips: 2 passed.
  • Offload and optimization tests: 35 passed.
  • Kernel implementation, dataclass, and AST tests: 363 passed, 6 xfailed.
  • Black, clang-format, whitespace, Ruff, and Pylint hooks passed.
  • Genesis 200-step O1/O3 runtime guard completed with no overflow.

alanray-tech and others added 4 commits October 1, 2026 12:11
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
@alanray-tech

Copy link
Copy Markdown
Owner Author

AI attestation (2026-10-01): the code in this PR was generated by GPT-5.6 Sol. No manual code tweaks were made after generation. If a maintainer requests a stronger-model pass, I will switch to Claude Opus for the follow-up.

@github-actions

github-actions Bot commented Oct 1, 2026

Copy link
Copy Markdown

@alanray-tech
alanray-tech merged commit ff9d863 into dev/compile-time-opt Oct 1, 2026
41 of 65 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant