I build LLM agents and the machinery that makes them trustworthy: post-training loops, data synthesis, evaluation gates, and the sandboxing that lets an agent run code safely. MS student, open to ML Engineer / AI Engineer internships working on LLM agents, post-training, or evaluation.
What I have shipped
- Agent post-training. Designed and landed a bounded self-evolution loop for an SFT → DPO → GRPO ReAct tool-use agent: diagnose weak task families from eval traces, mine bad cases, synthesize targeted training data from train-only parents, freeze every round behind SHA256 manifests, and accept the new policy only if a no-regression gate passes on frozen held-out splits. Agreed with the maintainer before writing code; merged. Then ran one real round on a free T4 (Qwen2.5-0.5B): the targeted data was 100% new rows, nothing regressed, and the gate accepted with ood pass³ 50% → 75%, reported with its small-sample caveats.
- Verified generation. Built a Claude Code skill where the model emits a typed spec instead of final output, every factual claim is an executable check against the data, and a gate blocks compilation on failure. Ran an ablation with real model calls: measuring first raised first-draft acceptance from 0% to 100% and halved cost.
- Agent-framework reliability. Found and fixed release-day, Windows and dependency-floor bugs in Transformers, tiktoken, LlamaIndex, Datasets, Haystack, LangGraph, PEFT, DSPy, LiteLLM, Pydantic AI and cassetter by running their suites where their CI does not (Windows, newest dependencies, and the oldest versions each project declares it supports), bisecting to the exact dependency or torch version, and shipping a regression test with each fix. The Transformers 5.20.0 report was confirmed by a core maintainer within 19 minutes of filing.
| Project | Contribution | Status |
|---|---|---|
| AgenticArXiv-RL · SFT→DPO→GRPO pipeline for a ReAct tool-use agent | Designed and implemented the Roadmap-P3 bounded self-evolution loop (diagnose → mine bad cases → targeted data synthesis → SHA256-frozen round manifests → no-regression gate), aligned with the maintainer first (#97, #101) · Made the documented SFT data pipeline runnable again on main: stale split default and hard-coded row counts broke it end to end (#102, #103) · Closed the loop end to end: self_evolve mix with an audited manifest, a targeted phrasing set so a round adds new rows instead of duplicates, a mixed-agent guard, and one real round on a T4 with the gate result reported as-is (#108, merged); the run also surfaced a snapshot-dependent abort in the offline evaluator (#107) · Full-precision trainers crashed on every pre-Ampere GPU: torch's is_bf16_supported() counts emulation, so the trainer picked bf16 on a T4 and the model loaded in bf16; fixed the precision probe and the weight dtype with tests (#105, #106) · Restored a green main: a new test failed on every CPU-only host because TRL's GRPOConfig defaults to bf16 (#91) · CLI entrypoint unusable from the repo root, fixed with a regression test covering both invocation styles (#96) |
6 merged |
| Transformers · Hugging Face | 5.20.0 shipped with every from_pretrained broken on torch 2.7: the distributed-checkpoint guard said torch>=2.7 but the module it imports only exists from torch 2.8, so all model classes failed to import on a version the package declares it supports. Reproduced with the published wheels (5.19.0 fine, 5.20.0 broken), bisected the module across torch 2.6 to 2.9, reported it the day of the release; a core maintainer confirmed within 19 minutes and the one-line guard fix landed from the thread (#49651) · The same torch floor also broke tensor-parallel loading on 2.6 and 2.7 (Shard API renamed three times across torch versions); PR resolves the right API in one helper, with a reviewer-requested refactor applied (#49657) |
merged · maintainers are shipping it in a patch release |
| nanobot · HKUDS | GitStore.diff_commits used dulwich.porcelain.diff, which only exists from dulwich 0.24.0, while the package declared dulwich>=0.22.0: on the declared floor every commit-diff call raised GitStoreError and the line-ending test failed. Resolved the project at its minimum versions, checked the API across five dulwich releases, declared the real floor; merged the same morning (#6252) |
merged |
| tiktoken · OpenAI | On Windows, several processes populating an empty token-cache at once crashed all but the first with FileExistsError: the cache write used os.rename, which refuses to overwrite on Windows. Reproduced with 6 concurrent processes (1 ok, 5 crashes, every run), switched to the atomic os.replace, and added a deterministic test that simulates the losing process (#620) |
in review |
| LlamaIndex · core | SimpleDirectoryReader(recursive=True, exclude=[...]) silently kept top-level files on the fsspec versions the package declares (** did not match zero directories before fsspec 2023.9). Bisected fsspec, made the exclude glob independent of the fsspec version; the existing test goes from failing to passing on the floor (#23536) |
in review |
| Datasets · Hugging Face | save_to_disk / load_from_disk document a path-like argument but passed it straight to fsspec, which only accepted Path objects from 2024.6; on the declared floor every Path raised TypeError. Bisected fsspec, os.fspath() at the five entry points, verified the CI deps-minimum job never pins fsspec (#8823) |
in review |
| Haystack · deepset | pydantic had no lower bound; on every release before 2.11 the JSON schemas Tool / ComponentTool generate for dataclass parameters differ (allOf wrappers, missing required). Bisected across six pydantic versions and declared the floor the code already needs, with a release note (#13286) |
in review |
| DSPy · Stanford NLP | LocalInterpreter on Windows: any guest subprocess hung forever and captured stdout carried stray \r. Bisected the hang to the worker's blocked stdin read, isolated fd 0, and replaced taskkill-based cleanup with a kill-on-close Job Object (shutdown 1.2 s → 15 ms) (#10571, #10572) |
in review |
| LiteLLM · BerriAI | The unit-test host-environment isolation deleted SYSTEMROOT on Windows, so every test that spawned a Python child failed with WinError 10106. Kept the Windows system variables through a dependency-free allowlist module, with an in-process regression test plus a Windows-only child-interpreter check (#44767) |
in review · CI green |
| cassetter · HTTP record/replay for pytest | One crashed xdist worker aborted the whole session with INTERNALERROR, hiding every other result; fixed with a pytester regression test (#143, #144) |
in review |
| prime-rl · Prime Intellect | The trainer could not be imported on hosts whose CUDA driver predates the installed cutlass flavor, CPU-only test runs included: the optional FlashAttention-4 import caught ImportError but cutlass raises RuntimeError. One-line guard takes the CPU suite from 23 collection errors to 0 failures (#4050, #4051) |
draft PR in review |
| LangGraph · LangChain | get_input_jsonschema() / get_output_jsonschema() raised PydanticSchemaGenerationError on every pydantic release between the declared floor (2.7.4) and 2.10 for state TypedDicts that use Annotated[Required[...]], the only spelling that lets a required key carry a reducer or metadata: pydantic only learned it in 2.11. Bisected across five pydantic releases with a pure-pydantic reproducer, then a fix that rewrites the hint to the equivalent Required[Annotated[...]] form before schema generation, with a unit test that holds on pydantic 2.10 and 2.14 (#9320, branch) |
issue filed · fix ready, awaiting assignment |
| PEFT · Hugging Face | Every prompt-learning method (prompt tuning, prefix tuning, p-tuning, multitask) crashed on transformers 4.43/4.44: the fallback written for pre-get_text_config versions tested membership with in on a config that is not iterable there, so the "old versions" branch never worked on old versions. Reproduced across three transformers releases, one-line fix plus a regression test that fails on every version without it (#3951) |
issue filed · fix ready, awaiting maintainer triage |
| Pydantic AI | Proposal: Windows support for LocalWorkspace via a kill-on-close Job Object, with a line-referenced analysis of the POSIX-only paths and a Windows spike verifying the Job Object teardown and anyio creation flags (#9873) |
triaged · under maintainer discussion |
- storymap-gen · a Claude Code skill plus Python pipeline that turns a topic and GIS data into an ArcGIS StoryMap. The model only emits a typed spec (pydantic); every claim about the data is an executable check against the layer (street-vs-grid bearing, layer statistics), and a quality gate refuses to compile a story with a failing claim. Ablation with real model calls: handing the writer measured facts before drafting raised first-draft acceptance from 0% to 100% and halved cost per story. 26 tests; all three layouts load in the live StoryMaps Builder.
- Agentic RL and post-training: GRPO / DAPO, RL with verifiable rewards, SFT → preference → RL curricula, reward design for tool use.
- Data engines: targeted synthesis from failure analysis, parametric task derivation, leakage control between train and held-out splits.
- Evaluation: pass@k / pass³ on frozen splits, LLM-as-judge with blind rubrics, bad-case libraries that only grow, deterministic offline replay of tool calls.
- Agent runtimes: ReAct tool-use loops, code sandboxes and process isolation (Job Objects, process groups), structured outputs as the contract between model and code.
- Reproducibility: SHA256-frozen data manifests, recipe pinning, honest Before/After reporting, including when the gate says no.
Python, PyTorch, TRL / PEFT, Hugging Face Transformers, pydantic, pytest; DSPy, LiteLLM, Pydantic AI from the inside.