native-fuzz.md

Native generated-input campaigns

Run the existing Crowbar properties without AFL or instrumentation:

tools/dev native-fuzz --rounds 10000 --batches 3 --seed 42
tools/dev native-fuzz --target websocket --rounds 100000 --batches 10 --seed 100
tools/dev native-fuzz --target request --input path/to/failure.input

The shared toolchain/fuzz-targets.json catalog currently selects 14 targets: core values, request/response heads, chunked bodies, client/server engine, partial writes, adapters, exchange isolation, URL, forms, router, multipart and WebSockets. Each batch is a fresh process, using successive explicit 64-bit seeds. --timeout bounds each batch's execution (default 120 seconds), including properties that stop making progress. The runner terminates and joins timed-out processes. A timeout is a failed investigation, never a passing shorter run.

Reports under _artifacts/native-fuzz/run-*/ preserve source, catalog and binary hashes, compiler, selected case, seed, command, requested trials, elapsed time, exit status and each process's raw log. A passing exit without exactly one passing registered property is rejected. Schema3 also records generated, checked, skipped and failed callbacks, maximum generated/checked input lengths, and the hash of a separate counter file. Checked means the property returned successfully; it is not a count of assertions or unique paths. Counts must account for every requested trial, with no failed callbacks and at least one checked input. Missing, inconsistent, negative or duplicate counters cannot yield PASS. Each run's seconds is measured inside the child, from its first property callback until final counter export (timing_scope: child_campaign). It includes generation between callbacks and final child reporting, but excludes startup before the first callback, parent polling, build and source/report verification. This is elapsed time, not CPU time. wait_seconds and wall_seconds record parent waiting and total batch/verification time separately. Incomplete children have no accepted campaign duration; reported child time cannot exceed the parent batch elapsed time. Earlier reports included verification in their elapsed field and cannot satisfy the release-duration contract by relabeling that value. Failure or interruption preserves a FAIL report and the active batch's diagnostic log. Completed batch logs remain available. Tracked sources must stay unchanged throughout a campaign.

When a generated-input property throws, the harness also saves its exact input bytes in a private, exclusively created .input file. It never overwrites an existing capture, and a capture error cannot replace the original property exception. The failed run records the capture path and hash when available. Process termination, timeout, generator failure before the property is called, or an unwritable capture directory can leave no captured input; keep the seed, command and log as well. A shared guard counts oversized generated inputs as skipped before entering the property. Only completed callbacks count as checked. Counter export happens after Crowbar's exit-hook execution; a killed process may have no counter file and never produces accepted evidence.

--input replays one selected target directly on those bytes, without generation. It accepts a regular file of at most 64 KiB and snapshots it into the report directory. It cannot be combined with --rounds, --batches or --seed. The report identifies raw replay separately from seeded trials. Target-specific length guards still apply; oversized raw inputs fail rather than passing without checks. Engine/adapter/isolation inputs are capped at1024 bytes, web properties at2048, and core/HTTP1 at65536. Crowbar's positional FILE argument is generator entropy; it is not a substitute for raw-byte replay.

To reproduce, use the report's command and HTTP_KIT_FUZZ_CASE selection with the matching sources/toolchain. A null selection means unset that environment variable. Keep the original report and input when minimizing a failure; add a small deterministic regression before fixing the implementation.

The generator mixes Crowbar's short arbitrary bytes (at most64 bytes in the locked version) with independently generated bytes at explicit length boundaries: empty, one byte,64/126/256/1024/2048/8192/16384/65536 neighborhoods, restricted to the target cap plus one. Large inputs are composed from short primitive blocks. The deliberate cap-plus-one cases prove skip accounting without inflating the checked budget. HTTP1 and web properties retain their grammar-aware companions and segmentation checks. Same numeric seeds across generator revisions are not equivalent inputs; compare source and binary hashes before replaying evidence.

These are seeded generator trials, not coverage-guided search. The existing scenario harness provides shrinking for its own scenario format; this runner does not automatically shrink Crowbar failures. Use the separate minimizer below for captured bytes. Release durations and independent review are separate requirements. A PASS here does not certify release readiness, bounded RSS, or the skipped AFL campaign. CI integration is deferred by request.

Resumable duration campaigns

tools/dev native-campaign --session-seconds 43200
tools/dev native-campaign --resume _artifacts/native-campaigns/run-EXAMPLE \
  --session-seconds 43200

The default targets are the same14 properties. Each must reach all three policy minima:1800 seconds measured inside successful children,100000 checked inputs, and20 distinct seeds. --seconds, --checked, --seeds, --target, --rounds, --seed and --timeout can select a smaller diagnostic run; the report retains those choices and does not label them release evidence. Runtime tuning, instrumentation and inherited fuzz-control overrides are cleared in child environments. The recorded case and seed select each property explicitly.

The initial10000-trial batch calibrates subsequent batches toward at most30s or half the configured timeout. Growth is capped at fourfold and ten million trials. Each batch runs in a separate process with a fresh successive seed per target. The overall session time budget defaults to12h and cannot be increased above it. The runner stops starting batches when the remaining budget cannot cover a full child timeout plus5s reserve. Setup, evidence verification and process cleanup can extend wall time; this is a cooperative work budget, not a hard external process deadline. Resume may change only that session budget.

An exclusive process lock prevents simultaneous writers. Checkpoints use atomic replacement and retain the original child report, log and counters for every completed batch. Resuming requires identical source, executable, compiler, dependency-lock, target-catalog and release-policy hashes. It verifies every retained report and attachment, seed sequence and child counter before continuing. Missing, modified, duplicated or unrecorded batches are rejected. Keep the directory in place because child evidence records absolute paths.

PAUSED means the budget ended between complete batches and can resume. PASS means the recorded campaign minima are met. Both return exit0, so automation must inspect status. FAIL and interrupted RUNNING checkpoints cannot resume; investigate their preserved active batch, then start a new campaign if appropriate. A timeout or interrupted child cannot become a passing shorter run or silently disappear from history. A failed resume validation leaves the old report intact.

Campaign completion does not resolve historical findings, run regression or mutation inventories, prove coverage/RSS limits, or approve a release. Those remain separate release gates; no empty finding list or review approval is manufactured by this command. Finish source/docs/locks before final campaigns.

Minimize a captured property failure

tools/dev native-minimize --target request --input path/to/failure.input \
  --attempts 1000 --seconds 300 --timeout 5

This native OCaml tool deletes contiguous ranges while preserving the original property exception class and exact backtrace. It reproduces the original twice and requires two matching replays before accepting each smaller input. A different failure site cannot replace the original finding. Missing failure identity, unexpected process exits, timeouts and inconsistent replays abort with a FAIL report; they never count as a successful reduction. This tool minimizes property exceptions, not process crashes or hangs.

Each run retains original.input, best.input, every probe input/log and a report under _artifacts/native-minimize/run-*/. The report records source/binary/input hashes, failure identity hash, budgets and probe outcomes. Keep tracked sources and the binary unchanged during minimization. Inputs are limited to 64 KiB, attempts to 10,000, total requested time to one day, and each replay to one hour. Defaults are 1,000 attempts, 300 seconds overall and 5 seconds per replay. Process cleanup can add time after a timeout; there is no total RSS or log-size guarantee.

A completed PASS establishes that no single byte deletion preserves that failure under this replay oracle. It does not establish the globally smallest input or explain the defect. Exhausting the attempt/time budget between probes retains the best twice-reproduced input as BUDGET_EXHAUSTED, with minimality explicitly false. Timing out inside a replay is inconclusive and produces FAIL. The command prints the outcome and report path; callers must inspect the report rather than treating a zero exit alone as completed minimization. Add the reduced input as a permanent regression test and keep the original evidence before fixing the defect.