Skip to content

feat(cli)!: unify output flags and exit codes across gesso subcommands #507

Description

@wadakatu

Context

bin/gesso dispatches four subcommands (bin/gesso:57-77): doctor, coverage:merge,
coverage:gate, stubs. Counting every flag each one accepts, including --help:

Subcommand Flags Parser
doctor 10 src/Cli/DoctorCommand.php:107-183
coverage:gate 6 src/Cli/CoverageGateCommand.php:100-133
stubs 9 src/Cli/StubsCommand.php:86-124
coverage:merge 22 src/Coverage/CoverageMergeCommand.php:120-179

47 flags across four commands, parsed by four hand-written parseArgv() loops that
disagree on flag names, on what --format means, on whether a format and a destination
are the same argument, on whether an unknown flag is an error, and on what exit code an
unreadable input file produces. Nothing here is a missing feature — it is the same four
concepts spelled four ways.

Current behavior

1. Output format and output destination are the same flag on coverage:merge

--format exists on two of four subcommands and means something different on each:

  • doctor: --format=text|json (src/Cli/DoctorCommand.php:200).
  • coverage:gate: --format=text|markdownno json; the allow-list is
    ['base_spec', 'spec', 'coverage', 'spec_name', 'format'] (src/Cli/CoverageGateCommand.php:123,
    usage at :154-156).
  • stubs: no --format at all. VALUE_OPTIONS is
    ['spec', 'coverage', 'spec_name', 'adapter', 'output', 'namespace', 'base_class']
    (src/Cli/StubsCommand.php:72). Verified: php bin/gesso stubs --spec=… --format=json
    [Gesso] Unknown argument(s): --format, exit 2.
  • coverage:merge: no --format. Instead five destination flags, each hard-wiring one
    format (src/Coverage/CoverageMergeCommand.php:195-204):
    --output-file=<path> (Markdown), --junit-output=<path>, --json-output=<path>,
    --html-output=<path>, --github-step-summary=<path> (Markdown), plus
    --console-output=default|all|uncovered_only, which is a verbosity mode, not a format.

The PHPUnit extension mirrors the same conflation:
output_file / junit_output / json_output / html_output are paths, while
validation_output is a format (text|json, src/ValidationOutputFormat.php:15-18)
and console_output is a mode — all six read as one family
(src/PHPUnit/OpenApiCoverageExtension.php:334-369). The Laravel side adds a third
--format value set: openapi:routes --format=text : Output format: text or json
(src/Laravel/Commands/OpenApiRoutesCommand.php:50).

2. One concept, several spellings

Concept Spellings in the shipped CLI
request-path prefix --strip-prefix repeatable (DoctorCommand.php:138) vs --strip-prefixes=<a,b> (CoverageMergeCommand.php:142-146)
write here --output=<dir> (StubsCommand.php:147) vs --output-file=<path> (CoverageMergeCommand.php:195)
which spec --spec=<path> (doctor/gate/stubs) vs --specs=<a,b> = spec names, not paths (CoverageMergeCommand.php:191) vs --spec-name=<name> (CoverageGateCommand.php:152, StubsCommand.php:143) vs --spec-base-path=<path> (CoverageMergeCommand.php:190)

--spec is a filesystem path in three subcommands; --specs in the fourth is a
comma-separated list of logical names. openapi:stubs --spec= accepts either
(src/Laravel/Commands/OpenApiStubsCommand.php:40).

3. coverage:merge silently accepts unknown flags

doctor, coverage:gate and stubs collect unrecognised names into invalid_options
and exit 2. coverage:merge's parseArgv() has a catch-all default: $opts[$name] = $value;
(src/Coverage/CoverageMergeCommand.php:172-174) — anything unknown is stored and ignored.
Verified on this checkout:

$ php bin/gesso coverage:merge --spec-base-path=tests/fixtures/specs \
    --totally-bogus-flag=1 --sidecar-dir=/tmp/nope ; echo $?
[OpenAPI Coverage] WARNING: no sidecars found in /tmp/nope
0

$ php bin/gesso doctor --totally-bogus ; echo $?
[OpenAPI Doctor] Unknown argument(s): --totally-bogus
2

A typo'd --min-endpont-coverage=90 in a CI workflow disables the gate the user opted
into and exits 0.

4. Exit codes are not comparable across subcommands

  • doctor: EXIT_OK=0 / EXIT_DIAGNOSTIC_FAILURE=1 / EXIT_USAGE=2 (DoctorCommand.php:90-92).
  • coverage:gate: EXIT_OK=0 / EXIT_UNCOVERED_CHANGE=1 / EXIT_USAGE=2 (CoverageGateCommand.php:84-86).
  • stubs: EXIT_OK=0 / EXIT_USAGE=2 only — no 1 (StubsCommand.php:69-70).
  • coverage:merge: bare integers, no constants. 2 for configuration errors
    (:288, :301, :311, :334, :342, :348) and 1 for both gate failures and
    I/O failures (:364, :371, :384, :394, :403, :415, :488, :493, :506, :545).

So "an input file cannot be read" is exit 2 for coverage:gate
(docs/coverage-gate.md:120-126) and exit 1 for coverage:merge (:364, unreadable
sidecar directory). Documentation matches that fragmentation: ## Exit codes sections
exist only in docs/doctor.md:108 and docs/coverage-gate.md:120; docs/stubs.md:50 is
one prose sentence; docs/parallel.md:37-58 tabulates all 22 merge flags and no exit code.

Proposed change

Format is --format. Destination is --output-file. Never the same flag.

  1. --format=<name> on all four subcommands. text and json everywhere;
    coverage:gate keeps markdown; coverage:merge accepts
    text|markdown|json|junit|html. Default stays each command's current default
    (text; markdown for a merge run that passes --output-file, see 3).

  2. --output-file=<path> on all four: where the primary report goes. Absent → stdout.
    stubs --output=<dir> becomes --output-dir=<dir> (it is a directory, not a file),
    with the same rename on openapi:stubs.

  3. coverage:merge collapses --json-output / --html-output / --junit-output /
    --output-file (4 sink flags) into --format + --output-file for the primary
    report, plus one repeatable --report=<format>:<path> for the additional simultaneous
    sinks a merge run needs today. --output-file=r.md alone keeps writing Markdown.
    --github-step-summary stays as-is: it carries $GITHUB_STEP_SUMMARY fallback
    semantics, not a plain sink. --console-output is renamed --console-report,
    keeping its values: it is a verbosity mode, and under
    ADR 0005
    output is reserved for "a file is written". refactor(config)!: collapse duplicated configuration keys into one key per concern #502 renames the matching
    extension parameter to console_report in the same major.
    No output format is removed and no combination becomes unexpressible.

  4. --spec always takes a path; --spec-name always takes a name.
    coverage:merge --specs=<a,b> becomes --spec-name=<name> (repeatable, comma-accepting).
    --strip-prefixes becomes --strip-prefix (repeatable, comma-accepting), matching
    DoctorCommand.php:138-142.

  5. coverage:merge rejects unknown flags with exit 2, like the other three.

  6. One exit-code contract for all four, with named constants on every command class:

    Code Meaning
    0 Success. Warnings may still have been written to stderr.
    1 The command ran and the answer is "no": diagnostic error, uncovered change, threshold miss, baseline violation.
    2 Usage error, an input that cannot be read, or an output that cannot be written.

    coverage:merge's I/O failures move from 1 to 2. stubs keeps 1 unused and
    documents it as reserved.

  7. New docs/cli.md holding the flag matrix and that exit-code table;
    docs/doctor.md, docs/coverage-gate.md, docs/stubs.md and docs/parallel.md link
    to it instead of restating it.

  8. Every renamed flag is still accepted for the whole of v3.x and writes one line to
    stderr: [Gesso] DEPRECATED: --json-output is deprecated; use --report=json:<path>. Removed in v4.0.
    Deprecated spellings are removed in v4.0. No gesso migrate ci command is built —
    the migration is sed.

Compatibility

Breaking. docs/versioning.md:89-93 puts this squarely on the compatibility surface:

  • CLI surfaces by major (commands, flags, exit codes, and versioned inputs and output where applicable):
    • v2.x: the doctor, coverage:merge, coverage:gate, and stubs subcommands of bin/gesso

docs/versioning.md:95 covers the Laravel openapi:stubs rename ("flags, exit codes, and
versioned JSON output"), and :96 covers the PHPUnit extension parameters, which are
not touched by this issue — output_file and json_output are renamed by #502, and
console_outputconsole_report / validation_outputvalidation.format are ADR 0005's,
carried out there rather than here. The command classes themselves
are @internal (docs/versioning.md:165), so parseArgv() signature changes are free;
the CLI surface they implement is not.

Breaks, all deprecated-not-removed in v3.0:

  • coverage:merge: --specs, --strip-prefixes, --json-output, --html-output,
    --junit-output deprecated. Unknown flags now exit 2 instead of being ignored — this is
    the one change that can turn a currently-green CI step red, and it does so only where a
    flag was already having no effect.
  • coverage:merge I/O failure exit code 12. Scripts testing for non-zero are
    unaffected; scripts branching on == 1 change.
  • stubs / openapi:stubs: --output deprecated in favour of --output-dir.
  • coverage:merge: --console-output deprecated in favour of --console-report.
  • coverage:merge gains the collapsed flags refactor(config)!: collapse duplicated configuration keys into one key per concern #502 defines — --min-coverage,
    --baseline, --baseline-stale, --strict-required,
    --strict-additional-properties — replacing nine single-purpose flags. refactor(config)!: collapse duplicated configuration keys into one key per concern #502's
    --report-output is not added; the sinks are --format / --output-file /
    --report from this issue.

Migration:

sed -i '' 's/--json-output=/--report=json:/g; s/--html-output=/--report=html:/g;
           s/--junit-output=/--report=junit:/g; s/--specs=/--spec-name=/g;
           s/--strip-prefixes=/--strip-prefix=/g' .github/workflows/*.yml

Acceptance criteria

  • --format is accepted by all four subcommands; --format=json produces machine-readable output on each, and --format=<garbage> exits 2 on each with the same message shape.
  • --output-file=<path> is accepted by all four; omitting it writes the same bytes to stdout. Covered by a test per subcommand.
  • coverage:merge --output-file=r.md with no --format writes byte-identical Markdown to the v2.4.0 output for the same sidecar set (regression fixture).
  • --report=json:a.json --report=junit:b.xml --report=html:c.html in a single merge run writes all three, matching what --json-output/--junit-output/--html-output write today.
  • Each deprecated spelling (--specs, --strip-prefixes, --json-output, --html-output, --junit-output, --console-output, stubs --output) still works and emits exactly one DEPRECATED line on stderr, asserted per flag. The nine flags refactor(config)!: collapse duplicated configuration keys into one key per concern #502 collapses (--min-*, --baseline-file, --coverage-baseline-*) are deprecated by refactor(config)!: collapse duplicated configuration keys into one key per concern #502, not here.
  • php bin/gesso coverage:merge --spec-base-path=… --totally-bogus-flag=1 exits 2 and names the flag.
  • CoverageMergeCommand exposes EXIT_OK / EXIT_FAILURE / EXIT_USAGE constants and every return 1 / return 2 uses them; a test asserts an unreadable sidecar directory exits 2, and a threshold miss under --min-coverage="endpoint=90,strict" exits 1.
  • docs/cli.md exists with the full flag matrix and the shared exit-code table; docs/doctor.md, docs/coverage-gate.md, docs/stubs.md, docs/parallel.md link to it and no longer contain a divergent copy.
  • gesso <cmd> --help for all four lists the same --format / --output-file pair and an Exit codes: block; docs/parallel.md gains one.
  • composer ci passes, including generated Markdown lint.

Names frozen by ADR 0005 (#520). Where this body and the ADR disagree, the ADR is correct.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions