Skip to content

Split the functional test workflows by trust level - #278

Merged
emgarten merged 3 commits into
mainfrom
emgarten-ci-workflow-review
Oct 13, 2026
Merged

emgarten merged 3 commits into
mainfrom
emgarten-ci-workflow-review

Conversation

@emgarten

@emgarten emgarten commented Oct 13, 2026 •

Copy link
Copy Markdown
Owner

The emulator functional tests don't use any secrets, but they only ran automatically for pull requests from this repository. Forks and Dependabot needed a second workflow with the opposite condition. The cloud tests need the repository secrets, so they must run only for trusted code.

This splits the functional workflows by what they need instead of by who opened the pull request.

Changes

  • functional.yml runs only the emulator tests, for pushes to main and for every pull request, including forks and Dependabot. New pushes to a pull request cancel the older run, the same as dotnet.yml.
  • functional-cloud.yml (new) runs the cloud tests for pushes to main and for pull requests from this repository. The job is skipped for forks, which GitHub doesn't give the secrets to, and for pull requests that Dependabot opens. GitHub does give the secrets to a run on a Dependabot pull request when someone else starts it, for example by pushing a commit, so the job checks the pull request author as well as the actor.
    • After reviewing one of those pull requests, a maintainer can run the cloud tests for it from main:
      gh workflow run functional-cloud.yml -f pr_number=<number> -f sha=<full SHA of the reviewed commit>
      The run checks out the head of the pull request. Before it runs anything from the pull request, it checks that the head is exactly the reviewed commit. If someone pushes after the review, the run fails instead of using the secrets with unreviewed code, and it doesn't upload anything from that commit. GitHub runs the copy of the workflow on the branch the run uses, so the docs say to keep the default branch.
    • Runs wait for each other instead of being cancelled, because a cancelled run can leave sleet-test-* containers and buckets behind in the test accounts.
  • Removes functional-emulators.yml and functional-manual.yml, which these two workflows replace.
  • dotnet.yml runs the three builds as one matrix job. The check names (build-linux, build-mac, build-win) and artifact names are unchanged.
  • Updates CONTRIBUTING.md, the CI section of local-env/README.md, Sleet.slnx, and adds a cloud tests badge to README.md. The badge only counts push runs, so failed manual runs for pull requests don't affect it.

Testing

  • actionlint, including shellcheck, passes on all workflows.
  • Ran the reviewed commit check locally. Only the exact full SHA passes. A short prefix, a different commit, or an empty value fails.
  • build.ps1 passes.
  • This pull request runs functional.yml, functional-cloud.yml, and the dotnet.yml matrix, and they pass. The manual run can't be tested until the workflow is on main, because GitHub only starts workflow_dispatch runs for workflows on the default branch.

- functional.yml runs the emulator tests for every pull request, including
  pull requests from forks and Dependabot, because they don't use secrets.
- functional-cloud.yml runs the cloud tests for pushes to main and for pull
  requests from this repository. A maintainer can run it for other pull
  requests after review, with the full SHA of the reviewed commit. The run
  stops before running pull request code if the head is a different commit.
- Remove functional-emulators.yml and functional-manual.yml, which the two
  workflows above replace.
- Run the dotnet.yml builds as one matrix job with the same check names.
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 13, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-13T02:11:32.060009Z 6eaccd9 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

…s after a failed check

A run on a Dependabot branch that someone else starts, for example by pushing a commit, gets the repository secrets, so check the pull request author as well as the actor. A manual run that fails the reviewed-commit check no longer uploads files from the unreviewed commit.
GitHub runs the copy of the workflow on the branch the run uses, so a run from another branch might not check the reviewed commit.
@emgarten
emgarten merged commit 4ca911c into main Oct 13, 2026
6 checks passed
@emgarten
emgarten deleted the emgarten-ci-workflow-review branch October 13, 2026 13:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant