Skip to content

[SDK Sentinel] Stabilize local activity shutdown test - #2766

Open
sdk-sentinel-bot wants to merge 1 commit into
temporalio:mainfrom
sdk-sentinel-forks:automation/sdk-sentinel/ci-flake/go/38007672126-1-c93a04ebedf1
Open

sdk-sentinel-bot wants to merge 1 commit into
temporalio:mainfrom
sdk-sentinel-forks:automation/sdk-sentinel/ci-flake/go/38007672126-1-c93a04ebedf1

Conversation

@sdk-sentinel-bot

@sdk-sentinel-bot sdk-sentinel-bot commented Oct 10, 2026 •

Copy link
Copy Markdown
Contributor

Caution

This PR contains untrusted AI-generated code. Do not approve or run CI until a maintainer has reviewed the diff. SDK Sentinel verified that GitHub Actions remained approval-gated with zero executable jobs when this PR was opened.

Summary

The local-activity shutdown integration test intermittently stopped its worker before any workflow task started. This appeared in both [macOS embedded-server CI](https://github.com/temporalio/sdk-go/actions/runs/37868249424/job/113620043209) and [Linux cloud CI](https://github.com/temporalio/sdk-go/actions/runs/37959626170/job/113918841431). A [preceding main run](https://github.com/temporalio/sdk-go/actions/runs/37847881040) was successful before the [earliest captured failure](https://github.com/temporalio/sdk-go/actions/runs/37868249424).

Root cause

ExecuteWorkflow returns before the server schedules work and the worker begins the local activity. The test waited a fixed 100 ms before stopping the worker, so slower scheduling could produce zero started workflow tasks and test setup no longer exercised the intended shutdown behavior.

Fix

Signal when the first local activity starts and wait for that signal before stopping the worker. This establishes the required ordering without weakening the history assertions.

Validation

The focused test passed six times with default caching and once with WORKFLOW_CACHE_SIZE=0. One pre-change attempt passed without reproducing the flake. The fixed repository validator repeated the unchanged baseline's unrelated doclink collision, so macOS, cloud, and check CI remain required.

Validation status: validation-incomplete

  • Flake confidence: high
  • Fix confidence: high
  • Value: high
  • Patch scope: standard — 1 changed file; 1.2 KiB
  • Local reproduction: not-reproduced — The unmodified focused integration test passed once; CI supplied two identical failures across Linux/cloud and macOS/embedded-server modes.
  • Regression coverage: pass-after-only — The existing focused integration test passed six times after synchronization and once with workflow caching disabled.
  • Unchanged baseline (go run . check) — did not pass (failure)
  • Independent candidate (go run . check) — did not pass (failure)

Investigator-run checks

  • cd internal/cmd/build && go run . integration-test -dev-server -run "TestIntegrationSuite/TestLocalActivityWorkerShutdownNoHeartbeat" — passed; 7 attempt(s). One pre-change attempt passed without reproduction; six post-change attempts passed.
  • cd internal/cmd/build && WORKFLOW_CACHE_SIZE=0 go run . integration-test -dev-server -run "TestIntegrationSuite/TestLocalActivityWorkerShutdownNoHeartbeat" — passed; 1 attempt(s). The post-change focused test passed with workflow caching disabled.
  • .ci-flake-runtime/input/validate.sh — failed; 1 attempt(s). Failed at the same pre-existing doclink collision as the trusted unchanged baseline: package gax contains duplicate Version keys.
  • git diff --check — passed; 1 attempt(s). The candidate diff has no whitespace errors.

Required target CI

  • CI / cloud-test (stable, false) on Linux
  • CI / integ-test (macos-intel, oldstable) on macOS
  • CI / check after the pre-existing doclink collision is resolved

Residual risks

  • The intermittent failure did not reproduce locally before the edit.
  • Local Linux dev-server tests do not reproduce cloud or macOS scheduling.
  • Full repository validation remains blocked by the unchanged baseline's doclink failure.

Automation provenance

If this finding should not be fixed, apply the sdk-sentinel:false-positive label and close the PR. Sentinel will suppress the finding until its affected source changes.

SDK Sentinel recurrence history

  • Unique failed CI run attempts: 2
  • First occurrence: 2026-10-09 01:21 UTC
  • Latest occurrence: 2026-10-09 16:47 UTC
  • Recent occurrences:

@sdk-sentinel-bot
sdk-sentinel-bot requested a review from a team as a code owner October 10, 2026 00:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant