Add EC2 capacity provider observability sample - #1945
Conversation
A Strands agent on an AWS-managed EC2 instance (Instances compute type / capacity provider), behind an AgentCore Gateway, with end-to-end OpenTelemetry: the client's W3C traceparent becomes the parent of the agent's spans, so a single trace spans client -> Gateway -> runtime -> EC2 -> Bedrock, and gen_ai.* usage lands in the GenAI Observability screen. This sits between two existing samples: 08-http-ec2-capacity-provider (the CP, no observability) and 04-observability-with-strands (observability, serverless). It is the intersection: observability OF a capacity provider. What it sets up (deploy.py, idempotent): three IAM roles, a private subnet with no public IP and a zero-ingress SG, the 7 VPC endpoints a private subnet needs, the image (built on CodeBuild, no local Docker), the CapacityProvider, the runtime, the Gateway, and Transaction Search. invoke.py exercises both the Gateway and the direct path and verifies the trace survived; cleanup.py removes everything in the order the service requires. Follows the repo conventions: same file layout as the neighbouring samples, image built via CodeBuild with the Dockerfile inlined in deploy.py (no versioned Dockerfile), plain public boto3 (>= 1.43.66). Validated end to end in us-east-2 and us-west-2 on a real account: cold start 57-76s, warm 2-3s, every trace matched, EC2 with no public IP (IMDS public-ipv4 -> 404), and the full deploy -> invoke -> cleanup cycle leaves nothing running. The README documents the traps hit in practice, each with the fix: the OTLP region must have a single source (ADOT signs SigV4 with the region parsed from the endpoint, so a baked-in region wins — deploy.py sets it per region on the runtime); every VPC endpoint is load-bearing and a missing xray/bedrock-runtime one fails silently; the Gateway needs protocolType omitted and serverProtocol HTTP; one deployment per VPC (interface endpoints take one subnet per AZ); and instanceProfileArn is omitted so there is no shared fixed-name IAM role. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
Latest scan for commit: Security Scan ResultsScan Metadata
SummaryScanner ResultsThe table below shows findings by scanner, with status based on severity thresholds and dependencies: Column Explanations: Severity Levels (S/C/H/M/L/I):
Other Columns:
Scanner Results:
Severity Thresholds (Thresh Column):
Threshold Source: Values in parentheses indicate where the threshold is configured:
Statistics calculation:
Detailed FindingsShow 3 actionable findingsFinding 1: B310
Description: Code Snippet: Finding 2: B602
Description: Code Snippet: Finding 3: B310
Description: Code Snippet: Report generated by Automated Security Helper (ASH) at 2026-08-13T17:57:51+00:00 |
The workflow lints changed files only, and it was red on every script here. GitHub caps annotations at 10 per step, so the PR page showed 10 of the 25 the same ruff run reports locally. 13 of those 25, in deploy.py, invoke.py and agent/agent.py: - RUF015: a single-element slice picked the main route table, so a VPC without one raised a bare IndexError that said nothing. Now next(..., None) plus an explicit error naming the VPC and why the S3 gateway endpoint needs a main route table to attach to. - BLE001 + S112: except Exception where the intent was a missing file or malformed JSON, narrowed to OSError / json.JSONDecodeError. Narrowing the resource-policy parse clears its S112 as well. - PLW1510: subprocess.run without an explicit check. The run_command tool ignores the exit code on purpose, so check=False states it. - F541 and I001: f-strings with no placeholders, and a missing blank line. Four BLE001 are left on purpose, where the broad catch IS the behaviour: the IMDS must not be able to kill the agent, and a failed EBS write is reported in the response rather than raised. Those get a noqa with the reason. ruff format is a separate commit. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The region fell back to ap-southeast-2, left over from where this sample was first developed. A cleanup that silently targets a region you never deployed to is worse than one that refuses to run: it either deletes something you did not mean to touch or reports nothing to delete while the EC2 instance and its EBS volume keep billing. deploy.py always writes the region, so there is nothing to guess. Refuse without it, the same way resolve_region() in deploy.py refuses to default. Also narrows the state-file read to OSError / json.JSONDecodeError (BLE001). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
deploy.py defaulted BEDROCK_MODEL_ID to au.anthropic.claude-haiku-4-5 and
agent/agent.py to us.anthropic.claude-haiku-4-5 — two different geographic
profiles in the same sample. A geographic prefix (us./eu./apac./au.) only
resolves inside its own geography, so cloning this sample and running
export AWS_REGION=us-east-2 && python deploy.py && python invoke.py
failed for every region outside Australia: the deploy succeeds, the instance
boots, and then every invoke returns 424/500 while CloudWatch shows
ValidationException: The provided model identifier is invalid
Model id: au.anthropic.claude-haiku-4-5-20251001-v1:0
The earlier end-to-end runs missed it because BEDROCK_MODEL_ID was exported in
the shell, so the default was never exercised outside ap-southeast-2.
global.anthropic.claude-haiku-4-5-20251001-v1:0 resolves in every region this
sample can run in, including sa-east-1, where global is the only prefix
available. It is also what the neighbouring 08-http-ec2-capacity-provider
sample uses (global.anthropic.claude-sonnet-4-5). The README table and the
docstring now carry the same value, with the reason.
Verified in us-east-2 after the change: deploy from scratch, image rebuilt on
CodeBuild, invoke through the Gateway and direct, both 57s cold, both traces
matching client to agent, EC2 with no public IP, EBS mounted at /mnt/data.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Five screenshots from a real run, in the sections that explain them: - a single trace, from POST /invocations down to execute_tool whoami, with the agent answering that it is on an m6g.large with an aarch64 kernel, 2 vCPUs and 7.7 GB in us-east-2a. That is the whole point of this sample in one frame - the same trace as a timeline, showing the two event-loop passes the model needed to read a tool result, with the captured prompt and toolUse payload beside it - spans aggregated by capability, where the model span resolves to AWS::Bedrock::Model and gets priced, and a tool span averages 9 ms against 1.25 s for chat - the capacity provider as the console renders it, which doubles as a way to check your own deployment: instance profile shows as "-" (the omission this README explains) and the lifecycle pair caps a forgotten sample at 1 hour - model cost per trace: $0.041 for the run, ~$0.002 per trace The account ID is redacted in the capacity provider screenshot. The intro now says three regions rather than two, which is what was actually measured. Follows the repo layout: an images/ directory next to the sample, like the neighbouring samples under 01-hosting-agents. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
What this adds
A new sample: a Strands agent on an AWS-managed EC2 instance (Instances compute type / capacity provider), behind an AgentCore Gateway, with end-to-end OpenTelemetry. The client's W3C
traceparentbecomes the parent of the agent's spans, so a single trace spans client → Gateway → runtime → EC2 → Bedrock, andgen_ai.*usage lands in the GenAI Observability screen.Path:
01-features/02-host-your-agent/01-runtime/01-hosting-agents/10-ec2-capacity-provider-observability/Why
It sits between two existing samples and fills the gap between them:
08-http-ec2-capacity-provider— the capacity provider, but no observability04-observability-with-strands— observability, but serverlessThis is the intersection: observability of a capacity provider.
What
deploy.pysets up (idempotent)IAM (3 roles) → private subnet with no public IP and a zero-ingress SG → the 7 VPC endpoints a private subnet needs → the image (built on CodeBuild, no local Docker) → CapacityProvider → runtime → Gateway → Transaction Search.
invoke.pyexercises the Gateway and the direct path and verifies the trace survived;cleanup.pyremoves everything in the order the service requires.Conventions followed
deploy.py(no versioned Dockerfile, per repo.gitignore).boto3(≥ 1.43.66) — no vendored SDK.Validation
Validated end to end on a real account in us-east-2 and us-west-2: cold start 57–76 s, warm 2–3 s, every trace matched, EC2 with no public IP (IMDS
public-ipv4→ 404), and the full deploy → invoke → cleanup cycle leaves nothing running.The README documents the traps hit in practice, each with its fix: the OTLP region needs a single source (ADOT signs SigV4 with the region parsed from the endpoint, so a baked-in region wins —
deploy.pysets it per region); every VPC endpoint is load-bearing and a missingxray/bedrock-runtimeone fails silently; the Gateway needsprotocolTypeomitted andserverProtocol=HTTP; one deployment per VPC (interface endpoints take one subnet per AZ); andinstanceProfileArnis omitted so there is no shared fixed-name IAM role.