llms.txt

# AWS DevOps Agent Tools

> Open-source tools for AWS DevOps Agent that extend its capabilities for incident response, root cause analysis, and operational troubleshooting. This repository contains skills, custom agents, and MCP servers that can be used with AWS DevOps Agent, as well as templates for writing your own. Skills follow the open Agent Skills specification; MCP servers follow the Model Context Protocol.

## About This Repository

This is the primary open-source collection of tools for AWS DevOps Agent — the AI-powered operations agent from AWS that automates incident investigation, root cause analysis, and operational tasks. It contains three types of tools:

- **Skills** — domain-specific instructions, decision trees, and runbooks the agent follows during investigations. Uploaded to DevOps Agent as zips.
- **Custom agents** — pre-built agent configurations (a system prompt plus assigned tools and skills) for recurring operational workflows such as reports and operational reviews. Created in the DevOps Agent web app.
- **MCP servers** — Model Context Protocol servers that connect the agent to external systems and data sources, tailored to work alongside the skills and custom agents here. Deployed and registered as endpoints.

Tools can be used with these AWS DevOps Agent types:
- Chat tasks — conversational operational queries and analysis
- Incident RCA — automated root cause analysis during active incidents
- Prevention — proactive operational reviews and best practice assessments

## Available Skills

- [AWS Health Events Skill](skills/aws-health-events/SKILL.md): Retrieves and analyzes AWS Health events (service issues, scheduled changes, account notifications) to identify AWS-side events that correlate with observed operational issues
- [Support Cases Skill](skills/support-cases/SKILL.md): Searches and analyzes AWS Support cases to find historical incidents with similar symptoms, proven remediations, and recurring patterns
- [AWS EKS Operations Review Skill](skills/aws-eks-operations-review/SKILL.md): Comprehensive read-only EKS best-practices review with 288 core checks across 9 pillars (Operations, Resilience, Security, Scalability, Performance, Observability, Networking, Cost, Control Plane), PASS/FAIL/N/A grading, QA gates, and remediation shards — aligned with the AWS EKS Best Practices Guide
- [EKS Upgrade Readiness Skill](skills/eks-upgrade-readiness/SKILL.md): Performs read-only Amazon EKS pre-upgrade readiness assessments aligned with the AWS EKS Best Practices Guide covering infrastructure prerequisites, EKS Upgrade Insights, API deprecations, addon compatibility, data plane inventory, PDB/topology safety, and capacity planning, producing a scored READY / NOT READY / READY WITH WARNINGS verdict
- [RDS Operation Review Skill](skills/rds-operation-review/SKILL.md): Performs comprehensive Amazon RDS and Aurora operational reviews aligned with the AWS Well-Architected Framework covering security, reliability, performance, cost optimization, and backups
- [MSK Operations Skill](skills/msk-operations/SKILL.md): Operates, troubleshoots, and assesses Amazon MSK Provisioned clusters (Standard and Express brokers) — performance issues, consumer lag, storage and EBS problems, rolling restarts and Kafka version upgrades, CloudWatch monitoring and alarms, and Kafka client (producer/consumer) tuning
- [CRM Production Investigation Guidelines Skill](skills/crm-production-investigation-guidelines/SKILL.md): Sample skill demonstrating how to write production investigation guidelines for the Incident Triage agent type, showing application-specific architecture, incident isolation rules, and structured investigation procedures
- [Skip Scheduled Maintenance Skill](skills/skip-scheduled-maintenance/SKILL.md): Sample skill demonstrating how to skip low-priority incidents during a scheduled maintenance window, filtering MEDIUM and LOW severity alarms while preserving escalation for HIGH and CRITICAL incidents
- [Enrich with AWS Security Agent Skill](skills/enrich-with-aws-security-agent/SKILL.md): Queries AWS Security Agent CloudWatch logs to retrieve code-level security findings (file, line number, vulnerability type) during incident investigations with potential security root causes
- [Wiz Security Context Skill](skills/wiz-security-context/SKILL.md): Queries the Wiz MCP server for a resource's security context (vulnerabilities, misconfigurations, secrets, active threats, malware, toxic combinations) to determine whether an operational anomaly is an operational issue or a security incident
- [Service Quota Check Skill](skills/service-quota-check/SKILL.md): Checks AWS service quota utilization during investigations and before provisioning resources, flags quotas at 85%+ utilization, and requests increases via the Service Quotas API or recommends support cases
- [ECS Operation Review Skill](skills/ecs-operation-review/SKILL.md): Performs comprehensive Amazon ECS operations reviews across 6 pillars (Resiliency & HA, Observability, Security, Operations, Performance, Additional Analysis) using read-only AWS APIs, with a 7-day CloudWatch metrics baseline, per-pillar PASS/FAIL/N/A scorecards, recommended alarm thresholds for IDR onboarding, and a prioritized remediation-linked report
- [DMS Operational Review Skill](skills/database-migration-service-expertise/SKILL.md): Conducts AWS Database Migration Service operational reviews with 5-category health scoring, task failure troubleshooting, migration cutover runbooks, version deprecation tracking, and cost optimization
- [Redshift Support Specialist Skill](skills/redshift-support-specialist/SKILL.md): Amazon Redshift domain expertise for query optimization, operational reviews, and cost optimization on provisioned clusters and Serverless workgroups, via the awslabs.redshift-mcp-server MCP server
- [S3 Resiliency Review Skill](skills/storage-s3-resiliency-expertise/SKILL.md): Reviews one or many S3 buckets across nine resiliency, security, and data-protection dimensions using read-only control-plane calls, producing a rated report with prioritized findings and remediation guidance
- [VPC DNS Investigation Skill](skills/aws-vpc-dns-investigation/SKILL.md): Diagnoses VPC DNS resolution failures and validates DNS control-plane changes before they are applied, driving the aws-vpc-dns-diagnostics MCP server to observe live resolution from inside the affected subnet and to simulate a proposed change
- [AWS Routing Skill](skills/aws-routing/SKILL.md): Read-only analysis and troubleshooting of AWS routing and BGP path selection across Cloud WAN, Direct Connect, Transit Gateway, VPC, and VPN — traces the end-to-end path, applies each construct's route-evaluation order, engineers traffic with local-preference communities and AS-path, flags non-deterministic selection, and produces describe/get/list validation commands grounded in public AWS documentation
- [Bedrock Adoption Readiness Skill](skills/bedrock-adoption-readiness/SKILL.md): Assesses an AWS account's readiness to run Amazon Bedrock at production scale across IAM governance, data retention (ZDR), quota and capacity headroom, and operational observability, covering both the standard Bedrock and bedrock-mantle (OpenAI-compatible) surfaces with multi-region discovery
- [Analytics OpenSearch Expertise Skill](skills/analytics-opensearch-expertise/SKILL.md): Performs read-only health assessments of Amazon OpenSearch Service domains through 24 deterministic checks across cluster health, storage and shards, performance, security, and cost optimization, producing a structured findings report with prioritized remediation guidance
- [FSx for Windows SLA Optimizer Skill](skills/storage-fsx-windows-sla-optimizer/SKILL.md): Reviews one or many Amazon FSx for Windows File Server file systems for SLA readiness across seven availability dimensions (deployment type, Active Directory health, throughput and storage sizing, backups, maintenance window, and alarms) using read-only control-plane calls, with usage-pattern trend analysis (peak-aware throughput sizing, weekday/weekend profile, and storage growth projection) that produces a rated report and flags over-provisioned or idle capacity as cost-optimization opportunities
- [AI/ML Access Diagnostics Skill](skills/aiml-access-diagnostics/SKILL.md): Diagnoses IAM and access failures for Amazon Bedrock and SageMaker calls by tracing the authorization chain from caller identity through iam:PassRole, role trust policy, role permissions, resource policies, and SCPs to identify which hop denied the call
- [AI/ML GPU Cluster Evidence, Readiness, and Fault Verdicts Skill](skills/aiml-gpu-training-cluster-investigation/SKILL.md): For SageMaker HyperPod (Slurm and EKS), AWS ParallelCluster, and self-managed GPU clusters: proves hour by hour whether GPU Xid evidence was actually arriving per node, gives each node a replace, reboot, or leave-alone verdict against an explicit evidence bar, labels every cause as proven or hypothesis, and runs a pre-flight readiness check before long runs (Capacity Block or training plan end, extension offerings, spare replacement capacity, recovery settings, EFA security group, idle reserved GPUs)
- [Bedrock Operation Review Skill](skills/bedrock-operation-review/SKILL.md): Performs comprehensive Amazon Bedrock operational reviews aligned with the AWS Well-Architected Framework and Bedrock best practices across five pillars — security, performance, service quotas, cost optimization, and resilience — using control-plane and CloudWatch APIs only (no model invocations or prompt/response content read)
- [AgentCore Observability Setup Skill](skills/agentcore-observability-setup/SKILL.md): Validates and bootstraps Amazon Bedrock AgentCore observability across runtime agents, Memory and Gateway resources, built-in tools, and agents hosted outside the runtime, verifying telemetry wiring via read-only CloudWatch, X-Ray, and AgentCore APIs and prescribing exact remediation for gaps it cannot directly read
- [AWS Backup Coverage Review Skill](skills/aws-backup-coverage-review/SKILL.md): Determines which backup-eligible resources are protected by AWS Backup and which are not across all enabled Regions, using read-only APIs and an independent resource inventory, then evaluates plan frequency and retention, cross-Region and cross-account copies, vault encryption and Vault Lock, and per-Region resource type opt-in through 23 fixed checks
- [AgentCore Operational Review Skill](skills/agentcore-ops-review/SKILL.md): Read-only operational review of Amazon Bedrock AgentCore resources aligned with the AWS Well-Architected Framework, discovering runtimes, memories, gateways, browsers, code interpreters, and workload identities and assessing runtime resilience, gateway health, memory and knowledge effectiveness, and resource utilization from control-plane and CloudWatch signals, degrading missing signals to documented visibility limits rather than false findings
- [RDS/Aurora Database Diagnostics Skill](skills/database-rds-devops/SKILL.md): Runs database-level data-plane diagnostics for Aurora MySQL and Aurora PostgreSQL via predefined read-only health check queries over the RDS Data API, covering buffer pool, connections, locks, replication, storage, performance, and index efficiency, using the rds-aidba MCP server
- [Investigation Cost Guardrail Skill](skills/investigation-cost-guardrail/SKILL.md): Estimates and caps the cost of paid API calls during investigations across all AWS services and native agent tools, enforcing per-investigation budgets, flagging expensive operations, requiring time windows, and cancelling when thresholds are exceeded
- [CloudTrail Cost Optimization Skill](skills/cloudtrail-cost-optimization/SKILL.md): Identifies and quantifies read-only Amazon CloudTrail cost optimization opportunities — duplicate management-event trails, unnecessary Read events, high-volume KMS/RDS Data API noise, overly broad or duplicate data events, CloudTrail Lake ingestion/retention, and S3 lifecycle — producing a severity-ranked report of savings with compliance-aware recommendations
- [Config Cost Optimization Skill](skills/config-cost-optimization/SKILL.md): Identifies and quantifies read-only AWS Config cost optimization opportunities — continuous-vs-daily recording frequency mismatches, over-broad allSupported recording, duplicate global-resource recording across Regions, high-churn configuration-item drivers, redundant rules and conformance packs, and S3 lifecycle — producing a severity-ranked report of savings
- [GuardDuty Cost Optimization Skill](skills/guardduty-cost-optimization/SKILL.md): Identifies and quantifies read-only Amazon GuardDuty cost optimization opportunities by attributing per-protection-plan spend from AWS/GuardDuty CloudWatch usage metrics — high-cost/low-signal plans, the Runtime Monitoring / VPC Flow Log charge offset, S3 Protection and Malware Protection scan volume, free-trial cost projection, and duplicate multi-account coverage — framing every reduction as a cost-vs-security tradeoff
- [SageMaker AI Operational Review Skill](skills/sagemaker-ai-ops-review/SKILL.md): Performs read-only Amazon SageMaker AI operational reviews across eight pillars and twenty checks — security, performance, cost optimization, service quotas, resiliency, operational excellence, sustainability, and Well-Architected best practices — producing severity-ranked findings with one recommendation per High or Medium finding
- [Firehose Operation Review Skill](skills/firehose-operation-review/SKILL.md): Performs read-only Amazon Data Firehose operational reviews against the Well-Architected Framework across seven pillars (Security, Reliability, Performance, Service Quotas, Cost Optimization, Operational Excellence, Sustainability) using Firehose, CloudWatch, and Service Quotas control-plane APIs, producing a severity-ranked report artifact

## Available Custom Agents

- [AWS Health Report](custom-agents/aws-health-report/README.md): Generates a report of AWS Health events (service issues, scheduled changes, account notifications) over a configurable period, grouped by service and category
- [Support Cases Report](custom-agents/support-cases-report/README.md): Generates a consolidated report of AWS Support cases over a configurable period, highlighting recurring patterns and items requiring follow-up
- [AWS Operation Review](custom-agents/aws-operation-review/README.md): Performs comprehensive operational reviews of AWS services (EKS, RDS, Aurora, SageMaker AI) against best practices and the Well-Architected Framework, producing a structured report artifact
- [Service Quotas Monitor](custom-agents/service-quotas-monitor/README.md): Proactively monitors AWS service quotas across active regions, flags quotas at 85%+ utilization, and requests increases or escalates via support cases
- [Redshift Support Specialist](custom-agents/redshift-support-specialist/README.md): Amazon Redshift support agent for query optimization, operational reviews, and cost optimization, paired with the redshift-support-specialist skill
- [Unified Security Cost Optimizer](custom-agents/unified-security-cost-optimizer/README.md): Reduces spend on AWS security and governance services (CloudTrail, Config, GuardDuty) without weakening security posture, routing each service to its cost-optimization skill and producing a consolidated, severity-ranked report with estimated savings and cost-vs-risk tradeoffs

## Available MCP Servers

- [Redshift MCP Server](mcp/aws-redshift-mcp-server/README.md): Serverless (Lambda + API Gateway, SigV4) deployment of the standard awslabs.redshift-mcp-server, giving the agent read access to Redshift via the Redshift Data API
- [RDS AIDBA](mcp/rds-aidba/README.md): Read-only, query-allowlisted diagnostic access to Aurora MySQL and Aurora PostgreSQL clusters via the RDS Data API
- [VPC DNS Diagnostics MCP](mcp/aws-vpc-dns-diagnostics-mcp/README.md): Observes live DNS resolution from inside an affected subnet and simulates proposed DNS control-plane changes before they are applied
- [EKS Node Diagnostics MCP](mcp/aws-eks-node-diagnostics-mcp/README.md): Collects and analyzes diagnostic logs from EKS worker nodes via SSM Automation (kubelet, containerd, CNI, iptables, dmesg, and more) not accessible through the Kubernetes API or CloudWatch
- [ECS Instance Log MCP](mcp/ecs-instance-log-mcp/README.md): Collects and analyzes diagnostic logs from ECS container instances via SSM Automation (ECS agent, Docker/containerd, system logs, networking, GPU) not accessible through the ECS API or CloudWatch
- [EMR Spark Troubleshooting MCP — DevOps Agent Integration](mcp/emr-spark-troubleshooting-mcp-connect/README.md): Registers the AWS-managed Apache Spark Troubleshooting MCP endpoint (SageMaker Unified Studio) with AWS DevOps Agent using SigV4; creates the IAM role and the Service (capability provider) only — no MCP server code is deployed

## Key Concepts

- AWS DevOps Agent skills are structured instruction sets that teach the agent how to investigate specific operational scenarios
- Skills follow the open Agent Skills specification (agentskills.io)
- Skills are uploaded as zip files via the AWS DevOps Agent Operator Web App
- Each skill contains a SKILL.md file with frontmatter metadata and step-by-step instructions
- Skills can reference supplementary documents in a references/ directory
- Custom agents pair a system prompt (SYSTEM_PROMPT.md) with assigned tools and skills, and are created in the DevOps Agent web app to automate recurring workflows
- MCP servers implement the Model Context Protocol to connect the agent to external systems and data sources, and are deployed and registered as endpoints

## Documentation

- [AWS DevOps Agent Product Page](https://aws.amazon.com/devops-agent/)
- [AWS DevOps Agent User Guide](https://docs.aws.amazon.com/devopsagent/latest/userguide/about-aws-devops-agent.html)
- [AWS DevOps Agent Skills Documentation](https://docs.aws.amazon.com/devopsagent/latest/userguide/about-aws-devops-agent-devops-agent-skills.html)
- [AWS DevOps Agent Custom Agents Documentation](https://docs.aws.amazon.com/devopsagent/latest/userguide/working-with-devops-agent-custom-agents-index.html)
- [Connecting MCP Servers to DevOps Agent](https://docs.aws.amazon.com/devopsagent/latest/userguide/configuring-integrations-and-knowledge-connecting-mcp-servers.html)
- [Agent Skills Specification](https://agentskills.io/home)
- [AGENTS.md Specification](https://agents.md/)
- [Model Context Protocol](https://modelcontextprotocol.io)
- [Extend AWS DevOps Agent with Custom Skills](https://builder.aws.com/content/3BDdQAFY2bSmtjecZC7vbOQGSEV/extend-aws-devops-agent-with-custom-skills-for-your-operational-workflows)

## Repository Structure

- skills/ — All skill directories
- skills/<name>/SKILL.md — Main skill instructions (what DevOps Agent reads at runtime)
- skills/<name>/README.md — Human documentation, prerequisites, upload guide
- skills/<name>/references/ — Supplementary reference documents included in the skill
- skills/<name>/evals/ — Hand-written evals.json (eval definitions) plus skill evaluation tool output: structure/, best-practices/, and functional/ results, versioned per run (see CONTRIBUTING.md)
- custom-agents/ — All custom agent directories
- custom-agents/<name>/SYSTEM_PROMPT.md — The agent's system prompt
- custom-agents/<name>/README.md — Purpose, prerequisites, creation and execution guide
- mcp/ — All MCP server directories
- mcp/<name>/README.md — What the server does, its tools, and deployment/registration steps
- cloudformation/ — IAM policy templates that skills require
- docs/ — GitHub Pages (mkdocs) documentation site
- CONTRIBUTING.md — Contribution guidelines
- llms.txt — This structured repository overview for AI tools