This repository contains a lot of material. This guide helps you navigate it based on who you are and what you're trying to accomplish.
Each path is designed around the five core competencies — but different roles emphasize different competencies. Choose the path that fits your situation, or build your own by combining elements from multiple paths.
| Path | Who It's For | Time Investment | Primary Competencies |
|---|---|---|---|
| Foundation Builder | New to AI/LLMs, want solid fundamentals | 2-3 weeks | Systems Design, Evaluation |
| Production Engineer | Building AI into products, need reliability | 3-4 weeks | Safety, Evaluation, Governance |
| AI Systems Architect | Designing AI infrastructure, leading teams | 4-6 weeks | All five competencies |
| AI-Augmented Developer | Using AI tools daily, want to use them well | 1-2 weeks | Output Review, Evaluation |
| Technical Leader | Making decisions about AI adoption | 2-3 weeks | Governance, Evaluation, Systems Design |
| AI for Science Practitioner | Building AI for research/discovery | 6-8 weeks | All five, science-specific |
You're relatively new to working with LLMs and AI systems. You may have experimented with ChatGPT or Copilot, but you haven't built production systems. You want to understand how these systems actually work — not just how to use them — so you can build on a solid foundation.
- How LLMs work (enough to make good decisions, not enough to train one)
- Core prompting patterns that transfer across models
- How to think about AI as a system component
- How to evaluate whether your AI integration is working
- The vocabulary to discuss AI systems with colleagues
| Competency | Emphasis |
|---|---|
| Systems Design | ●●○○○ |
| Evaluation | ●●○○○ |
| Safety | ●○○○○ |
| Output Review | ●●○○○ |
| Governance | ●○○○○ |
Week 1: Understanding the Landscape
- Introduction — Timeline and context for how we got here
- Deep Learning Guide — How LLMs actually work (concepts, not math)
- AI Glossary — Reference as you go, don't memorize
Week 2: Core Techniques
- Basic Prompting — Foundational patterns
- Intermediate Prompting — Reasoning techniques
- Advanced Prompting — Agentic patterns and composition
Week 3: Putting It Together
- Multi-Modal Prompting — Working with images and other modalities
- AI Tools — Survey the landscape of available tools
- Project: RAG Evaluation Pipeline — Your first hands-on build
You can:
- Explain how an LLM generates text (conceptually)
- Choose appropriate prompting techniques for different tasks
- Identify when a task is well-suited for LLMs vs. when it isn't
- Set up basic evaluation for an AI feature
- Have informed conversations about AI system design
After completing this path, most people move to either Production Engineer (if building products) or AI-Augmented Developer (if using AI tools daily).
You're building AI features into production systems. You've moved past experimentation — now you need things to work reliably, fail gracefully, and be defensible when leadership asks questions. You care about uptime, costs, and user trust.
- How to design AI systems that fail safely
- How to build evaluation pipelines that catch problems before users do
- How to implement guardrails that actually constrain behavior
- How to monitor AI systems in production
- How to respond when things go wrong
- How to communicate technical decisions to stakeholders
| Competency | Emphasis |
|---|---|
| Systems Design | ●●●○○ |
| Evaluation | ●●●●○ |
| Safety | ●●●●○ |
| Output Review | ●●○○○ |
| Governance | ●●●○○ |
Week 1: Foundations (Review or Complete)
- Advanced Prompting — Ensure you understand agentic patterns
- Agents — Agent architectures and orchestration
- Competencies — Understand the framework
Week 2: Evaluation & Measurement
- Project: RAG Evaluation Pipeline — Build evaluation infrastructure
- Pattern: Evaluation-First Design
- Template: Eval Suite Template
Week 3: Safety & Reliability
- Project: Agent with Guardrails — Build systems that fail safely
- Pattern: Graceful Degradation
- Pattern: Human-in-the-Loop
Week 4: Operations & Governance
- Project: Incident Response Simulation — Practice failure response
- Template: Incident Runbook
- Template: Architecture Decision Record
You can:
- Design an AI system with explicit failure modes and fallbacks
- Build an evaluation pipeline that runs automatically
- Implement guardrails that constrain agent behavior
- Write a runbook for operating an AI system
- Respond to an AI system incident with a structured process
- Document architectural decisions so they're defensible later
By completing this path, you'll have:
- A working evaluation suite with defined metrics
- An agent with implemented guardrails and monitoring
- An incident runbook and post-mortem document
- Architecture Decision Records for your design choices
You're responsible for AI infrastructure decisions at your organization. You're not just building features — you're designing systems, setting standards, and helping teams adopt AI responsibly. You need deep competency across all five areas, plus the ability to teach others.
- How to design AI architectures that scale and evolve
- How to establish evaluation standards across teams
- How to build safety into organizational practice
- How to create governance frameworks that work
- How to communicate AI decisions to executives and regulators
- How to build AI for high-stakes domains (science, health, finance)
| Competency | Emphasis |
|---|---|
| Systems Design | ●●●●● |
| Evaluation | ●●●●● |
| Safety | ●●●●○ |
| Output Review | ●●●○○ |
| Governance | ●●●●● |
Weeks 1-2: Core Projects
Complete all five core projects in order:
- RAG Evaluation Pipeline
- Agent with Guardrails
- AI Code Review Exercise
- Cost-Benefit Analysis
- Incident Response Simulation
Weeks 3-4: Patterns & Templates
Study and adapt all patterns and templates:
- Patterns Library — Understand each pattern deeply
- Templates Library — Customize for your organization
Weeks 5-6: AI for Science (High-Stakes Domains)
Complete at least one AI for Science project:
- Health Reasoning Agent — Safety-critical AI
- Drug Discovery Pipeline — Multi-agent scientific reasoning
- Scientific Event Engine — Event-driven discovery
Ongoing: Case Studies & Leadership
- Review Case Studies — Learn from real scenarios
- Develop your own case studies from organizational experience
- Create learning paths for your teams
You can:
- Design AI architectures for complex, multi-system environments
- Establish evaluation standards and governance frameworks
- Make build/buy/partner decisions for AI capabilities
- Communicate AI strategy to executives and board members
- Navigate regulatory requirements (EU AI Act, HIPAA, etc.)
- Mentor others on responsible AI development
- Adapt practices for high-stakes domains
By completing this path, you'll have:
- All artifacts from core projects
- At least one AI for Science project artifact
- Organizational templates adapted for your context
- A governance framework document
- Case studies from your experience
You use AI coding assistants (Copilot, Claude, Cursor) daily. You're productive with them, but you've also noticed they sometimes generate plausible-looking code that's subtly wrong. You want to use these tools more effectively — getting their benefits while catching their failures.
- How to review AI-generated code critically
- Common failure patterns in AI-generated code
- When to trust, verify, or reject AI suggestions
- How to maintain expertise while using AI assistance
- How to stay productive without sacrificing quality
| Competency | Emphasis |
|---|---|
| Systems Design | ●○○○○ |
| Evaluation | ●●○○○ |
| Safety | ●○○○○ |
| Output Review | ●●●●● |
| Governance | ●○○○○ |
Week 1: Understanding the Tool
- Advanced Prompting — How to guide AI effectively
- Agents — How AI coding tools work (conceptually)
- Competencies — Focus on "AI Output Review & Oversight"
Week 2: Building Judgment
- Project: AI Code Review Exercise — Practice critical review
- Template: AI Review Checklist — Your personal checklist
You can:
- Identify common AI code generation failure patterns
- Review AI-generated code systematically
- Know when AI suggestions need verification vs. when they're safe
- Maintain deep understanding of your codebase despite AI assistance
- Explain your AI tool usage practices to others
By completing this path, you'll have:
- Annotated code reviews with reasoning
- A personal AI code review checklist
- Documentation of failure patterns you've encountered
You're a tech lead, engineering manager, or CTO making decisions about AI adoption. You need to understand AI systems well enough to evaluate proposals, allocate resources, and communicate with stakeholders — but you're not writing production AI code yourself.
- How to evaluate AI feature proposals
- How to assess costs, risks, and benefits
- How to communicate AI decisions to non-technical stakeholders
- How to build governance frameworks
- How to identify when AI is the right solution vs. when it isn't
| Competency | Emphasis |
|---|---|
| Systems Design | ●●●○○ |
| Evaluation | ●●●○○ |
| Safety | ●●○○○ |
| Output Review | ●○○○○ |
| Governance | ●●●●● |
Week 1: Understanding the Landscape
- Introduction — Context and history
- Competencies — The framework your teams should develop
- AI Tools — Survey of available tools and frameworks
Week 2: Evaluation & Decision-Making
- Project: Cost-Benefit Analysis — Practice making defensible recommendations
- Template: Architecture Decision Record
- Template: Cost Estimation Worksheet
Week 3: Governance & Communication
- Project: Incident Response Simulation — Understand failure modes
- Template: Stakeholder Brief
- Ethical Charter — Values and principles
You can:
- Evaluate AI feature proposals with appropriate skepticism
- Make defensible build/buy/wait decisions
- Communicate AI decisions to executives and boards
- Establish governance frameworks for your organization
- Ask the right questions when reviewing AI systems
By completing this path, you'll have:
- A cost-benefit analysis document
- An incident response plan
- A governance framework outline
- Templates adapted for your organization
You're building AI systems for scientific discovery — drug discovery, medical reasoning, observatory coordination, or other research applications. The stakes are higher than typical software: wrong answers waste research effort, mislead scientists, or harm patients. You need all five competencies, plus domain-specific practices.
- How to build AI systems where correctness matters more than speed
- How to design for uncertainty quantification and provenance
- How to implement safety in domains where failure causes real harm
- How to build multi-agent systems for complex scientific reasoning
- How to create audit trails for scientific reproducibility
- How to communicate uncertainty to researchers and clinicians
| Competency | Emphasis |
|---|---|
| Systems Design | ●●●●● |
| Evaluation | ●●●●● |
| Safety | ●●●●● |
| Output Review | ●●●●○ |
| Governance | ●●●●● |
Weeks 1-3: Core Foundation
Complete all five core projects — the fundamentals apply everywhere:
- RAG Evaluation Pipeline
- Agent with Guardrails
- AI Code Review Exercise
- Cost-Benefit Analysis
- Incident Response Simulation
Weeks 4-5: AI for Science Overview
- AI for Science Overview — Why science is different
- Study the patterns with a science lens:
- Provenance and reproducibility
- Uncertainty quantification
- Multi-agent coordination
- Human expert oversight
Weeks 6-8: Domain Projects
Complete the AI for Science projects relevant to your domain:
For Health/Medical:
For Drug Discovery/Chemistry:
For Physical Sciences/Observatories:
You can:
- Design AI systems that quantify and communicate uncertainty
- Build provenance tracking into scientific AI pipelines
- Implement safety constraints appropriate to your domain
- Create multi-agent systems with appropriate coordination
- Audit and explain AI-assisted scientific conclusions
- Navigate domain-specific regulatory requirements
By completing this path, you'll have:
- All core project artifacts
- At least one complete AI for Science project
- Domain-specific templates and runbooks
- A governance framework for scientific AI
The predefined paths above are starting points, not constraints. You can build your own path by:
For each competency, honestly evaluate where you are:
| Level | What It Looks Like |
|---|---|
| Novice | Haven't thought about this systematically |
| Developing | Aware of the concepts, limited practice |
| Competent | Can apply in familiar situations |
| Advanced | Can adapt to novel situations, teach others |
Which competencies matter most for your role? Where are your biggest gaps between current state and where you need to be?
Choose materials that target your gaps:
- Foundations — For conceptual understanding
- Projects — For hands-on practice
- Patterns — For reusable approaches
- Templates — For practical tools
Be realistic. Deep competency takes time:
- A single project: 3-5 days of focused work
- A full learning path: 2-8 weeks depending on scope
- Mastery: Ongoing practice over months
Each project produces artifacts. Collect them. They demonstrate competency better than certificates.
If you're just starting: Pick the path closest to your role and follow it sequentially.
If you're experienced: Skim the paths, identify gaps, and go directly to the materials that address them.
If you're a manager: Use this to identify learning paths for your team members based on their roles and growth areas.
If you're building a training program: Adapt these paths for your organization's context and tools.
For each path, track:
| Milestone | Status | Date Completed | Notes |
|---|---|---|---|
| Foundation materials | ☐ | ||
| Project 1 | ☐ | ||
| Project 2 | ☐ | ||
| Patterns studied | ☐ | ||
| Templates adapted | ☐ | ||
| Portfolio complete | ☐ |
The goal isn't speed — it's genuine competency development.
Ready to begin?
- Choose your path from the options above
- Review the Competencies framework
- Start with the first material in your path
- Build the portfolio artifacts as you go
Questions about which path to choose? Start with Foundation Builder — the fundamentals transfer everywhere.
This guide is a living document. As new projects and patterns are added, learning paths will be updated. Contributions welcome.