Track B · Technical AI
From prompts to production.
Level 1 Fundamentals
How LLMs Actually Work Tokens · Attention · Training
Tokens, embeddings, and attention at an intuition level, plus why context, latency, and cost are shaped the way they are.
Prompt Engineering as a Discipline System prompts · Structured output
System vs. user prompts, JSON/XML output, few-shot and chain-of-thought — hitting a target structured output reliably.
Calling the API SDKs · Streaming · Cost control
Anthropic and OpenAI SDKs, streaming, retries, rate limits, and caching — building a CLI tool with proper error handling.
Level 2 Building Real Applications
Tool Use & Function Calling Schemas · Routing · Failure modes
Schema design, parallel tool calls, and failure modes like bad args and hallucinated tools — building an agent that queries a real API.
RAG: Retrieval-Augmented Generation Chunking · Reranking
Vector DBs, hybrid search, and reranking — plus when RAG beats long context and when it doesn't, measured on your own docs.
Agents: Design Patterns ReAct · Reflection · Multi-agent
Plan-and-execute, memory tiers, and browser agents — building an agent that completes a real multi-step task.
Evals: How You Know It Works Eval-driven development
LLM-as-judge, rubrics, and golden sets — writing 30+ evals for your own system and running them in CI.
Level 3 Production
Shipping AI Systems Observability · Guardrails
Tracing, logging, replay, and input/output validation — deploying an instrumented app and fixing a bug in real traffic.
Safety, Security & Cost Threat modeling agentic systems
Prompt injection, data exfiltration, and cost engineering — red-teaming your own agent and patching what breaks.
Fine-Tuning & Distillation SFT · DPO · RLAIF
When fine-tuning is worth it, building a dataset from production traces, and beating the base model on evals.
Level 4 Frontier
Model Internals & Interpretability Features · Circuits · SAEs
Mechanistic interpretability and activation steering — probing a small open model and finding a feature to steer.
Reasoning, RL & Frontier Training o-series · R1 · Claude thinking
What changed with reasoning models, RL from verifiable rewards, and where post-training is heading next.
Capstone Project Demo day · Write-up
Ship a non-trivial system — agent, RAG, evals, deployed, and monitored — with peer review and a lessons-learned write-up.
By the end you can
- Explain LLM behavior to a non-technical stakeholder without lying
- Build a tool-using agent that survives real user input
- Design a RAG pipeline that beats long-context by the numbers
- Write evals before you write features
- Instrument, deploy, and observe an AI system in production
- Threat-model an agent and patch prompt-injection vectors
- Decide when fine-tuning is worth the operational cost
- Read a frontier paper and know which ideas matter for your stack