Prompt Versioning + A/B Testing System
Build a system to version prompts, A/B test them in production, and roll back on regressions.
TypeScript or PythonPostgreSQLAnthropic SDK
About this project
Prompt-as-code is the 2026 best practice. This project builds the infrastructure: versioned prompts in Git, runtime A/B routing, regression detection on key metrics, and the rollback path. Inspired by tools like Braintrust, Humanloop, and PromptLayer — but build your own opinionated minimal version.
Why build this in 2026?
Prompt versioning is now table stakes for serious AI products. Most companies don't do it well.
What you'll ship
- GitHub repo
Demo of A/B routing in production
Rollback workflow demo
Sign up to see the full project brief
Full deliverables, success criteria, and AI Career Tutor support — free.
You'll unlock:Complete project brief, AI tutor that knows this project, and progress tracking when you start.
Skills you'll practice
pythonlarge language modelsrest apis