AI SDLC production readiness checklist

Use this checklist when an AI-enabled system is leaving prototype mode. It focuses on delivery controls that make AI systems easier to release, observe, operate, and improve.

1. Product and ownership readiness

• The problem statement is specific enough to evaluate.
• The user journey or workflow step is documented.
• The business owner, technical owner, and operational owner are named.
• The team knows which failures are acceptable and which are not.
• Success metrics include quality and operational behavior, not only adoption.

2. Architecture readiness

• The system boundary is clear: model, prompt, retrieval, tools, data stores, APIs, and user interface.
• The model or vendor dependency is documented.
• Data access is least-privilege and traceable.
• Prompt, retrieval, and tool-calling behavior can be versioned.
• The design supports rollback or fallback if the AI component fails.

3. Evaluation readiness

• The team has representative test cases.
• Expected behavior and unacceptable behavior are written down.
• Evaluation covers correctness, relevance, safety, robustness, cost, and latency where applicable.
• Human review is used where automated scoring is insufficient.
• Evaluation results are stored with the release version.

4. Security and governance readiness

• Input, output, and data-retention risks are reviewed.
• The system is tested for prompt injection, data leakage, over-permissioned tools, and unsafe output paths.
• User permissions carry through retrieval and tool access.
• Sensitive data handling is explicit.
• The release decision records residual risk.

5. Release readiness

• The release has a named owner and deployment checklist.
• Monitoring is active before broad rollout.
• Rate limits, budget controls, and abuse controls are defined where needed.
• Support teams know how to triage incidents.
• A rollback or kill-switch path exists.

6. Production monitoring

• Usage, quality, cost, latency, error rate, and escalation signals are reviewed.
• Feedback loops are connected to product and model improvement.
• Changes to prompts, retrieval, tools, or model versions are tracked.
• Incidents trigger learning, not only repair.
• Drift in user behavior or data behavior is reviewed periodically.

7. Questions to ask before launch

• Can we explain what changed between the prototype and production version?
• Can we reproduce the evaluation evidence for this release?
• Can users tell when AI is involved where that matters?
• What is the support path when the system produces a harmful or wrong output?
• What is the smallest safe rollout population?

Practical scoring

Use a green, amber, red score for each section. Do not block low-risk internal tools with unnecessary ceremony, but do not let high-impact systems ship without evidence, ownership, and monitoring.