Risk & Reliability Agent — Engineering Guide

InfinitySDLC Engineering Guides · 10/12

Reference implementation guide, not a report of a completed client deployment. Code, configurations, metrics and policies are illustrative. Adapt and validate them before production use.

The reliability agent quantifies operational risk before and after change. It combines SLOs, error budgets, dependency graphs, capacity, failure history and resilience evidence into explicit risk decisions.

10.1 Reliability knowledge model

10.2 Deterministic risk features + model reasoning

Compute numeric features outside the LLM: change size, number of critical services touched, current burn rate, recent incident rate, dependency centrality, test coverage delta, restore-test age and rollout reversibility. The agent then explains the score and identifies missing controls. This hybrid pattern is more auditable than asking the model for a 1–10 risk score from prose.

risk_score = (
  0.20 * criticality +
  0.20 * change_surface +
  0.15 * error_budget_pressure +
  0.15 * dependency_centrality +
  0.10 * recent_incident_rate +
  0.10 * rollback_complexity +
  0.10 * validation_gap
)

10.3 Agent decisions

• Recommend canary percentage and observation duration.
• Require additional load/restore/failover evidence for high-risk changes.
• Detect hidden correlated dependencies from topology and incident history.
• Identify SLOs at risk and explain business impact.
• Generate reliability review comments and proposed engineering tasks.
• Never waive an explicit SLO or policy gate autonomously.

10.4 Chaos and resilience testing

Let the agent design experiments, but use a deterministic chaos controller with environment allowlists, blast-radius limits and abort thresholds. Production chaos requires explicit approval. Feed experiment evidence back into the reliability knowledge model so the agent does not repeatedly recommend tests already performed recently.

Implementation Blueprint: Quantified Inputs, Explainable Decisions

Feature store

• Change: symbols/services touched, schema migrations, config/infra/auth/payment risk tags, dependency fan-out.
• Runtime: current SLO burn, saturation, capacity headroom and recent deployment state.
• History: incidents by service/change type, rollback frequency and flaky-test rates.
• Resilience: restore/failover test age, multi-zone status, fallback coverage and RTO/RPO evidence.
• Operational context: maintenance window and on-call/owner coverage for high-risk changes.

Compute feature values deterministically and store them with the decision. The model explains the risk and can request missing evidence, but it does not invent the numeric inputs.

Decision schema

decision: CONDITIONAL
risk_score: 0.71
blocking_controls:
  - restore test older than 90d for payments-db
  - error budget remaining only 12%
required_before_release:
  - restore rehearsal in staging
  - max canary 5% for 30m
  - explicit SRE approval
evidence: [slo:payments:30d, resilience:restore:2026-05-14, change:PR-8821]

Source and Shared Prerequisites

Adapted from the September 2026 Enterprise AI Agent Mesh handbook, Article 10 and Blueprint 10. The Enterprise Agent Platform Foundation guide provides the shared identity, MCP, retrieval, sandbox, audit and evaluation design, plus the source handbook’s further-reading list. Validate model, protocol and tool versions before production use.

Explore the Series

Previous guide · Series foundation · Next guide