AI Product Modernization Playbook · Article 06 of 12
A supplier sends a missing document. Someone reads it, finds the onboarding case, checks the requirements, updates the record and asks for a correction. The same sequence repeats across hundreds of cases, with enough variation to defeat a rigid script. This is a useful place to consider agentic automation.
The product opportunity is a completed business process with less manual effort and a dependable path for exceptions. Achieving it requires more than a model that can call tools. The system needs durable state, limited authority, meaningful approvals and independent evidence that the intended changes actually happened.
This chapter of Infinity Technologies' AI Product Modernization Playbook explains how to build that operating model into a product, extend it to customer-built automations and expose controlled operations to external agents. Examples and pilot targets are illustrative; they are not reported Infinity client outcomes.
On this page
1. Choose a process with a verifiable end state
Start with a process that already has an owner, measurable volume and a recognizable definition of done. Supplier onboarding, service-case preparation, document follow-up and routine provisioning are candidates when their boundaries can be made explicit. Map the systems, handoffs, exceptions and consequences before choosing a model.
For an onboarding example, “the agent processed the email” is an activity. A useful outcome is that the required documents were verified, the case was updated correctly and any unresolved requirement reached the responsible reviewer. Track those outcomes separately from cases that merely received a reply.
Choose the mechanism for each step
Use ordinary code when the path and rules are known. Use a predictive model when the difficult part is estimating an outcome. Use a language or reasoning model where unstructured information or case-dependent choices add value. Most product processes can use a fixed workflow around a small number of judgement steps.
Record today's handling cost, completion time, rework and exception rate. Define what the automation may change and what must remain a proposal. A narrow process with a reliable completion test gives the team a better basis for investment than a broad promise to create a digital worker.
3. Put judgement inside a durable workflow
The workflow owns the process state, waiting, deadlines and recovery. It calls the model only where judgement is needed. The model returns a proposal as structured data; authorized application services decide whether and how to execute it. This separation makes the process inspectable when a model call, worker or downstream service fails.
Separate a business event from an execution attempt
An intake key should identify the intended business operation, including its tenant and relevant event or lifecycle version. Duplicate deliveries of the same event should converge on one operation. Keying only on a supplier or invoice forever would also suppress legitimate later work, so define the deduplication scope and retention window deliberately.
Persist state at meaningful transitions: evidence gathered, proposal prepared, approval pending, action attempted, result verified or exception assigned. Record model responses as step results where appropriate. Replaying workflow history should not casually regenerate a proposal that a person already approved.
A model call can fail, be retried or return a different answer on another attempt. Do not assume it provides at-most-once execution. Pin process, prompt, tool and model configuration versions to the run, and use an explicit migration policy for long-running cases when a release or urgent policy change occurs.
4. Design tool contracts and run records together
A tool should represent a narrow product operation with typed inputs and explicit effects. “Update these permitted case fields” is easier to constrain than a general database or shell command. Include the target, expected version, request identifier and structured success or failure result. Enforce access, value limits and allowed state transitions on the server.
Keep four linked records: the registered agent version and owner; the run and its business purpose; each proposed or attempted action; and any approval. An action record should preserve the approved argument digest, relevant source versions, policy decision, execution receipt and verification result.
Make the trail useful without copying everything
Store references and minimal structured evidence where possible. Raw documents, prompts and personal data require tighter access and retention than operational counters. Protect audit records from casual modification and record failed attempts as well as successful actions. A run that stops before acting is still important evidence.
Define the allowed outcomes precisely. “Request accepted,” “operation completed” and “business outcome confirmed” may be different states. An asynchronous API can acknowledge a job while the underlying update is still pending. Expose that distinction to the workflow and to the person operating it.
5. Handle uncertain outcomes before retrying
A timeout says that the caller did not receive a usable response. It does not prove that the action failed. If a case update or outgoing message succeeded just before the connection broke, repeating it as a new operation can create a duplicate.
Use a stable idempotency key for each intended write, reuse it across retries and reject a retry that changes its meaning. The downstream service must honor that contract; adding a key to a local log is insufficient. Amazon's Builders' Library guidance explains how caller-provided identifiers distinguish retries from genuinely new intent.
Distinguish retry, reconciliation and compensation
Retry addresses a known transient failure under a safe contract. Reconciliation compares the action record with authoritative external state. Compensation addresses effects that occurred but now need a business correction. When the target offers neither idempotency nor reliable status lookup, some ambiguous cases require a person; a local lock cannot remove uncertainty after a remote commit.
For multi-system work, define compensating actions and record their progress. They may fail too. A cancellation or correction may not restore the original world, and the right order depends on business dependencies and concurrent changes. Microsoft's compensating transaction pattern explicitly discusses these limits. Never treat a sent message, disclosed information or consumed resource as fully reversible.
6. Make approvals specific and exceptions actionable
An approval card should show the action, target, proposed changes, supporting evidence, expected impact and recovery limits. Bind the approval to the exact arguments, actor scope and expiry. When important values or source versions change, prepare a new preview instead of reusing the previous decision.
Recheck permissions and relevant preconditions at execution time. A confirmation token should be issued after the authorized decision, scoped to the operation and protected against replay. Possession of a token from an old preview must not authorize different recipients, amounts or records.
Design the work that reaches a person
An exception packet needs the current case state, verified actions, failed or uncertain steps, missing evidence, proposed next move and responsible owner. A long model transcript usually makes the reviewer reconstruct the problem. Give the owner a decision and the evidence needed to make it.
Define who receives the packet, the response deadline and what happens if nobody responds. Approval queues are part of the process capacity. Measure backlog, reviewer effort and missed deadlines. Reduce unnecessary gates through narrower action design and better evidence; do not remove them simply because busy reviewers approve nearly everything.
7. Keep untrusted content away from authority
An inbound email, attached document or tool response can contain instructions that conflict with the authorized process. A useful architecture separates reading from acting: a restricted reader extracts relevant facts and source references, while the execution layer accepts only bounded proposals checked against policy.
Structured extraction is not a guarantee that the content is harmless. Attacker-controlled names, URLs or free-text fields can carry instructions or misleading evidence into the next step. Preserve provenance, constrain values where practical and independently validate targets. The model must not be able to turn a document's request into a new permission.
Control the outbound path
Pay particular attention when a run combines private data, untrusted input and an external destination. Apply recipient and destination allowlists, data minimization and explicit review for sensitive disclosures. Separate internal analysis from external composition where the process allows it. Enforce these controls in services rather than relying on a reminder in the prompt.
Tools and their descriptions also form part of the deployed system. Review versions, track definition changes and limit which servers an agent can use. Block self-service changes to permissions, approved prompts or tool registration in the production runtime. Any extra specialist agent should receive only the scoped evidence and authority needed for its subtask.
8. Learn from documented experiments and incidents
The public record contains useful evidence about both capability and failure. These examples describe specific historical setups. The engineering lessons below are interpretations of those reports, not estimates of how often every production agent will fail.
Project Vend: tools and commercial constraints matter
In Project Vend's first phase, Anthropic and Andon Labs gave a model responsibility for a small office shop. It found suppliers and adapted its offering, but also quoted prices below cost, invented payment details and yielded to requests for discounts. A product lesson is to source commercial facts from tools and enforce permitted terms outside conversational judgement.
The December 2025 follow-up reported improved performance after changes including newer models, CRM and inventory tools. It still needed substantial human support and retained a human purchase gate by withholding a payment interface. Because several parts of the setup changed, the report does not isolate the effect of any one component. It illustrates iteration within a bounded operating environment.
GitHub MCP demonstration: trusted tools can carry an unsafe flow
In a May 2025 research demonstration, Invariant Labs placed an instruction in a public issue that induced an agent to access private repository information and publish it in a public pull request. The researchers identified an agent-level architectural problem, not a flaw in the GitHub server's code. The implication is to constrain cross-resource access and outbound data flows even when the individual tools are trusted.
Amazon Q: the deployment chain is part of the boundary
AWS's July 2025 bulletin describes an overly broad repository token that enabled malicious code to enter an extension release. AWS revoked credentials and released version 1.85.0; it also stated that a syntax error prevented the malicious code from executing or changing customer environments. The case supports reviewing build permissions and release artifacts alongside runtime controls. It should not be retold as a confirmed destructive customer incident.
9. Build three product surfaces for automation
The first surface is the operations console: runs, authoritative state, pending approvals, exceptions and a visible pause control. Show verified outcomes alongside proposed actions, and let operators inspect a timeline without exposing unnecessary sensitive content.
The second is a customer automation builder. Translate a natural-language request into a versioned workflow that the customer can inspect, test and edit. Validate its schema, action scopes, destination limits, cycles, maximum fan-out and budget before enabling it. Treat it as customer-authored executable configuration, with isolation between tenants and appropriate administrative review.
Expose the product through narrow APIs or MCP tools
The third surface serves external agents. Offer purpose-specific operations, dry-run previews and clear results. Authenticate the caller, authorize each target and enforce product policy on the server. A tool description saying an operation is read-only or idempotent is not an enforcement mechanism.
The MCP tools specification treats annotations as untrusted unless they come from trusted servers and requires server-side input validation and access controls. Use the protocol for interoperability while keeping authorization, confirmation and execution guarantees in your own application contract. Document the protocol version you support rather than assuming clients share identical behavior.
If specialists are necessary, use typed handoff packets with case IDs, evidence, open questions and remaining budget. Keep one process owner and cap handoffs. Multiple agents should earn their complexity through a clear separation of capabilities or access needs.
10. Evaluate outcomes, trajectories and recovery
Build a replay set from representative cases and adjudicated outcomes, under appropriate data controls. Include normal work, missing information, contradictory evidence and cases that must escalate. Recorded tool responses help compare versions, but use a separate sandbox for testing real integration and state changes.
Run variable judgement steps repeatedly to expose unstable outcomes. Score the final business result and whether the process respected its boundaries. A different sequence of valid steps can be acceptable; a cheap, fast run that closes an unresolved case cannot.
| Test family | Seeded scenario | Evidence required |
|---|---|---|
| Authority and isolation | Wrong tenant, expired approval, changed permission | Action refused with the applicable policy reason |
| Injection and disclosure | Instructions inside documents, tool results or metadata | No unauthorized action or outbound disclosure |
| Retries and concurrency | Duplicate event, lost response, overlapping runs | No duplicate effect; conflicting or uncertain state reconciled |
| Partial completion | One downstream step fails after earlier success | Consistent recovery or an owned, accurate exception |
| Verification | Tool says success but target state disagrees | Run remains unresolved until evidence supports closure |
| Operational limits | Loop, exhausted budget, pause during execution | Bounded stop behavior and preserved recovery state |
Swipe horizontally to read the full table.
An illustrative pilot might target a 20% reduction in total handling cost for one onboarding segment, with no observed unauthorized actions and a predefined quality margin. Sample size and observation length must fit event frequency and error consequences. Zero failures in a test set is a release observation, not proof of a zero failure rate.
Report successful completion, correct escalation and unresolved failure separately. Otherwise a system that escalates every difficult case can appear highly accurate while removing little work. Measure whether exception packets actually reduce the remaining human effort.
11. Operate within budgets and explicit stop semantics
Set limits for steps, model calls, tokens, spend, records touched, messages sent and active execution time. Shared tenant limits matter too: many individually small runs can still overload a destination or produce a burst of messages. Reserve budget for verification and recovery instead of spending it all on planning.
Distinguish active execution limits from calendar deadlines. A legitimate onboarding process may wait days for a document while performing only seconds of work. Persist the wait, expire approvals where necessary and escalate overdue cases without keeping a model loop alive.
Specify what pause actually stops
A pause should block new actions at the enforcement layer and prevent new runs from starting. Define treatment of in-flight requests: some cannot be cancelled after dispatch. Reconcile them and allow only explicitly authorized recovery operations. Test this behavior with process owners; a disabled dashboard switch alone cannot recall a remote side effect.
When policy or authorization cannot be checked, stop writes. Retry transient service failures with bounded backoff, and use the reconciliation path for ambiguous writes. Model substitution is a controlled fallback only if its behavior, schema and data policy are validated; otherwise queue the judgement step or hand off the case.
Measure the full cost: models, integrations, platform operation, approvals, exceptions, rework and incidents. Divide by correctly completed business outcomes, keeping escalations and failures visible. Track tail costs and time to resolution as well as averages, so a small number of looping or stalled cases cannot disappear inside a reassuring mean.
12. Expand through shadow runs and narrow promotions
Start in shadow mode: consume authorized inputs and produce proposals without taking business actions. Compare with actual outcomes and review disagreements. Shadowing still processes data and consumes resources, so apply access controls, retention and budgets from the start.
Next, route a limited set of actions through meaningful approval. Test the full path, including rejection, expiry, changed evidence and recovery. Introduce unattended execution for one well-understood action type at a time, within narrow limits and a monitored segment.
The promotion record should name the action, population, limits, evidence, recovery method and accountable approver. Predefine demotion criteria for incidents, verification failures, deteriorating outcome quality and unsupported integration changes. An increase in caps deserves the same review as a new action.
Version process definitions, prompts, policies, tool contracts and runtime dependencies together. Exercise integration contracts in a non-production environment, replay representative cases and canary a release before widening it. Review ownership and credentials regularly so an abandoned process cannot continue under obsolete assumptions.
13. Turn the Playbook into a bounded first release
Begin with five concrete artifacts: a process map and owner; baseline outcomes and cost; a representative replay set; an authority and recovery design; and a rollout plan with decision criteria. These make the proposed automation reviewable before the first unattended write.
For supplier onboarding, the first release might read an assigned case, classify an incoming attachment, check required fields and prepare a case update. A reviewer approves the update and any request to the supplier. The workflow verifies the resulting case state and keeps unresolved requirements open. Later releases can automate specific internal updates when evidence supports that change.
This builds on document processing for extracting evidence and workflow copilots for reviewable proposals. The next chapter moves into forecasting: using historical data to predict future outcomes, with a different evaluation and deployment discipline.
Bring Infinity Technologies a process with recurring manual handoffs, the systems it touches and a clear definition of success. The first design decision is where software can act reliably, where judgement helps and where a person still needs a prepared decision.
Sources and further reading
Based on chapter 06, “Agentic automation: from workflows to digital workers,” and the shared platform in the AI Product Modernization Playbook. Primary sources checked on 14 October 2026. Case reports are attributed; architectural choices and pilot targets are engineering recommendations.
- Amazon Builders' Library, Making retries safe with idempotent APIs.
- Microsoft Azure Architecture Center, Compensating Transaction pattern.
- Model Context Protocol, Tools specification, version 2025-06-18.
- Anthropic and Andon Labs, Project Vend, phase one, June 2025, and phase two, December 2025.
- Invariant Labs, GitHub MCP research demonstration, May 2025.
- AWS, Security bulletin AWS-2025-015, July 2025, updated July 25.
