How to Build AI Chat and Voice Assistants into Your Product

1. Define the job and the outcome

Start with a contact reason that already appears in your support data. Order tracking, invoice explanation, delivery rescheduling and subscription changes have different information needs and different consequences when the system is wrong. Treat each as a separate workflow with an owner, an allowed action set and a definition of success.

For a support-oriented assistant, the Playbook proposes resolution without a repeat contact for the same issue within seven days as the primary outcome. The assistant cannot certify its own success. Compute the outcome from product events and support records, joining follow-up contacts across channels. Mark abandoned conversations separately so that a customer giving up does not count as a successful resolution.

Choose a narrow first release

Select three to five candidate intents by volume, handling effort, data availability and reversibility. Launch one customer-facing intent first. A delivery-status lookup is often easier to constrain than a refund dispute because the first needs an authenticated read while the second changes account state and may require specialist judgement.

Write the feature contract before choosing a model: the user job, primary metric, guardrail metrics, required identity, allowed data, spending limits, fallback, autonomy level and person responsible for release. Keep customer satisfaction, missed escalations, latency and cost beside the primary outcome.

2. Learn from deployed assistants

Klarna shows why integrations matter

In its February 2024 launch report, Klarna said its assistant handled 2.3 million conversations in its first month and that repeat inquiries fell by 25%. The service covered tasks including refunds, returns and payment questions, and customers retained access to live agents. These are company-reported results from that period, not a forecast for another product. Source: Klarna launch release.

The design lesson is an inference: conversational access becomes valuable when it reaches the systems where the customer’s problem can be resolved. A polished interface over disconnected documentation cannot complete the same jobs.

Erica illustrates the value of a defined scope

Bank of America reported in August 2025 that Erica had passed three billion interactions and served nearly 50 million users since launch. The bank described a library of more than 700 responses and continued refinement over years. Its examples included balance insights and appointment scheduling. Source: Bank of America.

This is evidence for sustained product investment and defined workflows. It does not establish that every assistant needs the same model architecture, or that a newly deployed language model inherits the reliability of a mature service.

3. Put the assistant inside a controlled workflow

The conversation service owns identity, session state and the current product context. A router identifies the intent and exposes only the tools needed for that turn. The model can compose an answer or propose an action; the application checks permissions, business rules and approval before anything commits.

Keep knowledge and account state separate

Retrieve explanations from approved documents with a current version, market scope and access permissions. Apply tenant and user filters before ranking. Retrieve live balances, dates and order status through authenticated product APIs. An embedding index is not the source of truth for an order that changed a minute ago.

Make action execution deterministic

A delivery change should move through explicit states: proposed, confirmed, committed or failed. The confirmation is bound to the precise order, date and operation. Recheck eligibility when committing because dispatch status can change after the customer sees the proposal. Use an idempotency key so a retry cannot create a second change.

Put model routing, request budgets, approved providers and telemetry behind a shared gateway. The feature asks for a task capability, while the gateway selects an evaluated model route. A fallback route must preserve the data policy and tool restrictions of the original request.

4. Assign autonomy to each workflow

A product should not have one global switch labelled autonomous. A documentation answer, an account lookup and a refund need different controls. Record the current level and maximum permitted level for each workflow.

For a first release, let employees review answer drafts at L1. Use L2 for a customer-requested account change: show what will happen and wait for confirmation. A simple, reversible action may later qualify for L3 under explicit limits and with a clear undo path.

Do not equate a successful offline test with permission to increase autonomy. Review live outcomes by intent and language, approval edits, reversals and missed escalations. The accountable owner authorises the change in configuration. Sensitive eligibility decisions and exceptional commitments belong in specialist workflows with their own controls.

5. Store the evidence needed to reconstruct a decision

Conversation history alone is insufficient for debugging. Record the product context that the assistant used, the tool result it saw, the applicable rule and the exact action that the user approved. Redact unnecessary personal information before it enters operational logs or evaluation datasets.

Keep references to sensitive source records where possible instead of duplicating their contents. Apply access controls and retention to the evidence store itself. A useful audit trail must also be a manageable data asset.

6. Choose models by task and measured performance

Intent classification and handoff summaries can often use a smaller model than multi-step planning. Answer composition needs strong performance on the product’s actual documents, languages and terminology. Planning needs reliable structured tool proposals. Evaluate those tasks separately.

Require structured output wherever software consumes the result. A router might return an intent, required identity level and handoff reason. An action planner should return only a proposal matching a strict schema. Schema validity helps with parsing; it does not prove that the action is permitted or that the answer is correct.

Compare candidates on held-out examples for task success, policy adherence, tail latency and total cost. Cache stable prompt prefixes when supported. Keep session state compact, cap steps and avoid repeatedly sending the entire transcript. Fine-tuning is an option after you understand the errors and have a suitable labelled dataset, rather than a prerequisite for the first version.

7. Integrate with the product experience

When the assistant opens from an invoice, pass an authorised invoice reference with the request. Do not ask the customer to retype information that the product already knows. Do not trust a record ID from the interface without checking ownership on the server.

Make the interaction legible: identify the assistant as AI, show source links beneath policy explanations and distinguish an action proposal from a completed change. A confirmation card should include the target record, the exact change and any charge or consequence. A success message appears only after the system of record confirms the operation.

Design handoff as a completed product flow

A person must be reachable without an argument with the assistant. Send the support agent a structured packet: verified identity level, intent, relevant records, evidence, actions attempted and the reason for escalation. Show the customer whether they are entering a live queue, creating a ticket or requesting a callback.

Use actual queue availability and service hours when presenting the next step. A transfer to an unattended inbox does not count as a successful handoff.

8. Treat voice as a separate engineering surface

Voice can reuse the same intent catalogue, tools, policies and decision records. It adds speech recognition, synthesis, turn-taking, interruptions and the possibility that a critical name or number was heard incorrectly.

A cascaded design converts speech to text, runs the established conversation workflow and converts its response to audio. It makes the intermediate text available for inspection. A speech-to-speech design can offer more natural interaction, but the implementation still needs equivalent checks on identity, actions and recorded evidence.

Test critical entities such as dates, amounts and addresses independently of general transcription quality. Read them back before a change, and offer a secure keypad or in-app input for sensitive values. When the customer interrupts, stop the spoken response and establish whether an action is merely proposed or has already committed. Interrupting audio is not a transaction rollback.

Set latency budgets using end-of-speech to first useful audio, and report the distribution for real accents, background noise and network conditions. The Playbook suggests roughly one second as an initial design ambition; validate feasibility with your channel and quality requirements.

9. Enforce permissions and limits outside the model

Customer messages, attachments and retrieved passages are untrusted inputs. They may contain instructions that attempt to override the assistant’s task. OWASP describes both direct and indirect prompt injection and recommends controls including least privilege and human approval for high-risk operations. No single prompt reliably removes this risk. Source: OWASP prompt injection guidance.

Separate extraction from privileged execution. Validate the proposed operation against the authenticated user and current product state. Limit available tools per intent. A policy service should decide whether a change is allowed, using rules that can be tested without a language model.

An illustrative action contract

{
  "operation": "reschedule_delivery",
  "order_ref": "ORD-10482",
  "requested_date": "2026-10-20",
  "proposal_version": 3,
  "approval_ref": "approval-for-this-proposal",
  "idempotency_key": "stable-operation-identifier"
}

The server derives tenant and customer identity from the session, verifies that the approval belongs to this proposal, checks current dispatch status and writes the result once. Values in this example are illustrative. The assistant cannot broaden its permissions by inventing a different customer ID or asserting that a supervisor approved the change.

Keep passwords, card numbers and full identity documents out of ordinary conversation flows. Direct customers to the product’s established secure interfaces. Detect unsupported commitments in the final answer, and ensure an answer claiming completion is backed by a committed operation.

10. Evaluate task completion and failure paths

Build a versioned test set from representative support cases before customer exposure. Include unanswerable questions, superseded policies, mixed-language requests, cross-account attempts, repeated submissions, changed order state and indirect injection in a retrieved document.

The Playbook offers starting targets such as 97% overall policy accuracy and 100% on a critical subset. Treat them as design proposals to calibrate, not universal acceptance criteria. Zero failures in a finite test set also does not prove zero production risk. Use exposure limits, monitoring and rollback alongside testing.

Validate automated graders against human reviewers. Use deterministic checks for source versions, amounts, permissions and action states. Break down results by intent, language and market; a strong average can hide a weak low-volume segment.

11. Operate the assistant as part of the product

Release prompt changes, model changes, tool changes and retrieval changes through the same evaluation gates. Start new routes in shadow mode, where they see representative inputs but cannot perform customer-facing actions. Canary a small, defined cohort and keep a tested rollback flag.

Use an explicit fallback sequence. An approved alternative model may take over. If action quality cannot be maintained, remove action tools and offer an answer-only experience. If the model layer is unavailable, offer search or a human queue. Explain the available mode to the customer.

Measure the full cost of resolution

Add model inference, retrieval, platform operation, speech services and human handoff costs. Divide by contacts independently classified as resolved without same-issue recontact within the observation window. State the accounting window and allocation method so comparisons remain meaningful.

A cheaper inference route can increase total cost if it causes longer conversations or more repeat contacts. Review the outcome and its cost together before optimising token spend.

12. Anticipate the failures that customers notice

• Apparent containment rises while customers leave. Join chat, email and phone contacts; inspect abandonment and repeat issues.
• An old policy keeps appearing. Retire superseded document versions at ingestion and test that recent answers cite active sources.
• The assistant repeats apologies. Detect repeated intent with no progress and offer a useful handoff.
• A retry creates a duplicate change. Persist action state, reuse idempotency keys and reconcile uncertain outcomes with the system of record.
• A secondary language degrades quietly. Maintain language-specific tests and a minimum human review sample.
• The assistant announces an uncommitted action. Generate completion language from verified execution state, never from the planner’s intention.

Review sampled conversations with support leads each week. Turn each recurring failure into a test case, a content-owner task or a product correction. Feedback is useful when it changes the system that produced the error.

13. Build in stages and expand with evidence

1. Map the work. Analyse contact reasons, identify the system of record for each intent and define resolution. Produce the feature contract and initial evaluation set.
2. Start with employee-reviewed drafts. Add approved retrieval, read-only tools and structured handoffs. Capture accepted, edited and rejected suggestions.
3. Expose one intent to customers. Ship clear sources, a visible human path, measured outcomes and tested fallback. Keep transactions behind explicit confirmation.
4. Add one bounded action. Implement policy checks, approval binding, idempotency, reconciliation and reversal before increasing traffic.
5. Expand by intent, then channel. Add languages and voice after the underlying workflow demonstrates reliable resolution. Re-evaluate each increase in autonomy.

This is a sequence of evidence gates, not a guaranteed delivery calendar. Integration complexity, content quality and the consequences of an error determine how long each stage takes.

The Infinity Technologies approach

The Playbook’s approach begins with discovery artefacts: a contact-reason analysis, an intent catalogue, an evaluation and adversarial test set, a model comparison on representative cases and a business case per intent. These artefacts connect engineering effort to a specific customer outcome.

The shared platform then supplies model access, retrieval permissions, policy checks, decision records and evaluation. Reusing those capabilities gives the next product feature an established foundation. For the related enterprise support architecture, read our Customer Support Agent engineering guide.

A useful first engagement ends with a concrete decision: which request the assistant will resolve, what evidence it may use, which actions it may take and how the team will know that customers are better served. Explore AI product development with Infinity Technologies.

Sources and series notes

Primary editorial source: The AI Product Modernization Playbook, 8 October 2026, Part 0 and Chapter 01. Company case figures are dated and attributed above. Architecture diagrams describe a reference design; the HERO is an editorial illustration, not a product screenshot.

This is article 01 of a 12-part series about AI inside software products. The next article covers knowledge and retrieval-augmented generation: how to deliver answers that are current, permission-aware and supported by evidence. Browse all Insights.