Testing AI agents without touching production

A support agent is asked to prepare a replacement for a damaged item. During a test it creates a replacement order, emails the customer and closes the ticket. The final reply sounds helpful. The test has also sent a real customer a promise nobody intended to make.

An agent test needs control over where actions land. A prompt saying “this is a test” offers no protection when the process still holds a production API key. Put the agent in an environment where its tools can only reach disposable records and captured messages.

Keep the execution code in the test

Run the same tool schemas, argument validation, permission checks and approval logic that the application uses. Replace the external service adapters underneath them. A fake order service can maintain a small database; a fake email service can append messages to an outbox that has no delivery worker.

Give the test runner separate credentials and explicit network rules. If it needs a hosted model, allow that endpoint while denying access to production business systems. Remove unused tools from the test process. Check that a missing test configuration fails at startup instead of falling back to a production URL.

Use synthetic or properly redacted customer records in fixtures. A fake adapter isolates side effects, but any text sent to a hosted model still leaves the runner. Keep real secrets and customer details out of the test data.

Describe the starting state and the allowed result

Give each case enough context to expose a decision. For the replacement example, the fixture might contain one damaged item, a valid shipping address and a policy requiring staff approval before dispatch. The requested task is to prepare a replacement for review. The expected result is one draft order linked to the ticket, with dispatch still blocked.

Write assertions for forbidden effects as well. The outbound message count should be zero. The original payment should be untouched. The ticket should remain open. Otherwise, a test that only checks for the draft can pass after the agent has done several unwanted things around it.

Anthropic’s guide to agent evaluations distinguishes the agent’s transcript from the resulting environment state. It also describes why an exact sequence of tool calls can be too restrictive: more than one route may complete a task correctly. Check records and required approval boundaries, then use the trace to explain a failure.

Case	Fixture or injected event	Required result
Replacement needs approval	Valid claim; no staff approval	One draft; no dispatch or customer email
Address is missing	Ticket has no delivery address	Request the missing information; create no order
Order reply is lost	Service saves the order, then times out	One order after recovery; the same receipt
Access is denied	Another account owns the ticket	No ticket content returned; no changes
Tool keeps failing	Service repeatedly returns a temporary error	Stop within the attempt budget; preserve the unresolved state

Make fake services fail at awkward moments

A fake that always returns success exercises very little of the integration. It should enforce the service rules the agent depends on: required fields, record ownership, valid state changes and duplicate-operation handling. Keep those rules small and explicit so failures are understandable.

Add failures at specific boundaries. Reject a request before it is accepted. Accept it and lose the reply. Return an incomplete page of search results. Let a delayed response arrive after the caller has timed out. These are different conditions, and a generic “throw an error” switch will not cover them.

For the lost-reply case, use the real operation ID and retry code. If the fake silently deduplicates by item name, it can conceal a bug where every retry invents a new operation ID. Make the fake honour the same idempotency contract as the intended service.

Reset between trials

Start every run from a fresh copy of the fixture, including its outbox, cache and operation records. Give each trial a unique namespace. An order left by an earlier run can make a broken lookup seem correct, or make a correct run fail because the fixture is no longer in its starting state.

Repeat cases when the model’s choices affect the result. Keep the model identifier, prompt version, tool definitions and limits with each run. Report successes out of the number of trials and inspect failures individually. Five successful runs of a small fixture set are useful evidence about those cases; they do not establish a reliable production failure rate.

Keep deterministic integration tests alongside these model-driven runs. A concurrent duplicate request or a crash between two writes is easier to reproduce by driving the execution code directly. The model does not need to rediscover the failure on every test run.

Check where the simulation stops

A local fake can tell you that the agent followed your assumed service contract. It cannot establish that the provider actually behaves that way. Use separate contract tests against a provider’s sandbox, where available, to check authentication, pagination, field validation and error formats. Compare those responses with the fake and update it when the contract changes.

A sandbox may omit production limits or behave differently under load. Record those gaps. A later read-only shadow run can expose differences in real inputs and retrieval, but it still cannot validate a real write. Keep shadow credentials unable to mutate records, and capture proposed actions for review.

Start with a few past cases where the allowed outcome is clear. For each one, write the initial records, expected changes and prohibited effects before running the agent. When a run fails, save the smallest fixture that reproduces it. That fixture should run again before the next prompt, model or tool change ships.