R/Pulse
Back to blog
agentic AIAppSectesting

How to test an AI agent before putting it into production

R/Pulse TeamAugust 28, 20262 min read

Agent approval meetings tend to circle the wrong question: does it answer correctly?

The answer is almost always yes, because that is what the team spent weeks tuning. The agent handles the task, the tone is right, the test cases pass.

The question that matters is a different one: what else can it do?

Ordinary software does what was written. An agent does what it was allowed to do, in whatever order it picks. So the test has to change too.

What to test

Compare real scope against intended scope

Take every credential the agent carries. List everything it permits in practice, not what the docs claim it does. The gap between those two lists is your risk surface.

A token described as "order lookup" usually reaches billing as well, because both live in the same service.

Test sequences, not isolated calls

Testing endpoints one by one approves dangerous combinations. Build chains of three to five calls, each valid on its own, and watch what they produce together. That is where you find the problems no scanner catches.

Treat every input as a possible instruction

Anything that enters the agent's context can carry a command. That covers a ticket, a PDF, another tool's output, a name field in your database. Test with hostile input from each of those sources, not just the user chat.

Push the limits

Agents fail in creative ways when something leaves the happy path. Check pagination caps, retry loops and cost per interaction. Look at what happens when a tool returns an error.

Try to reconstruct each decision

After running the tests above, explain why the agent chose each action. If the log cannot reconstruct that now, it will not help during a real incident.

One question separates a useful test from approval theater. Has anyone here tried to make the agent work against its own purpose? If nobody tried, the first person to try will be an outsider.

Who should run this test

Not the team that built the agent. Whoever designed the happy path has a blind spot for the hostile one. That is not a competence problem: testing your own hypothesis is a different exercise from trying to break it.

The work belongs to application security, and the method already exists. It is the same reasoning as an API pentest. What changes is the consumer: not a human running a script, but a system improvising its next call.

The approval bar

An agent is ready when the team can prove three things. That it knows everything the agent reaches. That it tested the combinations the agent could assemble. That it has enough trace to reconstruct any decision.

"This will delay the launch" is the most common objection, and it does not hold. The first item takes half a day and already changes the conversation. Most of the time, the list of what a credential permits is far longer than the team expected.

Share

Ready to put this into practice?

API Resilience Core: if we don't find high or critical risks, you don't pay.