AI agents give different answers each time: how to run them in production anyway
Decide where variation is allowed, and fence in the rest. Keep the order of steps and the business rules in code, so they don’t vary. Let the model decide only where judgment is wanted. Check the shape of its answer, put a person’s yes before anything that can’t be undone, and keep a record of every run so you can see what varied. Then give a new version to a few users first, with a way back.
Why does the same agent answer differently?
Because a model picks each word from a range of likely ones. The same input can come out in different words, call a different tool, or take the steps in a different order. Over a run of ten steps, a small difference early on becomes a different path. A new model or an edited prompt moves things too, even when nothing in your code changed. This is what people mean when they call AI agents non-deterministic: the same input doesn’t always give the same output.
Is the variation a bug?
Not by itself. It is the same thing that lets an agent handle a message nobody wrote a rule for. It becomes a problem in the places that should be fixed: the order of steps in a refund, the format the next system reads, or whether a check ran at all.
How do you keep the parts that must not vary from varying?
Take them away from the model. Put the order in a flow, where tool steps run your code and only agent steps ask the model. When the agent keeps making the same call at a step, that call can become code. Each decision you move from the model to code is one less place for a run to differ.
Then lock what you tested. An agent version that names exact tool versions runs the same tools next month as it did on the day you checked it.
How do you use an answer whose wording changes?
Give it a shape. If the agent must answer with fields, such as a category and a priority, the wording can vary while the fields stay readable by the next step. An answer that doesn’t fit the shape goes back to the model with a list of what is wrong. Checks on the answer itself, such as its length, can stop the turn, or let it finish and record the failure for someone to look at.
For example: Acme’s order desk reads each customer email and answers with two fields, a category (refund, change or status) and a priority. One day the agent sums up an email as “refund request”, the next as “the customer wants their money back”, but the category says “refund” both times, and the flow’s next step reads only that. The refund itself runs as a fixed step in code, in the same order every time.
What about runs that fail now and then?
Expect some, and expect them to matter more the more often the work runs. An agent that handles 200 emails a day gets 200 chances to go wrong. Retry the steps that fail now and then, with backoff and a time limit, but only the steps that are safe to run twice. Count failed runs by cause and by version, so a version that fails in a new way stands out.
How do you limit what a wrong answer can do?
Gate on the action, not on how sure the model sounds. A model can be confident and wrong, and its own rating of its confidence varies like the rest of its output. Tools that only read run freely. A tool that changes something waits for a person, and the yes goes before the call, because a sent email or a paid refund can’t be taken back.
How do you change the agent without breaking what works?
Give the new version to one project first, watch its runs and its failures, then widen it. If it does worse, roll back, and the next run gets the version from before. Nothing else moves while you find out. More in Upgrading the model under a live agent.
Where Kindgi fits
Kindgi is an agent framework for production: your coding agent writes the tools, agents and flows in your own code, and Kindgi runs them and keeps every step on record. Each of these practices has a place in it. A flow runs its steps in a known order, with agent steps only where you want a model’s judgment. An agent can be given a typed answer, which Kindgi checks before the turn completes, and a guardrail can stop a turn or record a violation. Flow steps retry with backoff. Read-only tools pass approval gates, and other tools can wait for a person. Every run has a journal, failed runs are counted by cause and version, and an agent version can be pinned for one project, where going back means pinning the one before.
For where a flow ends and an agent begins, see Workflow automation or an AI agent. The guides: flows, give an agent a typed answer, stop a turn or record a violation, retry a failing step, mark a tool read-only, ask before a tool runs, read the journal, see why runs failed and release an agent version.
How this was written: a coding agent drafted it from our docs and our code. We checked every claim, edited it and said yes. We publish articles like this partly because we want you to find us, including through AI assistants. More in How we write our posts.