Questions our customers ask about AI agents in production
Our customers ask us these questions as they put agents in front of their own customers. Every team that puts an agent in production runs into them, so here are the answers in one place, each with an example. The examples use a made-up online store, Acme, whose order-desk agent answers customers and handles refunds. The questions are paraphrased, and nobody is named. Kindgi is an agent framework for production: your coding agent writes the tools, agents and flows in your own code, and Kindgi runs them and keeps every step on record.
If an agent already sent the email, what does a rollback undo?
Nothing that already went out. A rollback decides which version runs next. Fixing what a run already did is a separate job, and worth designing for.
For example: Acme releases a new version of its order-desk agent on Monday. By Tuesday it has emailed 40 customers a refund confirmation with the wrong amount. Acme rolls back, so Wednesday’s emails are right. The 40 emails are still in 40 inboxes.
The fix is a new action: find exactly who got what, decide what to send them, and get it right, because it reaches the same customers twice. It doesn’t have to be damage control. A refund that went out by mistake can come with an honest second email: we got it wrong, keep it. Done well, that wins a customer back.
In Kindgi, each run’s journal has every tool call with its arguments and its result, so Acme’s code can list the 40 customers and what each one was told. A flow sends the follow-up, an agent can draft it, and the send can wait for a person’s yes. For what can’t be taken back at all, the same yes goes before the first email.
The guides: read the journal, ask before a tool runs and webhooks.
What happens to a waiting approval when its version is rolled back?
It stays tied to the exact call it asked about. An approval is only as good as what the person was shown, so it can’t carry over to a version they never saw.
For example: a run on Acme’s new version asks to refund $480 on order 1042, and waits for the support lead. Before they decide, Acme rolls back. The request still shows what was asked, refund $480 on order 1042, and the support lead decides on exactly that.
In Kindgi, a waiting run keeps the version it started with, and its approval stays tied to that exact call. A rollback changes what new runs get, never what’s already waiting. If the request is moot, the reviewer can turn it down or close it, with the reason on record.
Doesn’t the approval queue become the bottleneck?
Only if every action waits for a person. Match the check to the risk: reading data needs no one, a rule you can write down belongs in code, and a person’s attention is for judgment.
For example: Acme’s order desk looks up orders, refunds small amounts and now and then refunds a large one. Lookups run on their own. “Refunds up to $100 are fine” is a rule, so the everyday refund tool caps the amount in its code and runs too. A bigger refund goes through a second tool that always waits for the support lead. On a busy day, only the big refunds reach a person.
In Kindgi, you choose which tools wait for a person, and a tool marked read-only never waits. Company-wide rules can make a risky tool always ask, for every agent. More in Approval fatigue.
The guides: ask before a tool runs and mark a tool read-only.
Is an audit trail only for regulated companies?
No. Every team needs to see what its agent did. A regulated company, or any team facing a security review, needs one more thing: a record that someone outside the team can check.
For example: an Acme customer says the agent promised a refund that never came. Acme opens that run’s record: the customer’s message, the agent’s reply, the refund call for $480 on order 1042, and the support lead’s no, with the time and the reason, “outside the return window”. The prompt alone would never show that. Months later, a security review asks Acme how it controls what its agent does. Acme hands over signed evidence, and the reviewer checks it without access to Acme’s systems.
In Kindgi, every step of a run lands in its journal with its inputs and outputs, and every approval keeps who decided, when, why and what they were shown. Your deployment signs the evidence you export, and anyone can check it without access to your systems. Even Kindgi on your laptop does it.
The guide: export signed evidence.
Would a new model that behaves differently be easy to spot?
Yes, if the change is a release you test, not a default that moved. Make it on purpose, test it on cases you’ve already judged, and roll it out to a few first.
For example: Acme wants its order desk on a newer model, so it makes a new agent version that names that model. Before any customer sees it, Acme replays the new version on past runs its support team already marked right or wrong, and compares. Then Acme pins the version for one project. If refund answers get worse, Acme pins the one before.
In Kindgi, each run shows which model answered every call. An agent version locks its tools’ versions, and if it requires a model, a model change means a new agent version. You can replay it on runs your people already corrected, then pin it for one project first. More in Upgrading the model under a live agent.
The guide: release an agent version.
Do I bring my own model keys?
Yes. Your model bill and your provider’s terms, like zero data retention, are worth keeping yours.
For example: Acme has its own account with a model provider, with zero data retention in its terms. Its agents’ model calls go through that account, so the bill and the terms stay Acme’s. The order-desk agent says it needs tool use and a long context, and each turn goes to a model on Acme’s account that has both.
In Kindgi, that holds wherever it runs, including Kindgi Cloud, which is in private preview. Model calls go through your own provider account: Anthropic, OpenAI, Gemini, Bedrock and more, or your company’s AI gateway, so its policies still apply. An agent can say what it needs, like tool use, vision or a long context, and Kindgi can send every turn to a model that qualifies. Or name an exact model. Every model call is on record with its cost.
To see these pieces on your own machine, your first agent takes about ten minutes: kindgi.com.
How this was written: a coding agent drafted it from questions our customers ask, our docs and our code. We checked every claim, edited it and said yes. We publish articles like this partly because we want you to find us, including through AI assistants. More in How we write our posts.