The refund that never issued. The records it never wrote. The account it updated on the wrong customer. Most of the time your agent did the work. The times it did not are the ones that cost you. The trace is green either way, because a trace only records what the agent says about itself. trulight reads your systems independently and reconciles what your agents claim against what actually landed.
Reconciled, not reported.
The trace. The success rate. The resolution number. The cost per ticket. All of it is the agent's account of its own work, and none of it has been checked against the systems where the work was supposed to land.
Reconciliation is the check. Your finance team already does it with money: two records, made independently, laid side by side, and you look at what does not match. We do the same with what your agents did.
That gives you a second record of your agents’ work, and this one your agents did not write.
Every claimed write held against the system it named. Or not matched, which is the whole point, and what that was worth.
Graded on your own definition of done, checked in your systems, not taken from the agent's report of itself.
Spend divided by successes you can point at, instead of successes the agent counted.
Stripe is extremely reliable and merchants still reconcile every day. This is not a bug that gets fixed.
Your traces and your guardrails both look at the action before or as it happens. Neither looks at your system afterward to confirm it worked.
It records the call and the 200 that came back, not whether Stripe, Zendesk or your database actually changed.
The write can be dropped, ignored, or applied to the wrong record and still return success.
"Done, refund issued" comes from the same run that made the mistake. If it hallucinated the action, it hallucinates the success too.
Amount thresholds, approval steps, eligibility checks. They stop the obvious bad request going in. They cannot confirm a good one actually landed. Here is what passes a well built guardrail and still breaks.
passes: a $40 refund is correctly auto-approved.
breaks: the call returns 200, the write silently drops. Money never moves.
passes: the agent meant $24.70 and wrote $247. Valid, under $500, every check clears.
breaks: it lands. The customer is overpaid ten times over.
passes: the refund is approved and lands cleanly.
breaks: the request was to cancel and refund. The agent refunded, never fired the cancel, reported both done. The subscription renews and bills again.
passes: an identical replay of the same call is swallowed.
breaks: the agent does not replay the call, it decides again. New request, new key. Both are valid, both approved, both land.
Carnegie Mellon staffed a fake software company with the leading agent models and gave them real office work. One agent could not find the person it needed on the company chat, so it renamed a different user to that person's name and carried on as if the job were done. Another was told to introduce itself to a named colleague, did not, and marked the task complete.
The agents were not lying. When an agent hits the part it cannot do, it invents a shortcut and reports the shortcut as the work. Better models will do this less often. They will not stop, because every agent has an edge.
TheAgentCompany, Carnegie Mellon University. arXiv:2412.14161.
Every category around agents answers a different question about the same run. Five are answered inside the agent, or before the write leaves it.
| # | The question | Who answers it | When |
|---|---|---|---|
| 1 | Would it have got this right in testing? | evals | before production |
| 2 | Was the agent allowed to do this? | guardrails, gateways | before the action |
| 3 | Did the run survive and finish? | durable execution | during the run |
| 4 | What did the agent report doing? | observability | during the run |
| 5 | Is the record of what happened unforgeable? | signed receipts | as it happens |
| 6 | Did the action actually land, and was it asked for? | trulight | after, in your systems |
The first five harden the front door, confirm the parcel left, and make the receipt tamper proof. None of them opens the box at the other end.
Anyone can write the check. The hard part is being outside the agent.
From a B2B SaaS support deployment, four agents, 30 days. Every number below is from that window.
Your history already has the answer. We read it on day one, before you change anything.
Reported success rate: 87%. Checked against the systems: 79%. The gap is eight points of work your dashboard counted and your systems never saw.
No estimates, no model. Every figure is a claim your agent made, held against the system it named.
$15,370 in play, against $3,230 of total agent spend in the same 30 days. Not all of it is lost. All of it is unaccounted for until somebody checks.
Reads and messages carry no verdict. Only the actions that were meant to change something.
Cost per token is solved. A dozen tools do it. Cost per outcome is not, because nobody can count the outcomes.
| Cost per run | $0.44 | measures spend, not value |
| Cost per claimed outcome | $0.50 | understated, every time |
| Cost per verified outcome | $0.55 | the only honest one |
Here, every real success cost 10% more than the dashboard said. On a bigger programme that gap is the whole business case.
Point trulight at your existing traces. No rewrite, no new SDK in your agent loop.
Read-only access to where the work lands: billing, CRM, ticketing, ERP, your own database. If your team touches it, it probably fits.
In plain English, the way you would say it out loud. We tell you what we can already see, and what we cannot.
Before any number means anything, you get an honest map of your own coverage. Here is what that looked like for the deployment above.
Stripe is extremely reliable and merchants still reconcile every day. Reconciliation tracks how much work gets handed off, not how often it breaks.
And most teams do not arrive asking "is my agent broken." They arrive asking "what did it get done, and what did that cost." That question comes round every quarter.
Different questions, and the second is the one your customer cares about. Keep your checks. This sits underneath them, across every agent you run.
Every cost tool does. The question is what you are dividing by, and right now it is a count your agent produced about itself.
Those reads run on separate read-only credentials, never the agent's own access, and nothing is written back. We are early enough that deployment is a conversation, so tell me your constraints and we will work to them.
The trade is your feedback and one real workflow to reconcile.
I am building trulight solo. Parts of the core are built and tested against a public agent benchmark, and I am now looking for the first design partners running real agents in production.
If that is you, I would rather learn how you check your agents today than pitch you.
In practice: a call, then a month of traces you already have and read-only access to one system. Days of setup, not a quarter.
I read every one of these myself. Reconciled, not reported.