Replay your recent tickets through the AI before it sees a single live one
Simulations run the assistant against your past resolved tickets and produce a reviewable result set — each answer with its sources, confidence score and a human review — so you calibrate the confidence threshold on real data instead of guessing. Enable deflection gradually, group by group or inbox by inbox, once the numbers are convincing.
What Simulations do
The safe-rollout mechanism for Support Ticket AI and helpdesk copilot
Historical Ticket Replay
- Point at up to 100 of your most recent resolved tickets in Zendesk / Intercom / Freshdesk / Front / Linear
- The assistant answers each ticket as if it were live
- Every answer stored with its retrieved sources and confidence score
- Compare AI response side-by-side with what the human agent actually sent
Reviewable Results
- Per-answer human review — mark each answer good or bad
- Each answer shown with the sources it cited
- Confidence scores per answer help you set the auto-response threshold
- Nothing is posted to customers during a run
Gradual Rollout
- Answer only in the Zendesk or Freshdesk groups, Intercom team inboxes or Front inboxes you pick, and add more once simulation accuracy clears your bar
- Ship as agent-copilot drafts with Draft only until you're ready to auto-respond
- One threshold: above it Kelu replies (or drafts); below it, tags the ticket for your team
- Re-run simulations after every meaningful docs change to catch regressions
- Traces on real answers keep the review loop going after launch
The rollout question every ops lead asks
The reason support automation stalls at pilot isn't the AI — it's the risk. Nobody wants to enable auto-response and discover on Monday that the assistant misquoted a refund policy 40 times over the weekend. Simulations answer the question in advance: here's what the AI would have said to your last 100 tickets, here's how confident it was, here's where your reviewers marked it wrong. That's the artifact you take to a launch decision.
- Run a simulation before every helpdesk rollout
- Human-review queue lets support leads sign off ticket-by-ticket
- Draft only mode lets you deploy safely instead of all-or-nothing
- Re-run after any docs migration or reranker change to catch regressions
Simulation Results
Review AI-drafted answers and approve or reject each one.
| Ticket | Draft Answer | Agent’s Reply | Confidence | Review | Actions |
|---|---|---|---|---|---|
Ticket #4802 Webhooks stopped retrying | A failed delivery is retried 5 times, with backoff. After the last try it is marked failed and can be resent. | We retry 5 times, then mark it failed. You can resend it from the event log. | 86% | Approved | |
Ticket #4797 Change invoice currency? | You can change the currency before your next invoice, under Billing → Currency. | Yes, under Billing. It applies from the next invoice. | 72% | Pending | |
Ticket #4791 Custom renewal terms | I couldn’t find renewal terms for custom contracts in the Acme docs. | Looping in your account manager to confirm. | 31% | Rejected |
Answers you can check, from a system that won't guess
It says "I don't know"
When your content does not cover a question, Kelu does not guess. In a helpdesk it posts nothing and tags the ticket for your team.
Every answer from your docs shows its sources
Numbered links point to the page and section each part of the answer came from, so anyone can check it in one click.
A bad sync can't empty it
If one sync would remove more than 45% of a source's pages, Kelu holds the removals until a person approves them.
Got questions?
Frequently Asked Questions
Common questions about Simulations
Your most recent resolved tickets, up to 100 per run. The assistant answers each one from your indexed knowledge base as if it were live. Nothing is sent back to customers; the ticket state in your helpdesk is not touched.
A run covers up to 100 tickets and stops after 15 minutes. A workspace can rehearse up to 500 tickets a day, one run per integration an hour.
Each result shows the ticket's question, an excerpt of what your agent sent, the AI's answer with its sources, and a confidence score. Reviewers mark each answer good or bad.
Not yet. Simulations run on helpdesk integrations: pick the integration, review the outputs, calibrate the confidence threshold, and enable it at that threshold.
After any change that could plausibly regress retrieval: a large docs migration, a new source added, a reranker version bump, a model upgrade on the provider side, a persona/tone change. In practice, teams that ship weekly re-run a small representative simulation weekly and a full one monthly.
Ship with numbers, not with a hope
Run your first simulation against your most recent tickets and see each answer, its sources and its confidence before you enable anything customer-facing.
Free to start · No credit card required