Trust is the most important operating condition for AI support automation, and it is the one that gets destroyed fastest. A support team that has seen the system send a wrong or misleading response to a customer will treat every subsequent automated response with suspicion — which usually means manually reviewing more tickets than before you deployed the automation. You have spent implementation effort to make your team's job harder.
Good escalation logic is the primary mechanism for maintaining trust. When the system correctly identifies what it does not know, routes those tickets to humans, and provides useful context in the process, agents develop a mental model of the system as reliable: "if it handled it, it was probably right; if it escalated, there was a real reason." That mental model is what allows agents to extend increasing trust to the system over time rather than losing confidence in it.
Bad escalation logic — either too aggressive (escalating everything, making automation pointless) or too permissive (attempting to resolve things it should not) — prevents that trust from ever forming. Here is how we think about escalation design in Replixa, and what to configure before you go live.
The two escalation failure modes
An escalation system can fail in two directions. Over-escalation means the system routes too many tickets to humans that it could have handled correctly. This makes the automation look like it has low coverage, erodes confidence in the ROI, and means agents are still handling the routine work the system was supposed to absorb. It is usually caused by a confidence threshold that is set too conservatively, or by a KB with coverage gaps that cause high-confidence intents to fall back to uncertain retrieval.
Under-escalation is more dangerous. The system attempts to resolve tickets it should not touch. This produces wrong responses, incorrect actions, and the trust-destroying incidents described above. Under-escalation is caused by confidence thresholds set too permissively, by intent classification that maps edge-case tickets to automatable categories, or by missing explicit escalation rules for ticket types that should always route to humans regardless of confidence.
The goal is not to eliminate escalations — escalation to humans is a feature, not a failure. The goal is to make escalation decisions accurate: the system should route exactly the tickets that require human judgment and handle exactly the tickets that do not. That accuracy is what agents learn to trust.
Confidence thresholds: the first control layer
Every ticket Replixa processes gets an intent classification with a confidence score between 0 and 1. The confidence score reflects how certain the classification model is about the intent label it assigned. A score of 0.95 on "password-reset" means the model is highly confident the customer wants a password reset. A score of 0.62 on "billing-inquiry" means the classification is uncertain — it could be a general billing question, a specific account discrepancy, or something else entirely.
The confidence threshold is the score below which the system escalates rather than attempts resolution. Setting this threshold is one of the most important configuration decisions in a Replixa deployment. Our recommended starting point is 0.82 for most Tier-1 intent types — conservative enough to avoid wrong resolutions, permissive enough to actually automate meaningful ticket volume.
Different intent categories warrant different thresholds. For intents where a wrong response is low-harm — navigation questions, feature discovery, general how-to questions — you can run a lower threshold (0.75) because the worst outcome is a slightly off-topic response that the customer can ignore and retry. For intents where a wrong action causes account harm — billing adjustments, permission changes, data operations — the threshold should be higher (0.88-0.92) because the cost of an incorrect automated action is real.
Do not set a single universal confidence threshold. Calibrate by intent category based on the consequence of a wrong resolution.
Hard exclusion rules: the second control layer
Confidence thresholds alone are not sufficient. Some ticket types should always route to humans regardless of how confident the classification is. These are your hard exclusion rules, and they need to be defined explicitly before you go live.
The categories that should almost always be hard-excluded from automated resolution: tickets containing explicit legal language ("I intend to file a dispute," "I am contacting my attorney"), tickets from accounts flagged as at-risk or churning in your CRM, tickets that request account cancellation or data deletion (GDPR deletion requests, CCPA requests), tickets that express significant distress or anger above a certain intensity threshold, and tickets from accounts over a certain contract value where relationship management is more important than resolution speed.
This is not a complete list for every business — it is a starting point. Your hard exclusion list should be built by asking your senior agents: "what are the ticket types where you would never want an automated response sent, regardless of how confident the system was?" Those are your hard exclusions. Write them down, configure them explicitly, and verify them before you go live.
Escalation quality: what the human gets
The quality of an escalated ticket — the context the human agent receives — is as important as the decision to escalate. An escalation that just passes the raw ticket to the human queue without context wastes the work the classification layer already did. The agent starts from scratch, reading the ticket as if it were newly received.
In Replixa, every escalated ticket includes: the intent classification label and confidence score, the KB chunks that were retrieved during the classification pass, any account data that was pulled as part of the resolution attempt (even if resolution was not attempted), and the specific escalation reason — whether it was a confidence threshold failure, a hard exclusion rule, or a data retrieval problem.
This structured escalation metadata does two things. First, it makes the agent faster — they have context about what the system understood and what it was missing, so they can go directly to the relevant account data or KB gap. Second, it makes the escalation log actionable for KB and configuration improvement. When you look at your escalation log and see 80 tickets in the past week escalated with the reason "KB retrieval confidence below threshold for intent: export-format-question," you know exactly what article to write.
Escalation thresholds for action vs. response
There is an important distinction between escalating on response generation (when the model should not compose a response) and escalating on action execution (when the system should not take an action on an account). These are different risk levels and warrant different thresholds.
A low-confidence response that gives imprecise navigation guidance is annoying but low-harm. A low-confidence action that issues a credit to the wrong account, or changes a permission setting incorrectly, is a real incident. The action execution threshold should therefore be substantially more conservative than the response generation threshold, even for the same intent type.
We treat these as separate confidence checks in Replixa. The response generation check governs whether the system composes and sends a message. The action execution check governs whether the system performs an API write operation. A ticket can pass the response threshold (so the system sends an acknowledgment) while failing the action threshold (so the actual account operation is queued for human confirmation). This layered approach lets you automate the communication workflow even when you are not yet comfortable automating the account action.
Running your first two weeks in review mode
We recommend that every new Replixa deployment runs in review mode for the first two weeks before going fully live. In review mode, the system classifies all tickets, generates resolution responses and action plans for high-confidence intents, but routes everything to an agent queue for human review before anything sends or executes.
Review mode is not a live deployment — it is a calibration pass. Agents review the queue and mark each prepared response as "approve," "edit," or "override." The approval rate after two weeks tells you exactly how well your escalation logic and confidence thresholds are calibrated. If 90%+ of prepared responses are approved unchanged, your configuration is in good shape and you can go live with high confidence. If the approval rate is 70% or lower, you have calibration work to do — usually KB gaps or threshold adjustments — before you should allow fully autonomous resolution.
Review mode also builds agent familiarity with the system before they are trusting it unsupervised. That familiarity matters for long-term adoption. Agents who saw the system's reasoning during review mode are much more willing to trust its autonomous decisions afterward, because they have already verified the quality of its judgment on real tickets from their own queue.