Auto-reply detection
This experiment describes the design, validation, and deployment of an automated response detection capability for Ada's AI Agents operating over email. Prior to this work, AI Agents responded to automated messages—including out-of-office replies, confirmations, and promotional emails—creating unnecessary replies and, in some cases, infinite conversation loops.
Experiments & Findings
Background
AI Agents frequently receive automated responses such as Out of Office replies, promotional content, notification and confirmation emails, and more. AI Agents were previously responding to such emails, which was unnecessary and also had the potential to create infinite loops between the agent and the end-user inboxes. Such loops resulted in conversations with thousands of turns that were difficult to detect and end, and significantly increased inference costs. Although we already had header-based checks in place to catch Out of Office replies, there are no standard headers amongst all email clients; therefore, we continued to have Out of Office replies occasionally slip through the cracks and create loops.
Automated Response Detection expanded on the Unified Reasoner's End Conversation tool to add the ability to detect and classify automated responses. When an automated response is detected, the AI Agent does not send a response; this prevents unnecessary responses and breaks any loops before they become problematic. The tool can also detect when conversations are not progressing, such as when the end user is sending multiple messages that do not contain any actionable information and are unrelated to a customer support inquiry.
This project also included the precursor step of implementing the End Conversation tool for email conversations; previously, the tool was only implemented for voice conversations. By implementing it for email, Ada’s reasoning engine is now able to determine when an email conversation has been resolved, with either a high or low confidence. When the end of an email conversation is detected with high confidence, the satisfaction survey can now be served earlier; previously, it was served on conversation expiry (72 hours of inactivity). Now the client can configure the satisfaction survey to be served immediately when the end of conversation is detected, after a configurable time-based delay, or when the conversation expires.
Architecture
Two architectural approaches were considered and evaluated for this project:
- Pre-processing step: Classify incoming messages before they reach the reasoner
- In-reasoner detection: Embed detection in the End Conversation tool call
A pre-processing step guarantees coverage (we can guarantee the step runs for each message), but sees each message in isolation. In-reasoner detection loses the coverage guarantee, but the model can use the full conversation history when evaluating.
The in-reasoner approach was selected for several reasons:
- The LLM can use the entire conversation history. This is especially helpful for detecting loops, sneaky promotional content, and instances where the conversation is not progressing.
- It doesn't require additional LLM calls, which keeps latency down.
- It uses the existing End Conversation tool, which allows us to consolidate conversation-ending instructions and behaviors in one place, and apply them to other channels and modalities where appropriate.
Classification types
After reviewing user conversations, we decided to add a classification to each detected automated response based on the types we frequently observed. These classifications are primarily used for internal monitoring and analytics and include the following categories:
- Out of Office
- Confirmation / Notification
- Promotional
- Bot (ie, another AI Agent talking to Ada’s AI Agent)
- Other (to catch any automated responses that don’t fall into the above categories)
Validation and datasets
Anonymized user conversations were collected to use for LLM testing and to guide our prompt design for detecting automated responses. The dataset includes examples for each automated response classification type, as well as low and high-confidence examples of email conversations that are ended.
To validate that our updated End Conversation tool was correctly detecting and classifying automated responses and end of conversation, we ran an experiment using an internal endpoint that accepts a list of past events (eg, previous messages from the user and/or AI Agent) and an incoming message, and feeds them into Ada’s reasoning engine. Each example in our dataset contains a past events list, the incoming message, as well as the results we expect to receive. We were able to validate that our reasoning engine consistently detects end of conversation and automated responses without false positives. This experiment can be re-run anytime there are changes to the prompt that may affect the behaviour of the End Conversation tool to verify that there are no regressions.
Results
Since launch, the updated End Conversation tool has been successfully detecting the end of email conversations, automated responses, and conversations that are not progressing. The tool is successfully detecting these instances even when they are outside of the examples we explicitly provide in the reasoner instructions; an example of this is an automated event invitation that was received by an AI agent. Approximately 8,000 automated responses are being detected per week.