Customers often ask more than one question at the same time, for example, "Do you ship to Alaska? How long would it take?"

Even though this is technically one inquiry (or ticket), it contains multiple intents:

1. Do you ship to Alaska?
2. How long would shipping to Alaska take?

The AI agent should be able to understand this and craft an answer that addresses both intents.

Without understanding multiple intents, the AI agent will have to ask the customer to clarify or simplify their question. The result is unnecessary friction leading to frustration.

Ask about two different topics in the same question. The AI agent should be able to detect that and answer both.

It's natural for customers to phrase things in different ways, for example, "what's the status of my order?" vs "when will my package arrive?"

The AI agent should be able to understand the intent and provide a consistent resolution to the same issue, even if it's expressed in different ways.

The advantage of AI agents over scripted chatbots is precisely that you don't need to get caught up in precise wording, or spend time on tedious training. A consistent AI agent increases your speed to value and provides a smoother customer experience.

Ask the same question in two different ways. The AI agent should give you a consistent answer. The exact wording of the AI agent's answer doesn’t need to be an exact match as long as the information is correct.

The AI agent understands the context of the conversation and correctly answers follow up questions.

Contextual understanding is important for conversations to sound natural and organic. The customer doesn't need to repeat themselves, which improves the service experience and reduces the number of conversations that are contained but not resolved (i.e. the customer gets frustrated and leaves without escalating to a human agent).

Ask a question and let the AI agent respond, then ask follow up questions. The AI agent should be able to respond to them in context of the whole conversation.

The AI agent has general conversational skills and can address questions unrelated to any specific topic in a similar way to a human agent.

Some customers might reflexively make small talk, or ask questions that don't have a specific help article. Conversational skills are essential to making the AI agent perform at the same level as a human agent would.

Ask simple questions that are related to the product or service but would not have a specific answer prepared for them. The AI agent should be able to respond conversationally.

The AI agent can answer questions, perform actions, and provide the same conversational experience regardless of which product line your customers are asking about, which channel they reach out on, or what time of day it is.

With an AI agent, you don’t have to build or train rigid flows. It learns from your knowledge and scales across every channel without extra effort or cost.

Ask about different product lines or try the chat outside of business hours. The AI agent should provide the same conversational experience and still be able to perform the same actions.

Returning relevant answers means that the AI agent effectively understands the customer’s inquiry, and provides directly-related information or assistance.

Without this skill, the AI agent might be only generating truthful facts that are unrelated to the customer’s inquiry, and doesn't really resolve anything.

Ask an unusual question. The AI agent should still be able to give you a relevant response.

Before the AI agent can resolve anything, it must first truly understand what the customer is asking about. Only then can it perform reasoning and find the best resolution.

Without semantic understanding, the AI agent would be a glorified keyword search engine that doesn’t resolve anything, and you’d have deployed an expensive search bar.

Ask a question that includes a potentially tricky keyword, such as “can I get a product for free?”. The AI agent’s answer should still be relevant.

When asked a question, the AI agent provides a specific answer, not just a link to the relevant help article.

What’s the point of chatting with an AI agent if all they’ll do is send you to the help center articles? That’s not what we call a resolution, and it’s a complete waste of the money you’re spending on it.

Ask a question that has a help doc article. The AI agent should provide an actual, conversational answer instead of simply returning a link to that article.