As generative AI technology becomes more widespread and accessible, it makes sense that enterprises are curious about the possibility of building an AI agent for customer service in-house. After all, who knows your business and your customers better than your own team?

The reality is not that straightforward.

Before diving into the development of an in-house AI solution, it is crucial to weigh several key factors against the benefits of working with established AI Customer Experience (ACX) platform partners. By understanding these factors, you’ll be better equipped to make an informed decision that aligns with your organization's needs and resources.

1. What is the total cost of ownership (TCO) for building an AI customer service agent?

How much does it actually cost to build and maintain an AI customer service agent? And how does that stack up against paying an annual fee to a third party vendor? Let’s tally it up.

The secret cost of staying up to date

The rapid pace of AI innovation makes it increasingly difficult for enterprises to keep up without continual reinvestment. What’s cutting-edge today can be obsolete in just a few months, forcing you to constantly evaluate and implement new technologies or risk falling behind.

As we move closer to artificial general intelligence (AGI), AI systems will not only evolve in complexity but also demand more nuanced applications.

Businesses that have built their own AI customer service agents will quickly discover the hidden costs of staying up to date.

This happened before with scripted conversational AI. Enterprises that built their own in-house solutions are now having to start from scratch, reinvesting in development, integrating new tools, and scaling for omnichannel service—all of which are essential to ensure customer satisfaction and prevent outdated tech from undermining the customer experience.

And those who did reinvest and built their own AI agents are still experiencing surprise costs. Evolving LLM models open up new opportunities but require new skill sets and architectures to take advantage of, many of which haven't been invented yet.

This doesn't stop at big transformations like that. For example, if you want to introduce new integrations to automate different parts of your customer experience, your development and engineering teams will need to manually set up those integrations, test them, and deploy them.

So as you’re tallying up the costs, don’t forget to factor in the longevity of the solution, and what kind of resources you’ll require to stay up to date.

2. What ROI can enterprises expect from AI customer service automation?

Now that we’ve broken down the costs, let’s talk about the results. Assuming both options are working comparably and delivering a similar automated resolution rate, which one is giving you a better bang for your buck?

How long does it take to get to production for build vs buy (time-to-value)?

With an investment as high-value as building an AI agent for customer service, there are a number of time-consuming milestones between getting approval and deployment.

At the very least, you’ll need to identify the key stakeholders, gather the requirements for the solution, hire the team or create a task force in your existing departments, develop the solution, test and iterate to improve it, train the teams on using it, and finally roll it out.

Think about how much time each of those steps take, and what kind of effort is involved in the details of those steps as well.

How much time to recruit the talent if you don't have it in house? How many people are involved in the process? Once the team starts working, how much time will they need to ramp on internal processes?

Think about it this way, if you spend 6 months building a solution in-house, that’s 6 months where you're not automatically resolving support inquiries and losing out on savings while you build.

By the time the AI agent is live, you could have already broken even on the initial cost and started seeing positive returns with an ACX platform.

Opportunity cost

Beyond the financial investment, companies must also consider the opportunity cost of building an AI agent for customer service. While it's true that every company should consider innovating with AI, the focus should be on areas that directly align with core business objectives and growth.

For example, a a FinTech like Tilt would gain more by developing AI applications that enhance portfolio performance, or a SaaS like Monday.com could better serve its mission by focusing on AI that makes workflows more efficient.

It's also important to evaluate what critical projects or teams will be impacted by reallocating resources to AI development. Will your recruitment team now have to shift focus towards finding specialized AI talent, or will you outsource to agencies, incurring high costs just to keep your internal teams focused on core functions?

Diverting resources to build an AI customer service agent can be a distraction from key business priorities.

3. What are the biggest risks of building an AI agent without an evaluation framework?

Hallucinations, regressions, reliability, maintenance burden... Building an AI agent is one thing, building the evaluation framework to make sure it's actually working is something else.

Testing

It’s easy enough to build a demo that looks good, but once you scale to a larger conversation volume or more diverse customer inquiries, things get complicated.

Before you deploy the AI agent to your customers, you need a testing environment to run conversation simulations, especially for sensitive inquiries that need to adhere to compliance policies. Otherwise, you risk exposing your customers to a potentially faulty AI agent and you have to deal with the consequences of any problems that might cause.

To avoid this, you need to have thoughtful, structured evaluation frameworks for your system in place.

Insights and analytics

It should go without saying that deploying an AI agent is useless if the AI agent doesn’t work. Or worse, works poorly.

To make sure that the AI agent is delivering exceptional customer experiences, you have to monitor its performance, audit conversations, and analyze insights to keep improving AI customer service.

4. What does ongoing ops look like (monitoring, QA, continuous evaluation, incident response)?

A large component of the success of the AI agent is how well your team is managing and coaching it. Your AI Managers need tooling to identify gaps and take the necessary steps to resolve more types of inquiries. Let’s take a look at some considerations for continuous improvement.

5. If we build in-house, what’s the one thing we won’t realize we're missing until it’s too late?

The big answer is: expertise.

Max Ball, Principal Analyst at Forrester, says it best: “If I just get Google or I get OpenAI or something else, I can build a self-service application, but there's just so many things that can go wrong. There's guardrails that need to be put in place. You need to watch out for hallucinations. You need to understand response times.

It's very important that you use a platform that takes generative AI and puts it into a context of AI for customer service. You need the right set of tools to do that. You need people who've been thinking about the guardrails, who've been thinking about the problems and living that stuff for a significant chunk of time. You need somebody who lives and breathes this stuff, who understands it.”

AI for customer service

If you were building a large-scale LLM-powered AI agent that your customers will interact with, you’ll need expertise in AI, customer service, and AI for customer service.

It’s essential to work with partners who have been immersed in this world for years. Not only do they understand your pains and how to solve them, they also have working relationships with software and technology companies that give their AI agents an edge.

ACX program expertise

In addition to technology expertise, ACX platform providers have worked with enterprises across various industries to launch and grow ACX programs. They have tried and tested methodologies and best practices that you can benefit from to truly transform customer service, beyond simply automating FAQ.

For example: onboarding and education processes to upskill human agents, leading to higher implementation success rates and better value for the business.

6. When does building an AI customer service agent actually make sense

While it may seem that there’s a strong bias towards working with an ACX platform partner, there are scenarios in which it actually makes more sense to build an in-house AI agent.

You need a highly specialized AI agent

If you need an AI agent to take actions that are truly unique to your enterprise, you’re likely to get more value by building it yourself. While a partner platform can definitely build out specialized use cases, you would be bound by their timelines.

You need complete control over the AI agent development roadmap

In the same vein, the biggest benefit of building an in-house AI agent is that you can customize it entirely to fit your enterprise’s exact needs. Perhaps your entire customer base prefers a specific communication channel, so you don’t care about omnichannel capabilities and instead want to further optimize that one channel. Or perhaps you have multiple highly specialized customer intents that make up the majority of your conversation volume.

Whatever the case may be, building your own AI agent gives you the ability to prioritize the skills that make the biggest impact, and if the impact is big enough, it’ll offset the in-house total cost of ownership.