Top 4 things to expect when adopting a new AI CX tool

As merchants quickly adopt AI to enhance their customer service experience, the prospect of choosing a new tool can be daunting for every CX leader. The field is very new, and every tool is so different that it's difficult to know exactly what to expect.

One thing is certain, though: Every single Shopify merchant will adopt AI for their customer service. For the rest of the unknowns, find out below the 4 things that you should keep in mind when evaluating and setting up your new AI tool:

1. AI is not magical (yet)

Some people might want you to believe that it will be zero effort and the AI will start automating everything after just a single click, but well… it won't. Sure, AI might be able to scrape your website and gather some already public knowledge, but what about learning about your internal policies? What about accessing all the data that a human agent relies on a daily basis?

Learning from historical tickets is also challenging for AI. How many policies or products have recently changed in you business? Which decision was just a one-off and which are not? Think of the actions that’ll be needed to reach true automation.

2. Integrating AI will force you to improve your internal processes

No matter which AI tool you use, it will only be as good as your processes. If you don't have clear and structured internal processes yet, you may want to start defining those before adopting AI for your customer service.

If your AI is not fed with properly structured and clear instructions on what to do for each possible scenario you want to automate, it simply won't be able to perform properly. Without the right instructions, the results will just end up being somewhat random. So, have a clear idea of your internal processes and, based on those, start automating with the most defined ones.

3. AI will make mistakes

AI will sometimes make mistakes; that’s just the reality today.  However, with a good AI tool, the error rate should decrease quickly as the AI rapidly increases its wealth of source truth. In addition, make sure that your AI tools has safeguard and policy control features. If so, configure those features properly!

Another important factor is to pay attention to the LLM model behind the AI tool. OpenAI GPT-3.5 can be up to 60 times cheaper than GPT-4! It’s also way less accurate and has a much higher error rate. For your customers, we recommend never to go with the cheap route. If the AI you are considering is fast and cheap, it means it’s using a low-cost model, and its accuracy won’t be good when encountering complex CX cases.

Nonetheless, even with the best model and the right guardrails in place, the AI might fail occasionally. That’s why it's important to be open to a few errors. An error rate below 2-3% is acceptable, as it's on par with human error rates. As long as those errors are recoverable and the AI is still bringing a lot of benefits overall, things should be fine.

4. Metrics are Key; Track Your Progress

As a CX leader, you likely spend a good chunk of your day looking at metrics. This will be even more true with AI, as you will have access to a bunch of new metrics. The North Star however should be the your Automation Rate. Track it either through your AI tool's dashboard or directly through your help desk. If provided by your AI tool, ensure that the rate is not misleading and is properly computed. It should represent the % of tickets that are truly fully automated without human intervention (For example, adding a ticket field, a tag, or simply routing to an agent should not be considered automated).

Over time, it’s crucial that Automation Rate keeps going up and to the right. To achieve sustainable success, it’s best not to be over ambitious at the beginning. Start with a reasonable goal of 15% after a month and 30% after 2 months. That number might seem small, but it will pay dividends quickly over the long run.

Conclusion

The AI for CX landscape is changing everyday and is moving at breakneck speed. As a CX leader, it’s challenging to stay calm, evaluate, and pick the right AI solution. It’s a jump into the unknown as this is a new software category, and no 2 solutions offer the same features and benefits.

New features and innovations are constantly emerging, but that’s not a reason to postpone your decision. One thing is certain: AI will be adopted by every single business out there in the next 18 months. The space is maturing fast and if you invest in it, you can already automate a good fraction of your tickets.

Stay tuned! Next up, we will write about why using CSAT to evaluate your AI is not a good metric and is totally unfair to your human team. Good luck and happy automation!

‍