Computer Vision and Deep Learning for Customer Service

---

Table of Contents

---

---

Computer Vision and Deep Learning for Customer Service

To stand out in today’s competitive market landscape, it’s essential to retain your customers by providing the best products and services. According to a 2017 Microsoft report (Figure 1), around 55% of customers have stopped doing business with a branch because of poor customer experience.

Artificial Intelligence (AI) offers cost-efficient solutions to enhance business processes and deliver more satisfying customer service. The integration of AI-based APIs with mobile or desktop applications is further made easy by cloud services (e.g., Google, AWS, and Azure), accelerating its adoption in big businesses and industries.

According to Zoominfo, 80% of sales and marketing leaders say they already use artificial intelligence, particularly chatbot software, in their customer experience or have done so by 2020. Juniper Research, for its part, predicts chatbots will be responsible for cost savings of over $8 billion annually by 2022.

This series is about CV and DL for Industrial and Big Business Applications. In addition, this blog will cover the benefits, applications, challenges, and tradeoffs of using deep learning for customer service.

This lesson is the 2nd in the 5-lesson course: CV and DL for Industrial and Big Business Applications 101.

1. Computer Vision and Deep Learning for Government
2. Computer Vision and Deep Learning for Customer Service (this tutorial)
3. Computer Vision and Deep Learning for Banking and Finance
4. Computer Vision and Deep Learning for Agriculture
5. Computer Vision and Deep Learning for Electricity

To learn about Computer Vision and Deep Learning for Customer Service, just keep reading.

---

Computer Vision and Deep Learning for Customer Service

---

Benefits

Here are some of the benefits of using Deep Learning for customer services.

---

Offer Superior Personalization

AI algorithms can assess past interactions with a prospect and use this valuable information to provide personalized service to the customers. This keeps the customers engaged, feel listened to, and more valued. If a customer walks away delighted, it will most likely increase the business’s sales and brand value.

On the contrary, conventional non-AI-based algorithms can’t achieve that level of personalization. The agent would have to copy details between systems and make the experience interactive manually in most cases. Such a process frequently leads to errors and lost data.

---

Faster and Cost-Effective

One important factor to consider while aiming for better customer service is wait and response times. If there’s one thing that customers universally hate, it’s being made to wait – especially when they need help. Agent-based customer services often have high wait times since the workforce is limited. This can also lead to delayed response times.

With the increasing effectiveness of cloud-based AI algorithms, customer queries can now be addressed instantly. In addition, natural language processing (NLP) tools can help the system understand the language and solve the issue more quickly and effectively than a human agent and save costs.

According to Juniper Research (Figure 2), a chatbot interaction saves 4 minutes and $0.70 for the banking and healthcare sectors.

---

Improve Customer Analytics

The machine learning algorithm pulled data used to make future predictions when a customer needs assistance. This ensures that they continue along the sales process without any complications.

Furthermore, the algorithm can identify trends that can be leveraged in personalized ad campaigns. For example, if you know that 85% of customers who bought your floral vases are over 55, you can start marketing those vases to those over 55s and hopefully see the profits increase.

---

Applications

---

Email Support

Emails can take a lot of time to read and understand what the customer wants and how the company can help them. Further, if the email is not sent to the correct person/authority, it will have to be manually forwarded to the proper authority, which can further delay the response.

AI can significantly reduce the response time by scanning and tagging the emails to the correct authority or the person. It can also provide automated suggestions, references, solutions, etc., to write a proper draft for answering customer inquiries. As the AI algorithms evolve and develop the ability to learn from larger datasets, they can also be used to fetch some part of email drafts for people working in a call center.

DigitalGenius (Figure 3) provides this solution as their arsenal’s main customer service product. Their technology can scan and tag emails and send them to the correct person. This offers humans similar inquiries solved in the past to make the responses even quicker and more effective.

Magoosh, the student prep company, has improved its customer service team with the DigitalGenius solution. Magoosh has reduced its response time to client queries by nearly 50%. Now they can respond to every client within 24 hours. In addition, DigitalGenius states that over 83% of emails could be sorted and tagged by their solution.

Uber’s Customer Obsession Ticket Assistant (COTA) (Figure 4) provides the most accurate solution to the thousands of tickets surfacing daily on the platform across over 400 cities worldwide. COTA “understands” the ticket through natural language processing and then routes the ticket to the proper team.

Machine learning algorithms determine the top three ranked solutions for the human customer agent, who then picks the recommended solutions they think are the most feasible. This is the solution suggested to the customer. According to Uber, better ticket routing, thanks to COTA, increased efficiency by 10%.

---

Chatbots and Assistants

Chatbots are placed on company websites. Companies can use applications to address simple and common customer queries any time of the day (beyond working hours, too, when official reps are offline). Bots equipped with AI-based algorithms can understand the natural language and share appropriate responses or route customers to appropriate authorities if it doesn’t have the answer.

The good part is that the chatbot has such interactions. As a result, its response will be more accurate. In addition, the feedback received from the customers allows the chatbot to improve its performance.

IBM Watson Assistant (Figure 5) is a low-code visual builder that allows anyone to build powerful AI bots without writing code. Apart from answering just questions, the assistant can seamlessly integrate with existing customer service to solve real problems (e.g., purchasing, scheduling, etc.). Watson Assistant is also smart enough to remember previous customer support interactions and draw insights from previous behaviors.

Optimum, an internet, tv, and mobile provider, uses a chatbot-based service to process the words customers have sent and extract keywords that help it understand how to best help its customers.

Digital assistants (e.g., Cortona, Google Assistant, Siri, Alexa, etc.) provide a more personalized experience and should not be confused with chatbots. While chatbots simulate interaction with an agent, virtual assistants focus on specific areas in the customer journey to aid the customer. In addition, virtual assistants can respond more humanely by using natural language processing.

---

Speech Recognition

Audio calls or voice-based assistance is needed when queries are complex enough not to be solved via emails or texts. In addition, different accents, native languages, noises, bad pronunciation, and unknown speech patterns can make voice assistance challenging.

Improved speech recognition AI algorithms in call center management and call routing can allow a more seamless experience for customers — and more efficient processing. ​​Deep learning algorithms can analyze the audio to predict customers’ emotional tones. If a customer responds negatively to an automated system, the call can be rerouted to human operators and managers.

For example, Cogito (Figure 6) has used behavioral science and deep learning technology to build a tool that can analyze conversations in real time. Their algorithm is aware of the tone and content in the dialogue. The tool provides insights into customers’ feelings based on changes in volume and pitch and mimicking detection.

This information can help human representatives get additional information, tips to improve call quality, and feedback about their performance. According to Cogito, the callback rate has lowered by 10%, and customer satisfaction has grown by 28%.

---

Personalization

AI can use real-time data to deliver personalized content, product, and services and hence provide added value to the customer. Enabling personalization can help businesses increase their revenue as the customers are likely to purchase their products and services that are valuable to them. Further, it helps in customer retention by offering every customer a unique journey and experience.

According to an Accenture report, 41% of U.S. consumers abandoned a brand due to a lack of personalization and trust. However, personalization can be useful in many areas:

Product recommendations: Amazon uses machine learning to recommend products to its customers that they are likely to enjoy. This provides customers with a personalized experience and contributes to a delightful experience. The algorithm uses past customer orders and their browsing history to suggest recommendations.

Location-based product recommendations: Home Depot uses AI to offer localized design trends and products based on shoppers’ locations. For example, customers on the West Coast receive a different set of recommendations than those on the East Coast.

Artwork personalization: Netflix uses artwork personalization (Figure 7) algorithms to show personalized visuals for each movie or series title. For example, those who tend to watch movies of a certain genre or actor might be shown an image reflecting that genre or actor.

---

Analytics

The huge amount of data collected via customer interaction with the services mentioned above can be used to draw useful insights and patterns. Companies use machine learning to find trends in the data, discover customer interests and actions, and analyze them to make the necessary adjustments to their services and products to serve their clients better. According to Gartner, by 2040, data analytics will be used in 40% of customer experience projects.

For example, Air Canada used machine learning algorithms to look at the thousands of conversations with customers initiated during online booking alongside the user sessions stored on their server. As a result, the company could identify issues with their online booking platform and the most buggy devices and browsers by looking at customers’ complaints. Because of this, they were able to prioritize and address these issues quickly and save on labor costs for customer support.

Easyjet used customer data to suggest to its customers where they would like to visit next based on their past bookings and trips. Twiddy, a vacation rental company, makes pricing recommendations to homeowners by analyzing seasonal trends and the size/location of the home. They also analyze how rental volume and demand shift from week to week, thus providing its customers with helpful and actionable information.

Microsoft’s Dynamics 365 Customer Service Insights (Figure 8) uses advanced AI in NLP to improve customer satisfaction by helping agents and customer service managers make better decisions. It provides insights into customer satisfaction – boosting analytics and AI-powered features to spend less time searching and more time engaging with customers.

---

Challenges and Tradeoffs

Using AI in customer service applications comes with its challenges (Figure 9):

---

Integration Issues and Maintenance

Integrating AI-based tools with a company’s existing infrastructure can be challenging. The daunting task of collecting historical data, hosting complex cloud technologies, the high maintenance cost, and the need to make them smart enough to learn from feedback and an ever-changing environment make the overall process complex.

---

Unemployment and Upgrading

Job roles need to change since these AI-based tools will take most work out of the agent’s hands. This may involve agents learning new skills to take over more tech-driven tasks instead of their usual handling of phone inquiries. As a result, many investments have to be made to provide proper training and conduct hands-on workshops. This may not agree with some employees, leading to unemployment or reduced salaries.

---

Data Security

The AI systems are trained on a huge amount of confidential data. With the automation of customer services, any sudden breaches in the system can lead to the leakage of sensitive customer information. This can be harmful to the customers and the company. In legal cases, charges are filed against it.

​​Implementing an online system powered by AI that conducts security practices through PCI (Payment Card Industry) compliance can help aggregate the information from thousands of transactions and detect fraudulent activity beforehand. As a result, these payments can be eliminated before they happen, minimizing the risk of compromising customer data.

---

Lack of Emotion and Creativity

AI ≠ Human. Even though AI is trained to be human-like, it will never be the same. It will be as good as its data and will always lack human emotion and creativity. Not all customer queries are objective. Some are subjective, which involves creative thinking that only humans are capable of. Making an AI-powered customer service will involve making them smart enough to address novel and out-of-the-box queries. But, AI lacks this skill because everything in its system is wired to follow a guidebook.

---

---

Summary

Artificial Intelligence offers cost-efficient solutions to enhance business processes and deliver more satisfying customer service. The data machine learning algorithm pulled from the data can be used to make future predictions when a customer needs assistance. Some of these applications include

• Email support systems can be integrated with an AI-based ticketing service that automatically scans and tags the customer query mails to be directed to the right department or person.
• Chatbots and virtual assistants can be used on the company’s website and applications to solve and address general customer queries and troubleshooting.
• Speech recognition systems in call center management and call routing can allow a more seamless experience for customers — and more efficient processing. ​​These systems can analyze the audio to predict customers’ emotional tones.
• Personalized content, products, and customized and unique services can help businesses increase revenue as customers are likely to purchase products and services valuable to them.
• Analytics can be run on data collected via customer interaction and can be used to draw useful insights and patterns. Companies use machine learning to find trends in the data and discover customer interests and actions.

However, using AI in customer service applications comes with its challenges:

• Collecting the historical data, hosting complex cloud technologies, the high maintenance cost, and the need to make them smart enough to learn from feedback makes the overall integration process complex.
• Job roles need to change as agents need additional training and workshops to take on more tech-driven tasks.
• A sudden security breach in the system can leak confidential and sensitive information about customers.
• Making services smart enough to address novel and out-of-the-box queries is another challenging task.

I hope this post helped you understand the benefits, applications, challenges, and tradeoffs of using deep learning in customer service. Stay tuned for another lesson where we will discuss deep learning and computer vision applications for banking and finance.

---

Citation Information

Mangla, P. “Computer Vision and Deep Learning for Customer Service,” PyImageSearch, P. Chugh, R. Raha, K. Kudriavtseva, and S. Huot, eds., 2022, https://pyimg.co/0r254

@incollection{Mangla_2022_CustomerService,
  author = {Puneet Mangla},
  title = {Computer Vision and Deep Learning for Customer Service},
  booktitle = {PyImageSearch},
  editor = {Puneet Chugh and Ritwik Raha and Kseniia Kudriavtseva and Susan Huot},
  year = {2022},
  note = {https://pyimg.co/0r254},
}

---