Computer Vision and Deep Learning for Banking and Finance

---

Table of Contents

---

Computer Vision and Deep Learning for Banking and Finance

Banks are the financial backbone of any economy. As the primary credit supplier, they help businesses, institutions, and individuals by providing and managing short-term and long-term finance. In the last decades, banks worldwide have gone through a digital revolution that has laid the foundation for artificial intelligence (AI) adoption. As a result, financial institutions have started using AI to improve customer experience, manage portfolios and risk, enhance data security, and provide good loans.

According to Business Insider, around 90% of banks know the benefits of using AI in their operations and services. Moreover, many of them (about 75%) have started making huge investments in implementing and deploying AI-enabled solutions.

This series is about Computer Vision (CV) and Deep Learning (DL) for Industrial and Big Business Applications. In addition, this blog will cover the benefits, applications, challenges, and tradeoffs of using deep learning for banking and finance.

This lesson is the 3rd of the 5-lesson course: CV and DL for Industrial and Big Business Applications 101.

1. Computer Vision and Deep Learning for Government
2. Computer Vision and Deep Learning for Customer Service
3. Computer Vision and Deep Learning for Banking and Finance (this tutorial)
4. Computer Vision and Deep Learning for Agriculture
5. Computer Vision and Deep Learning for Electricity

To learn about Computer Vision and Deep Learning for Banking and Finance, just keep reading.

---

Computer Vision and Deep Learning for Banking and Finance

---

Benefits

Here are some benefits of using Deep Learning for banking and finance.

---

Cost Reductions

AI can help automate banking workflows and operations to help perform repetitive tasks more efficiently and accurately than humans. This makes the system more cost-efficient and less error-prone. Further, AI-based services can be used 24×7 to provide customer support and cut costs associated with human agents.

According to Business Insider (Figure 1), by using AI applications, banks will save around $447 billion by 2023. The front and middle offices will account for $416 billion. According to TCS research, “banking and FS executives found that investment in AI helped them reduce production costs by 13%. Additionally, executives reported a 17% average revenue increase in the area of their AI initiatives.”

---

Customer Support

AI in bank applications helps customers by addressing their queries even on weekends and holidays. AI-based virtual assistants and chatbots provide personalized content for credit reports, loan offers, payment alerts, fraudulent activities, financial summaries, and customer analysis (e.g., Bank of America’s Erica virtual assistant, see Figure 2).

---

Enhanced Security

Fraudulent activities are very prominent in banking and finance and are designed to fool humans. However, AI algorithms can detect these fraudulent activities with much higher accuracy. Further, these algorithms help banks ensure regulatory compliance by monitoring wire transfers, preventing illegal transactions, detecting laundered money, and perpetrating financial crimes by looking at customer behaviors and patterns.

---

Better Decision Making

AI helps banks make better decisions for providing safe and profitable loans and managing portfolios. Currently, banks use credit scores, credit history, and customer references to determine a client’s creditworthiness. AI can look at customer behaviors and patterns to determine if a customer with limited credit history might make a good credit customer.

---

Applications

---

Portfolio Management

One of the best advantages of AI in banking is portfolio management. In addition, advanced AI technologies can bring everything to our fingertips, removing the need for in-person banking.

AI algorithms can perform fundamental analysis on the text extracted from annual reports, news articles, Twitter posts, and economic reports. The algorithms can find hidden correlations between various asset classes and pinpoint stocks that can perform using a range of financial or firm-level variables. LASSO regression/framework, for example, can also investigate which domestic industry or market returns are the most significant predictors of returns among all other markets or industries.

The results of the fundamental analysis can optimize asset allocation in financial portfolios. For example, AI techniques often provide better estimates of returns and covariances than more conventional methods. Further, they can help construct portfolios that meet performance targets more closely than portfolios created using traditional methods. For example, genetic algorithms can solve complex optimization problems with constraints (e.g., restricting the number of assets or setting minimum holding thresholds, see Figure 3).

---

Risk Management

There are two types of risk management: market risk and credit risk. Market risk refers to the likelihood of loss because of market fluctuations. In contrast, credit risk refers to the risk of a counterparty not fulfilling its contractual obligations resulting in a loss of value.

Risk management is another domain where AI can excel. AI techniques can incorporate qualitative and quantitative data from news reports, online posts, and financial contracts to forecast risk variables, validate and use them to minimize the risk, and ensure that absolute risk is acceptable to clients (Figure 4).

Market Risk: Satellite images are being analyzed to predict sales at supermarkets or future crop harvests. Unsupervised AI approaches can detect anomalies in risk model output by evaluating all projections generated by the model, automatically identifying any irregularities, and supervising AI techniques to create benchmark forecasts as part of model validation practice. Comparing model results and benchmark forecasts will indicate whether the risk model produces predictions that differ significantly from AI’s.

Credit Risk: Asset managers need to monitor the credit risk of entire portfolios of individuals and their transactions. This practice involves modeling the solvency risk associated with institutions issuing financial products (e.g., equities, bonds, swaps, and options). Multivariate discriminant analysis, logit, probit models, support vector machines (SVMs), genetic algorithms, deep neural networks (DNNs), and their ensembles are widely used for credit risk modeling.

Financial institutions (e.g., J.P. Morgan, Bank of America, Morgan Stanley, and S&P Global) use Kensho, which offers analytical solutions using a combination of cloud computing and natural language processing (NLP). As a result, the system can provide answers to complex financial questions in plain English. Similarly, Ayasdi provides on-premise anti-money laundering (AML) solutions for enterprises to understand and manage risk.

---

Fraud Security

Banks and financial institutions are highly prone to fraudulent activities. Hence, it becomes essential to have a secure and robust system to protect the interests of their customers. Therefore, financial institutions must closely monitor potential fraudulent schemes, including phone/SMS fraud, illegal remittance using online banking, and illegal stock transactions. Most common fraud activities include:

• Unauthorized Transactions: Transactions where the customer does not approve nor is unaware. According to a Forbes report, roughly 8 in 10 mobile banking users are concerned about credit card fraud.
• Phishing Scams: There were more than $54 million in phishing scams that year through email and SMS messages.
• Identity Theft: According to Javelin’s 2021 Identity Fraud Study, total financial losses from identity fraud in 2020 were around $13 billion.

Most fraud detection systems use rule-based algorithms to trigger alerts in conditions such as “amount paid” and “number of deposits.” However, such vague rules can lead to false positives and unnecessary alerts, increasing staff workload. To improve efficiency, AI can learn from the past judgments made by the fraud monitoring staff and automatically filter out false positives based on similar patterns. These algorithms consider variables such as transaction amount, time, card use frequency, IP address of purchase, etc. (Figure 5).

Vectra is an AI-powered platform that automates threat detection, revealing hidden attackers explicitly targeting financial institutions. Shape Security, another AI-powered service, provides security against credit application fraud, credential stuffing, scraping, and gift card cracking by pinpointing fake users.

---

Regulatory Compliance

Banks and financial institutions must follow the compliance rules created by supervisory organizations in the banking sector. Noncompliance with these rules can lead to fines and loss of banking license. In addition, compliance rules are subject to frequent change, during which banks need to change and update their workflows to comply with the new regulations. During this period, they are also vulnerable to cyber and fraud attacks.

AI-based software can help banks identify which compliance rules are relevant and what workflow and services are affected and need updating. In addition, NLP can keep the financial institution and the client up-to-date with regulatory changes by analyzing and classifying documents and extracting helpful information like client information, products, and processes that can be impacted by the regulatory change (Figure 6).

Quill offers an NLP-based service that automatically selects a data chart and generates a few sentences to explain the insights. Quill-generated Suspicious Activity Reports (mandatory reports that banks and financial institutions need to file with supervisory bodies) are robust enough to ensure a bank remains compliant.

Conduct Surveillance, another NLP tool, can analyze human-agent conversations that have been recorded to determine if bank employees are acting within compliance when they interact with clients.

---

Loan and Credit Decisions

Providing safe, creditworthy, and profitable loans is essential for the functioning of banks. Currently, the banking industry is too confined to using variables like credit scores, credit history, and customer references to determine the creditworthiness of a customer seeking a loan. However, using AI and machine learning (ML) models, banks can leverage other non-traditional sources like bank transactions, investments, and tax returns to better determine a customer’s creditworthiness, loan limits, and pricing (Figure 7).

Credit Qualification: For many years, rule-based algorithms (e.g., logistic regressions) have been used to determine a customer’s creditworthiness. However, these days giant banking and financial institutions have started building complex models capable of analyzing structured and unstructured data collected from social media, browsing history, telecommunications, etc. The end-to-end automated system provides the likelihood of a person defaulting. In addition, banks can refine their qualification model by combining the computerized system with manual reviewing (for high-risk credit).

Limit Assessment: AI and ML algorithms can determine the maximum amount a customer can borrow. Systems leverage income statements, tax returns, monthly and annual spending, and customer investments to determine a customer’s disposable income and capacity to make regular loan payments.

Pricing: Banks efficiently decide the interest rates based on creditworthiness and the maximum loan limit of a customer. This helps them offer competitive rates, keep their risk costs down, and optimize the balance of total asset volume, risk, and interest income within a lending portfolio.

Platforms like Enova and Ocrolus are utilizing AI and ML to analyze individuals’ bank statements, pay stubs, tax documents, mortgage forms, and invoices and provide advanced analytics to businesses and banks to facilitate secure lending.

Kenya’s mSurvey, a startup, is using mobile phone applications to collect requisite data needed to feed and build real-time profiles for customers. AI and ML algorithms can easily use this real-time to determine their creditworthiness.

---

Process Automation

Everyday banking tasks include:

• filling out forms (e.g., credit card, application, bank opening, etc.)
• clearing checks
• processing Know Your Clients (KYCs)
• clearing customer doubts, etc.

Using AI-based services, employees can efficiently carry out mundane and time-consuming tasks. For example, banking applications frequently ask clients to take a selfie and provide an ID card to perform KYC verification. An AI algorithm behind the application checks if the selfie matches the ID card’s photo while verifying that the ID is not fake.

Arya.ai (Figure 8) automates check-processing and clearing using 6 different AI models for various fields, including date, magnetic ink character recognition (MICR), the amount in words and figures, account number, and signature extraction and verification. It can process 15-20 checks per second on standard production infrastructure.

The Adobe Automated Forms Conversion service uses deep learning to convert PDF forms to device-friendly, responsive, and HTML5-based adaptive formats to eliminate paper-based forms and make form completion more interactive.

---

Challenges

---

AI Bias

AI bias can arise when humans bring their assumptions while reading/annotating data or building the machine learning pipeline. A biased model can seriously impact the credit decision-making capability by biasing its prediction based on socio-economic factors. An intuitive way to bias the systems is to strip such variables from the data. However, as you reduce data quality, performance tends to drop.

Regularizing the algorithm is another approach that aims to penalize model parameters if they treat the minority or protected classes differently. For example, as a regularization, one can use the unfairness score, which measures the gap between different outcomes for people belonging to other classes and risk profiles.

---

Explainability

Explainability is essential for financial institutions as they must provide the logical rationale behind their credit-issuing decisions to potential customers. For example, imagine a person receiving a bad credit score and getting his loan declined when he needs it the most. Just saying that the ‘The computer did it’ doesn’t sound good. Furthermore, an explainable model can help the bank agents understand if the model’s decision is biased because of socio-economic factors. Sometimes, using large and complex models to get a few extra percentages of points is not a good idea as they have poor explainability.

---

Customer Mistrust and Data Security

Customer mistrust is the biggest challenge in adopting AI in the banking sector. Everyone has a conservative side toward their hard-earned money, and giving them access to a black-box algorithm doesn’t seem like the right choice to many people. Any mistakes in the management of risk and portfolio can cost customers. Further, any breach in the AI system can leak confidential and sensitive data about the customers.

Figure 9 mentions another challenge of using AI in the banking sector.

---

---

Summary

Financial institutions have started using AI in the following ways to make their systems more cost-efficient, less error-prone, and more secure:

• Portfolio Management: AI algorithms can perform fundamental analysis on the text extracted from annual reports, news articles, Twitter posts, and economic reports. This analysis is also used to optimize asset allocation in financial portfolios.
• Risk Management: AI techniques can incorporate qualitative and quantitative data from news reports, online posts, and financial contracts to forecast risk variables, validate and use them to minimize the risk, and ensure that absolute risk is acceptable to clients.
• Fraud Security: AI can learn from the past judgments made by the fraud monitoring staff and automatically filter out false positives based on similar patterns.
• Regulatory Compliance: AI can identify which compliance rules are relevant and what workflow and services are affected and need to be updated. NLP can extract helpful information that can be impacted by regulatory change.
• Loan and Credit: AI and ML models can leverage non-traditional sources like bank transactions, investments, and tax returns to better determine the creditworthiness, loan limits, and pricing of a customer.
• Process Automation: With AI-based services, employees can efficiently carry out mundane and time-consuming tasks like completing forms (e.g., credit card, application, bank opening, etc.), clearing checks, processing KYCs, clearing customer doubts, etc.

However, using AI in banking applications comes with its challenges:

• A biased model can seriously impact the credit decision-making capability by biasing its prediction based on socio-economic factors.
• Explainability is essential for financial institutions as they must provide the logical rationale behind their credit-issuing decisions to potential customers.
• Everyone has a conservative side toward their hard-earned money, and giving them access to a black-box algorithm doesn’t seem like the right choice to many people.
• Any breach in the AI system can leak confidential and sensitive data about the customers.

I hope this post helped you understand the benefits, applications, challenges, and tradeoffs of using deep learning in banking and finance. Stay tuned for another lesson where we will discuss deep learning and computer vision applications for agriculture.

---

Citation Information

Mangla, P. “Computer Vision and Deep Learning for Banking and Finance,” PyImageSearch, P. Chugh, R. Raha, K. Kudriavtseva, and S. Huot, eds., 2022, https://pyimg.co/up7ad

@incollection{Author_2022_BF,
  author = {Puneet Mangla},
  title = {Computer Vision and Deep Learning for Banking and Finance},
  booktitle = {PyImageSearch},
  editor = {Puneet Chugh and Ritwik Raha and Kseniia Kudriavtseva and Susan Huot},
  year = {2022},
  note = {https://pyimg.co/up7ad},
}

---