Computer Vision and Deep Learning for Transportation

---

Table of Contents

---

Computer Vision and Deep Learning for Transportation

Transportation is an essential part of our day-to-day life as it enables the carrying of goods from one place to another, trade, commerce, and communication to establish a civilization. The transportation sector has seen multiple revolutions over the past hundred years. Today we are at the stage where a significant breakthrough in transportation is achieved through Artificial Intelligence (AI).

AI is already changing the transportation industry by enabling cars, trains, ships, and airplanes to automate autonomously, making the traffic flow smoother. Besides making our lives easier, it can provide a safer, cleaner, smarter, and more efficient transportation mode for everyone. AI-led autonomous transport could, for instance, help reduce human errors involved in many traffic accidents.

The global market for transportation-related AI technologies reached $1.2 to $1.4 billion in 2017. Further, it is expected to grow to $3.5 billion by 2023, at a compound annual growth rate (CAGR) of 12-14.5% (Figure 1).

Further, the accessibility and affordability of high-end computation hardware (e.g., CPUs and GPUs) and IoT technologies (e.g., LTE and 5G) have created the possibilities for several applications of AI in the transportation sector. This series is about CV and DL for Industrial and Big Business Applications. This blog will cover the benefits, applications, challenges, and tradeoffs of using deep learning in the transportation sector.

This lesson is the 2nd of the 5-lesson course: CV and DL for Industrial and Big Business Applications 102.

1. Computer Vision and Deep Learning for Oil and Gas
2. Computer Vision and Deep Learning for Transportation (this tutorial)
3. Computer Vision and Deep Learning for Logistics
4. Computer Vision and Deep Learning for Healthcare
5. Computer Vision and Deep Learning for Education

To learn about Computer Vision and Deep Learning for Transportation, just keep reading.

---

Computer Vision and Deep Learning for Transportation

---

Benefits

---

Safety and Reliability

Human errors (e.g., speeding, distraction, and drunk driving) are responsible for more than 90% of road accidents. Hence, safety and reliability are arguably the most important factors for one working in the travel or transportation industries. Passengers need to know that they and their belongings are safe and that the vehicles or operators they travel with are reliable.

With the advent of AI technologies, safety levels can reach higher peaks. The ability of AI to analyze large quantities of data will allow travel and transport operators, as well as eventually the public themselves, to schedule public and private transportation services in a significantly improved manner. Researchers believe that AI-enabled vehicles could reduce traffic fatalities by up to 90% by 2050 in some developed countries.

---

Efficiency

Developing countries rank lower on the Logistics Performance Index (LPI) than developed countries because of a lack of efficiency due to inadequate infrastructure and poor customs procedures. AI will undoubtedly enhance the energy efficiency of transport systems which is an essential aspect of travel. In particular, AI can be helpful in e-logistics, where Internet-related technologies are applied to the supply and demand chain to match shippers with delivery service providers.

---

Pollution

Emissions from transport vehicles are significant contributors to increasing pollution and global warming. With the world becoming increasingly environmentally focused, a drastic reduction in the emission of the travel and transport industry is required to secure their long-term sustainability. AI can help in deploying new and innovative solutions to deal with increasing pollution by enabling scientists to discover more environmentally friendly methods of running vehicles and machinery for travel. McKinsey has predicted that self-driving trucks will reduce operating costs by some 45%. As a result, the environmental impact will also be greatly reduced (Figure 2).

---

Applications

---

Road Transport

Road transportation is one of the most significant areas where current AI is extensively applied. Worldwide, automotive manufacturers and technology firms are exploring various AI technologies and algorithms to develop intelligent vehicles for commercial and personal use. Such vehicles use a variety of sensors like GPS, camera, and radar in combination with actuators (devices that transform an input signal into motion), control units, and software to perform intelligent actions (e.g., self-driving).

---

Truck Platooning

Truck platooning refers to coupling several heavy goods vehicles (HGV) within a minimal distance to allow them to accelerate simultaneously or brake (Figure 3). A human drives the lead HGV, and drivers following in HGVs may only be present in case of complex traffic situations or unexpected incidents. It is expected that the responsibilities of the drivers in following HGVs will reduce progressively with the intervention of AI.

---

Traffic Management

Currently, AI algorithms are extensively used in vehicles to improve the traffic flow in the economy. For example, the short-distance ridesharing platform Uber uses AI techniques in all aspects of its services, from matching riders and drivers to route optimization. AI algorithms are also being applied in road traffic management to analyze the traffic pattern and volume and provide drivers with information on the fastest route. This relieves them of any traffic congestion and decreases their commute time. Such algorithms can also keep the traffic flowing via traffic signals and lights that rotate in real time to meet on-the-ground traffic flow demands.

For example, Surtrac from Rapid Flow Technologies (Figure 4), a Carnegie Mellon University spinoff, has coordinated a network of nine traffic signals on three major roads in Pittsburgh to install traffic control solutions. Their solution has reduced travel times by 25%, and wait times declined by an average of 40%. Vehicle emissions have also been reduced by 20%.

---

Aviation

The aviation industry is not new to the adoption of AI technology. However, with recent advancements in AI, the industry will significantly impact how its business is conducted. Managing the increasing air traffic is an important area where progress in automation and computing power can be helpful. The International Air Transport Association (IATA) report points out that developing unmanned aircraft systems (UAS) and UAS traffic management using enhanced computing capabilities will create new opportunities for improving existing traffic management systems, separation standards, and airspace planning design.

The Single European Sky ATM Research (SESAR) joint undertaking has supported several AI research projects and air traffic management (ATM), covering issues including:

• predictability of traffic at different flight phases
• passenger flow improvements at airports
• greater automation

A potential application of machine learning in the aviation industry is the translation of historical and real-time insights into customers’ behaviors into real-time strategies (e.g., adjusting the website content presented to the customer). This also includes social media sentiment analysis, which predicts customers’ needs based on their social media behavior. For example, the BigData4ATM project looks at how AI can analyze different passenger-centric geo-located data to identify patterns in passenger behavior, door-to-door travel times, and travel mode choices (Figure 5). With smart devices and related services, researchers can access large-scale, detailed longitudinal (dynamic) data to test hypotheses about passenger behavior.

Another area where AI can make a difference is ground handling. This includes high potential use cases like safety checks, aircraft movement operations (pushback and towing), aircraft turnaround operations (fuelling, catering, loading and unloading, de-icing and anti-icing), and ground transportation on the ramp (passengers, baggage, cargo, and mail). It can also facilitate airport security by digesting a large amount of historical and real-time data.

Further, aircraft makers are using AI to solve problems pilots face in the cockpit and predictive maintenance of aircraft. For example, an English start-up, Aerogility, is working with EasyJet to automate daily maintenance planning for its fleet, including forecasting heavy maintenance, engine shop visits, and landing gear overhauls. The Airbus manufacturer uses a similar tool, Skywise, to offer predictive maintenance and data analytics.

Several companies have also started using AI technologies in drones for cargo delivery of all sizes and types. For example, Nautilus, a California-based start-up, is developing a cargo drone with a 90-ton capacity.

---

Railway Transport

Railways were one of the most innovative sectors of the economy that, for the first time, allowed passengers to travel longer distances. They were also a significant factor in the industrial revolution. AI can help improve the manufacturing, operations, and maintenance of rail operators and infrastructure managers. It will improve management, lower costs, and enhance competitiveness concerning direct competitors or other transport modes.

---

Intelligent Train Automation

One of the most important contributions of AI in railway transport is the automation of train operation (ATO), which transfers the responsibility of managing train operations from the driver to a train control system with varying degrees of autonomy. This usually involves moving train drivers’ sensory and intellectual capacities to an automated driving module, ready to react to possible hazards. Further, the design module should understand and integrate passengers’ conduct on railway platforms, allowing automatic train door closure without danger.

Shift2Rail (Figure 6), an R&D joint undertaking by the EU, is developing and validating a standard ATO for all rail segments (mainline/high speed, urban/ suburban, regional, and freight lines). In addition, they are carrying out activities related to ATO to optimize resource utilization. As the container transport market is growing, many projects are being developed to synchronize container train movements better to improve the information and data exchange in real time.

---

Operational Intelligence

In railways, it is valuable to identify and be aware of potential failures before they happen to avoid service interruption. Today, AI can use the data provided by sensors (placed in critical places) to extract valuable insights and information and recommend maintenance actions. This will help train operators to reduce the fleet reserves they need to keep in case of breakdowns, as AI allows them to increase their reliability and effectiveness. Other benefits include:

• ‘Nowcasting’ and forecasting infrastructure or rolling stock conditions
• Faster and less comprehensive repair
• Maintenance costs reduction
• Better customer satisfaction

To illustrate some AI applications, the French operator Société nationale des chemins de fer français (SNCF) has started to apply predictive maintenance to pantographs, which can become fragile due to the wear effect (Figure 7). With time, the company says it will be able to forecast 80% of incidents on catenaries, which supply electrical power to the trains. According to SNCF, predictive maintenance also reduced incidents involving train switches by 30%, and this technology has been applied to many train systems and subsystems.

The SNCF rail predictive maintenance solution (Figure 7) involves the following:

• Step 1: The train communicates independently with IoT as it passes over instruments on the ground
• Step 2: Data is transmitted to data storage servers
• Step 3: Scientists and rail experts analyze the data
• Step 4: Once the data has been analyzed, it is retrieved for use by computerized maintenance management software

---

Asset Intelligence

AI in railways can also be used to assess the long-term performance of rail assets and recommend areas for improvement in their product design. AI can analyze data produced by rail infrastructure and train subsystems to help equipment manufacturers build a digital representation of physical entities, known as a digital twin. This allows the IT, operational, and engineering technologies to access the overall condition and understand asset degradation, failure, and customer behavior. These improvements represent a competitive advantage for equipment manufacturers and rail operators.

Laing O’Rourke, an international engineering company, uses AI and asset digital twin to schedule maintenance work, enabling the company to reduce scheduling activities to just 19 seconds.

---

Shipping, Navigation, and Ports

Over the past few decades, ship traffic has grown significantly, raising the stakes for maritime safety. Further, with increased container traffic at the port, adaptations to port terminals and better connections are needed. Ever-growing vessel sizes have amplified ships’ pressure on ports and their cities.

Moreover, the rising environmental issues have propelled the sector to adapt to greener rules in fierce international competition within the global maritime industry. AI can analyze the information to provide insights for better decision-making, improve safety and energy efficiency, and optimize logistics.

---

Maritime Shipping and Inland Navigation

Maritime operations require adaptation to changing conditions and decisions based on several parameters. Data from advanced navigation systems (e.g., radar, electronic navigation charts, auto-pilot systems, wave radars, oil spill detectors, and other sensors) are analyzed by AI algorithms to extract insights and carry out technical operations and maintenance. Automatic identification systems (AIS) transfer data (e.g., ship’s ID, position, course, speed, and destination).

With a combination of recorded ship movements and advanced image recognition systems, ships can be identified even if their AIS is off. This can detect anomalies in marine operations and improve sea-level safety. Further, Machine learning algorithms can predict delays due to bad weather conditions and traffic congestion and estimate future demand for oil prices.

Autonomous ships are an obvious application of AI in this area. For example, the EU-funded Maritime Unmanned Navigation through Intelligence in Networks (MUNIN) project has developed and tested the concept of an autonomous merchant ship primarily guided by automated on-board decision systems but controlled by a remote operator on shore (Figure 8). Moreover, the ship uses independent operation only during the deep-sea voyage, not in congested waters or during its port approach.

---

Ports

With growing amounts of information at large ports worldwide, a new application called port call optimization has become very popular. It uses a combination of IoT, cloud computing, and geographic information to optimize port operations, enhance its production efficiency, and improve relationships with its customers. In addition, the analyzed data can be used for forecasting and real-time planning to improve decision-making and support the port’s economic growth.

By using more advanced digital technologies, ports can become “smart” (Figure 9) to:

• provide a seamless supply chain
• optimize the allocation of relevant resources, services, and supervision
• autonomous loading and unloading (which containers to unload first and how to stack them)
• equipment scheduling (to optimize the use of cranes and vehicles)
• berth availability planning

Among the ports recognized as most advanced in becoming intelligent are Singapore, Rotterdam, Tianjin, and Dubai.

Data collected during container port operations are stored and analyzed as the basis for future AI-assisted tools, which are expected one day to manage the entire delivery cycle and further optimize terminal operations. In addition, these technological advances are understood as part of a broader supply chain transformation.

---

Challenges

AI in the transport sector has several risks and challenges associated with it. However, these risks and challenges can have significant socioeconomic effects and must be managed.

Loss of Jobs: According to a report by the Center for Global Policy Solutions, more than 4M people will lose jobs because of the quick transition to autonomous vehicles (AVs) in the United States. These jobs include drivers for deliveries, heavy trucks, buses, and taxis. In addition, AI will likely accelerate the growth toward a service-based economy which will speed up job losses for low-skilled workers.

Investment: A significant constraint on AI growth in the transport market is its high-investment nature in terms of talent, hardware, and software. Demand for AI experts is growing tremendously, and a lack of skilled AI talent will pose the most significant barrier to AI adoption in developed countries. In addition, huge investments and capital stock are required to take advantage of skilled people and business practices.

Poor and Underdeveloped Infrastructure: Low-income and developing countries like India face an enormous challenge in adopting AI because of their low-quality infrastructure. This includes roads, ports, maintenance, and repair stations. The lack of reliable power sources and weak telecommunications further make things difficult. Countries with few investments in technology research and complex infrastructure as a percentage of GDP may have a more challenging time harnessing the power of AI.

Regulatory Requirements and Privacy Concerns: Regulatory requirements for AI are always tricky to predict. Although research has shown that AVs reduce traffic deaths, it is still unclear who is ultimately liable if an accident, harm, or fatality occurs. Similarly, asking users to provide personal data to develop robust machine learning models requires privacy laws. These laws must be balanced against the benefit of having more data in a telecommunications network.

---

---

Summary

Artificial intelligence can provide a safer, cleaner, smarter, and more efficient mode of transportation for everyone in the following ways.

• Road Transport: Road vehicles can use a variety of sensors like GPS, camera, and radar in combination with actuators (devices that transform an input signal into motion), control units, and software to perform intelligent actions like self-driving.
• Aviation: Translation of historical and real-time insights into customers’ behavior into real-time strategies (e.g., adjusting the website content presented to the customer).
• Railway: AI can help improve rail operators’ and infrastructure managers’ manufacturing, operations, and maintenance. It will improve management, lower costs, and enhance competitiveness concerning direct competitors or other transport modes.
• Marine and Shipping: Data from advanced navigation systems (e.g., radar, electronic navigation charts, auto-pilot systems, wave radars, oil spill detectors, and other sensors) are analyzed by AI algorithms to extract insights and carry out technical operations and maintenance.

However, AI in transport comes with its challenges.

• Loss of Jobs: AI will likely accelerate the growth toward a service-based economy which will speed up job losses for low-skilled workers.
• Investment: A significant constraint on AI growth in the transport market is its high-investment nature in terms of talent, hardware, and software. Demand for AI experts is growing tremendously, and a lack of skilled AI talent will pose the most significant barrier.
• Poor and Underdeveloped Infrastructure: Low-income and developing countries like India face an enormous challenge in adopting AI because of their low-quality infrastructure. This includes roads, ports, maintenance, and repair stations.
• Regulatory Requirements and Privacy Concerns: Regulatory requirements for AI are always tricky to predict. Although research has shown that AVs reduce traffic deaths, it is still unclear who is ultimately liable if an accident, harm, or fatality occurs.

I hope this post helped you understand the benefits, applications, challenges, and tradeoffs of using deep learning in transportation. Stay tuned for another lesson where we will discuss deep learning and computer vision applications for logistics.

---

Consultation Services

Do you need some help with your industrial application of AI? Learn more about our consulting services.

---

Citation Information

Mangla, P. “Computer Vision and Deep Learning for Transportation,” PyImageSearch, P. Chugh, A. R. Gosthipaty, S. Huot, K. Kidriavsteva, and R. Raha, eds., 2022, https://pyimg.co/1u4c9

@incollection{Mangla_2022_CVDLT,
  author = {Puneet Mangla},
  title = {Computer Vision and Deep Learning for Transportation},
  booktitle = {PyImageSearch},
  editor = {Puneet Chugh and Aritra Roy Gosthipaty and Susan Huot and Kseniia Kidriavsteva and Ritwik Raha},
  year = {2022},
  note = {https://pyimg.co/1u4c9},
}

---