AI & Anomaly Detection in Under-Vehicle Inspection: The Future of Vehicle Security

There is a growing trend towards the use of Under-Vehicle Inspection Systems (UVISs) with the incorporation of Artificial Intelligence (AI)/Machine Learning and anomaly detection algorithms at security checkpoints, critical infrastructure security gates, Port of Entry facilities, and other High-Security Facilities. By utilizing AI and Machine Learning, under-vehicle inspectors are able to identify hidden threats (i.e., bombs, explosives, etc.), safety hazards (i.e., mechanical issues, structural deformation, etc.) as well as maintenance issues far more quickly and with considerably less false negatives than by only relying on human visual inspections.

Understanding AI-enabled UVIS

An Advanced UVIS generates one or multiple high-resolution imaging captures (including depth or multispectral data) of the undercarriage while the vehicle traverses over the scanning lane. After capturing the underside images of the vehicle, software analyzes the images based on baseline or learned model of “normal” established for that specific vehicle (or entire fleet) and determines if there are any deviations; i.e., failed attempts to hide objects, structural deformation, leakages and missing parts or components, as well as matching (threat profile) shape or structure to an explosive device. The operator is presented with identified and marked areas using annotated overlays that allow the operator to Accept, Reject or Escalate Inspection requests. 

Working with AI and Anomaly Detection UVIS

1. Pre-Processing – Correcting images (lens distortion to perspective), normalizing images, and denoising images provide consistent inputs for further processing.

2. Feature Extraction/Embedding – The Under-vehicle image is transformed into a dense Representation through the use of either Convolutional Neural Networks (CNN’s) or Transformer Vision Encoding.

3. Baseline Model – The used models may be either supervised through training on a labeled dataset or unsupervised/self-supervised anomaly detectors that learn the “normal” distribution. Unsupervised learning and deep learning models are becoming more widely utilised when there are fewer labeled threat data.

4. Anomaly Scoring & Localization – The model produces either anomaly score or heatmaps that indicate suspicious areas for the operator to validate.

5. Decisioning and Integration – Alert generations as a result of screening events will trigger warnings to an Access, VMS, and SIEM system and create an Audit Trail for reference at a later date.

Importance of Anomaly Detection

1. The true threats: Explosives, weapons and covertly concealed contraband represent a low probability, therefore the supervised classifiers must use a significant number of labeled examples in order to generalise their findings. By contrast, the unsupervised methods learn the normal and will therefore identify novel threats based upon divergence from what it has learnt as “normal”. According to recent studies, unsupervised deep models are successfully detecting anomalies in vehicle Sensor and Image Data.

2. Ability to rapidly adapt to changing items: The highly variable rate of Change with Fleet Vehicle Models, after-market parts and Environmental factors can create high variability in the learning curve. Models that learn per-site/per-fleet will dramatically decrease the number of false positives in comparison to using a one-size-fits-all rule-set.

Benefits for Security & Operations

• Increased Threat Detection Accuracy, Reduced Missed Threats: Artificial Intelligence (AI) is able to identify more subtle geometric changes as well as foreign objects compared to physical vehicle inspection done by a human via visual inspection.

• Increased Speed and Throughput for Automated Inspections: The average amount of time individual vehicles are stopped at checkpoints can be decreased from minutes to seconds due to automation. This increases throughput at checkpoints, whilst still ensuring safety during the inspection process.

• Standardization and Auditability: All inspection results are recorded and analyzed using the same consistent approach; thus, these results can be used to populate a SIEM and compliance log.

• Multifunctional Applications: This same technology is additionally capable of detecting maintenance-related defects, such as leaks, missing fasteners, and tire defects. These detections can facilitate predictive maintenance for fleets.

Examples of Real-World Applications

• Border crossings and ports – speeding up the screening of commercial and private vehicles to detect illegal cargo or modify compartments.

• High-Security End Sites – airports, energy centers, stadiums, and government compounds use UVIS as an additional line of defense against vehicle-borne threats.

• Logistics and Fleets – Automated inspection bays (some operated without an attendant) are used for automatic scanning of delivery vans, detecting safety deficiencies and damage to the undercarriage that improves road safety and minimizes downtime. Some Major transportation operators have initiated pilot programs with automated inspection bays.

• Industrial sites and Maintenance Yards – detect mechanical anomalies at an early stage for prompt intervention; also to extend the service life of industrial equipment.

Challenges: Technical and Operational

• Trade-Off Between False Positives and False Negatives: Overly aggressive or highly sensitive models will inundate operators with alerts, while overly permissive models will miss legitimate threats due to a lack of sufficient sensitivity.

• Adversarial attempts and concealment techniques: Attempts by adversaries to use camera deception combined with the exploitation of model blind spots increases the chances for successful attacks. Testing the robustness of images and using multi-modal sensors to reduce risk.

• Privacy and Legal Considerations: Image capture of private vehicles with linkages to identity systems require appropriate policy, data retention rules and secure access controls.

• Integration Complexity: In order to provide operational value, UVIS has to be integrated with physical access control systems (PAC), Vehicle Management Systems (VMS), Automatic License Plate Recognition (ALPR) and incident response work streams.

Emerging Trends That Will Shape The Next 3-5 Years

• On-Premise AI and Edge Inference: With running of anomaly detection being performed at the edge will decrease latency, reduce privacy risks and decrease bandwidth costs. Anticipate smaller, optimized models will be deployed on gateways.

• Sensor Fusion: The fusion of the 3D LiDAR or thermal and acoustic sensors will create a better detection of hidden cavities and material variations.

• Federated Learning and Continuous Learning: Operators are expected to be able to share non-identifiable model updates among fleets increasing the efficacy of anomaly detection while protecting sensitive imagery.

• Predictive Maintenance Convergence: UVIS will be an integral component of the vehicle health ecosystem. One scan can serve both security and preventive maintenance.

• Commercial Expansion and Growth: Findings from a number of market reports as well as vendor roll outs demonstrate the fast growth of under-vehicle surveillance and Vehicle Scanning technology as clients pivot to automatic, AI-based inspections.

To successfully deploy AI-powered UVIS systems, the following best practices should be followed:

• Identify potential use cases and define KPIs, including detection rates, false positive rates, and throughput.

• Pilot in controlled environments to gather representative data and refine models.

• Use a hybrid approach to detection that combines rule-based checks, supervised classification (using labeled data to find known threats), and unsupervised anomaly detection for detecting new or previously unknown threats.

• Integrate operations with alerts through a VMS, access control, and incident management systems to enable a rapid review process by a person.

• Implement appropriate governance and ensure that data is retained in accordance with a defined retention policy, that access is logged, that models are regularly audited, and that they are periodically updated using up-to-date data.

• Plan for robustness by conducting adversarial testing, by providing a multisensor fallback mechanism, and as new vehicle types enter service regularly recalibrating the systems to continue detecting them.

Conclusion 

Using AI for detecting anomalies in UVIS will lead to a significant advancement in practical inspection processes as AI will provide a faster and more efficient way to inspect vehicles and be capable of detecting many threats that human inspectors typically overlook. AI-enabled anomaly detection will also have many other benefits that are cross-functional in nature, including vehicle maintenance monitoring. As edge-device based AI continues to improve, and as sensor fusion and improvements to unsupervised anomaly detections mature, AI-enabled UVIS will become a common asset for both securing perimeters and fleet operations. The commitment of the market and the ongoing pilot testing in the enterprise marketplace indicate that this transition is taking place.