Four Reasons Thermal Cameras Can Improve Automatic Emergency Braking and Save Lives

This video is unavailable due to your cookie settings.

---

Despite the increased prevalence and sophistication of automatic emergency braking (AEB) in vehicles today, pedestrian fatalities are climbing. The U.S.-based Governors Highway Safety Association (GHSA) estimates there were 6,200 pedestrians killed in traffic accidents in the U.S. in 2018, the highest total since 1990. The GHSA also found that during the past 10 years, nighttime crashes accounted for more than 90 percent of the total increase in pedestrian deaths.

Car manufacturers currently include sections in their U.S. owner manuals describing potential situations where AEB or pedestrian collision avoidance systems are challenged.  They include pedestrians in darkness, wearing white, being near walls, pushing a stroller and bending forward to name a few common scenarios. How can AEB systems be improved, especially in cases where visible cameras and radar are challenged?

To solve this problem, automotive researchers and developers must consider multi-sensing solutions, including thermal cameras that “see” heat energy, with layered levels of protection to ensure safety in all common driving situations.

Here are four reasons why thermal imaging cameras can improve existing AEB systems to save lives:

1: Thermal Cameras Add a Layer of Protection Where Existing Technologies Fail

Thermal cameras detect and measure an entirely different wavelength of energy compared to other sensors. Long-wave infrared (LWIR) radiation, or thermal radiation, is part of the electromagnetic spectrum that is given off, absorbed or is reflected by everything warmer than 0 Kelvin (-273°C), which constitutes basically everything on earth.  Thermal cameras “see” equally well in daylight, in total darkness, or in blinding sun glare.  They work significantly better than visible cameras in smoke or in inclement weather such as fog. They are particularly adept at detecting body heat, even if a pedestrian is in a cluttered roadside background.

Thermal cameras see pedestrians in cluttered backgrounds and through darkness, fog, and sun glare.  

2: Thermal Cameras Provide More Time for AEB Systems to Take Appropriate Action

Machine learning datasets can be used to train algorithms to not only detect an object but also classify the shapes, letting the system know that the object is potentially a pedestrian along with an associated algorithm confidence level.  Thermal cameras can detect and classify a pedestrian on the roadway up to four times farther than the illumination distance of a typical car’s headlights, providing more time for an AEB system to identify and react to a potentially dangerous roadway situation. They provide additional assurance that an object is what the system thinks it is – day or night.

Annotated thermal datasets are available that include people, dogs, cars, bicycles and other vehicles.

3: Thermal Cameras Improve a Vehicle’s Overall Automated Decision-Making Process

Existing automotive AI programs paired with visible cameras struggle with equitable predictive performance of pedestrians with different skin tones.  Researchers at Georgia Tech University found that pedestrians with darker skin are at greater risk of being missed using the AI systems they evaluated versus those with lighter skin. Thermal cameras are unaffected by this recognition bias because they instead detect body heat. Skin tone is inconsequential for thermal.

4: Thermal Imaging is a Proven Automotive Technology

A common objection to greater deployment of thermal cameras is technology maturity and hardware cost. FLIR thermal sensors exist within automotive-qualified systems though tier-one automotive supplier Veoneer, on more than 600,000 primarily luxury vehicles. These thermal cameras are used for a passive early-warning system where the driver is alerted visually or audibly of potential hazards on the roadway ahead. The next step in this evolution is to connect these cameras to existing AEB systems to make them even more effective and useful for drivers.

FLIR thermal cameras, like the one installed in this upcoming Audi vehicle grill, are on the road today in driver-warning systems through Veoneer.

The adoption of thermal cameras for collision-avoidance systems remains in its infancy. However, where split-second action can be the difference between an accident and near miss, adding thermal cameras to AEB systems can’t wait. Furthermore, as adoption and manufacturing scale grows, costs will decrease, making thermal sensors increasingly more affordable for all AEB systems.