Cabin sensing

Cabin sensing

Driver and cabin monitoring is becoming a key safety function for almost every car and is supporting more and more features for the life-on-board.

We have developed software solutions for interior sensing which already meet future safety standards and further increase vehicle comfort.

In the cabin sensing solution, we combine many years of expertise in all aspects of interior sensing for human-machine interaction with extensive know-how in interior cameras, radar sensor technology, sensing algorithms, and validation tools. With real-time object monitoring of the entire vehicle interior, our solution can reliably detect objects in the vehicle, whether they are adults, children, animals, or physical items.

The solution is compatible with different hardware components and platforms (cameras, high-performance computers) and meets future safety regulations of the European Commission and the consumer protection organization European New Car Assessment Program (Euro NCAP).

Our cabin sensing software portfolio comprises the following functions:

Driver monitoring
Cabin monitoring
Sensor fusion
Cabin sensing validation

Driver monitoring

Features based on EU-NCAP¹ and EU-GSR²

Blink detection
Camera/(infrared) blockage
Eyelid state
Eye/pupil tracking
Driver attention/distraction
Driver engagement level
Driver identification
Driver impairment level
Driver presence detection
Drowsiness detection
Gaze tracking
Glasses and mask detection
Head pose tracking
Sleep/microsleep detection
Spoofing detection

Optional features

Activity detection (e.g., eating, speaking)
Age estimation (range)
Area-of-interest detection
Facial expression
Multiple drivers detection
Online calibration

Optional features (GB/T³)

Abnormal head pose detection
Phone usage
Smoking detection
Yawning detection
Long eye-closure detection

Well-being

In addition, our driver monitoring solution can detect driver vital signs measuring the health and well-being of the driver such as

Breathing rate detection
Health anomaly detection
Heart rate detection
Intoxication detection
Stress detection

¹ European New Car Assessment Program

² European General Safety Regulation

³ Chinese optional national standard, also known as Guobiao standard

Cabin monitoring

The cabin monitoring function extends the driver monitoring functionality by covering not only the driver but also enabling the detection of passengers and physical items throughout the complete cabin interior.

Features (regulation-driven)

Body pose (used for airbag ignition)
Child presence detection
Classification
Height estimation
Weight estimation
Object detection
Occupant presence detection
Seat belt detection

Optional features

Gesture detection
Hand tracking
Online calibration
Pet detection

Sensor fusion

Our solution can merge the data from a range of in-vehicle sensors, including various camera types and radar technologies. By cross-validating the information from the various sensors, detection results will have a higher degree of accuracy and reliability, avoiding false alerts.

For example, sensor fusion such as near-infrared (NIR) image plus time of flight (ToF) camera can be used to enable 3D body-pose detection.

Car interior with two front passengers and a dog in the back seat. Overlaid icons illustrate multiple sensing technologies: RGB camera, NIR (Near Infrared), ToF (Time of Flight), Radar, UWB (Ultra-Wideband), and Voice recognition. The graphic emphasizes sensor fusion for comprehensive occupant monitoring and interaction.

Cabin sensing validation

In the cabin sensing solution, we combine many years of expertise in all aspects of interior sensing for human-machine interaction with extensive know-how in interior cameras, radar sensor technology, sensing algorithms, and validation tools.

Building upon this extensive experience, we are able to offer a extensive range of tools and services when it comes to validation and verification of the software and the overall solution. Only by ensuring high recognition rates and a high overall solution reliability level can the safety of the driver and passengers be assured.

Our cabin sensing validation tools and services portfolio comprises the following tools and services:

Cloud-based algorithm performance validation
MetaSepia – web service for synthetic-data generation

Cloud-based algorithm performance validation

In the area of cloud-based algorithm performance validation, we can provide an extensive suite of solutions and services, such as

A scalable cloud-based infrastructure, accessible from anywhere in the world, to process and handle big data, providing fast availability of test results,
several thousand hours of existing recordings, the necessary infrastructure to create new recordings, and the possibility to extend with synthetic data where applicable, all to effectively validate and train the driver- and cabin-monitoring algorithms,
a number of cloud-based debug and investigation tools, used to look for any anomalies in the performance of the algorithms, with highly detailed KPI reports clearly depicting the performance of the algorithms,
high-accuracy ground truth systems,
hardware-in-the-loop (HIL) and software-in-the-loop (SIL) test systems,

all of which together with test services can be bundled into an end-to-end validation-as-a-service (VaaS) package if required.

Laboratory environment featuring a seating buck with a person positioned in a simulated driver seat surrounded by an array of sensors and metal frames. A monitor in the foreground displays real-time imaging data, illustrating the testing and calibration process for advanced in-cabin sensing systems.

In the lab in a fully equipped stationary seating buck.