Sensor fusion: why devices trust several sensors at once

Sensor fusion means processing data from several different sensors at once to get a more accurate and reliable result than any one sensor gives on its own.

Sensor fusion: several sensors give one accurate picture

In plain words

When you cross the road, you look with your eyes, listen with your ears and feel the ground shake from a truck. Each sense alone can be wrong, but together they give a reliable picture. Sensor fusion does the same for devices: the camera, lidar, GPS and gyroscope "consult" each other.

How it works

Every sensor has its own strengths and errors.
GPS gives coordinates but is imprecise and slow; a gyroscope is fast but gradually "drifts".
An algorithm (for example, a Kalman filter) weighs the data according to which sensor deserves more trust at that moment.
One sensor's errors are compensated by the others.
The output is a smooth, accurate position, speed or orientation.

Just as a person combines senses, a device combines sensors

Where you'll see it

In smartphones (screen rotation, compass, navigation in tunnels), fitness watches (steps, heart rate while moving), drones (stabilisation), robots and cars (cameras + radars + lidars) and VR headsets (head tracking).

Does it matter when buying

You see the result, not the term: a GPS track on your watch without "jumps", steady navigation in the city, smooth VR. That is why good watches and phones with quality sensors give more accurate data even if their GPS is "the same".

Myths

Myth: One good sensor beats several average ones. Reality: Often the opposite: several different sensors together are more reliable.
Myth: Sensor fusion is just averaging. Reality: The algorithms are smarter: they account for when to trust which sensor.

FAQ

What is a Kalman filter?

A popular algorithm that combines measurements with a prediction to get the most accurate estimate.

Why does navigation work in a tunnel?

The phone keeps calculating movement from the accelerometer, gyroscope and speed while there is no GPS.

Does a watch use sensor fusion?

Yes, for example to measure distance and heart rate more accurately while you move.