Humans need eyes to drive, and so do self-driving cars. The perception component of self-driving vehicles makes them possible to understand the world. It can be divided into two parts. First, different sensors installed in the car would capture a variety of information and then later be sent to some computers, and second, a number of well-trained neural networks would take in all those data as input and predict landmark features as needed. The landmark features can then be used in localization, mapping, and even motion planning. One could say perception is the very first component in the self-driving software stack pipeline.
Let’s break down into two parts: sensors and perception neural networks, and briefly explain details.
Sensors
Camera
The eyes of the car.
LiDar ((Light Detection and Ranging Sensors)
3D ruler to measure distance among surroundings.
Radar
2D ruler to measure distance among surroundings.
Internal Compass
- GPS (Global Positioning System) – can be off by a meter
- GNSS (Global Navigation Satellite System) – precise on the order of centimeters
- IMU – estimate accelerations
Perception Neural Networks
Software that helps the car see the world around itself as well as recognize and classify the things that it sees.