How Does RetinaFace Work?
RetinaFace works by passing an input image through a deep convolutional neural network backbone to extract hierarchical feature maps, then attaching multiple prediction heads at different scales to simultaneously predict face bounding boxes, confidence scores, and five-point facial landmarks. The entire process happens in a single forward pass, which is what makes it fast and efficient compared to older two-stage detectors.
Step 1: Feature Extraction with a Backbone Network
The input image is fed into a backbone CNN, typically MobileNet for lightweight applications or ResNet-50 for higher accuracy. As the image passes through successive convolutional layers, the network produces feature maps at progressively lower spatial resolutions but higher semantic richness.
Step 2: Feature Pyramid Network for Multi-Scale Detection
A Feature Pyramid Network (FPN) is layered on top of the backbone. The FPN takes feature maps from multiple depths of the backbone and combines them through a top-down pathway with lateral connections. This creates a set of feature maps at different spatial resolutions, each enriched with both local detail and global semantic context.
FPN Pyramid Levels
- P2 (largest spatial): Detects very small faces (16x16 pixels and below)
- P3: Detects small to medium faces
- P4: Detects medium faces
- P5 (smallest spatial): Detects large faces
Step 3: Context Modules
At each pyramid level, RetinaFace applies a context module before the detection head. The context module uses deformable convolutional layers to expand the effective receptive field, which is particularly helpful for detecting partially occluded faces.
Step 4: Multi-Task Detection Heads
Each pyramid level has a detection head with three branches running in parallel. The classification branch predicts whether each anchor contains a face. The bounding box regression branch predicts the offset to the face box. The landmark regression branch predicts the pixel coordinates of five facial keypoints.
Step 5: Non-Maximum Suppression
After the forward pass, hundreds of candidate detections are produced. Non-Maximum Suppression (NMS) eliminates redundant overlapping detections, keeping the highest confidence detection and suppressing others that overlap above an IoU threshold.
Conclusion
RetinaFace works through a carefully engineered combination of multi-scale feature extraction, context-aware processing, and multi-task learning. Each component addresses a specific challenge in face detection, resulting in a detector that is both accurate and efficient across a wide range of real-world conditions.
