YOLO (v1→v11)

Redmon 2016→2024 · Paper

One-line summary — “You Only Look Once” reframed object detection as a single-shot regression problem solved by one CNN forward pass, establishing the real-time detection family that SLAM systems most commonly use for semantic and dynamic-object reasoning.

Problem

Before YOLO, object detection repurposed classifiers to perform detection: DPM slid a classifier over the image at all locations and scales, and the R-CNN family generated region proposals, classified each, then refined, deduplicated, and rescored boxes in post-processing. These complex pipelines were slow — far from camera frame rate — and hard to optimize because each component was trained separately. Robotics needed detection that runs in one pass, at video rate, with a single trainable objective.

Method & architecture

λcoord∑i=0S2∑j=0B1ijobj[(xi−x^i)2+(yi−y^i)2]+λcoord∑i=0S2∑j=0B1ijobj[(wi−w^i)2+(hi−h^i)2]+∑i=0S2∑j=0B1ijobj(Ci−C^i)2+λnoobj∑i=0S2∑j=0B1ijnoobj(Ci−C^i)2+∑i=0S21iobj∑c∈classes(pi(c)−p^i(c))2\begin{aligned} & \lambda_{\text{coord}} \sum_{i=0}^{S^2} \sum_{j=0}^{B} \mathbf{1}_{ij}^{\text{obj}} \left[ (x_i - \hat{x}_i)^2 + (y_i - \hat{y}_i)^2 \right] + \lambda_{\text{coord}} \sum_{i=0}^{S^2} \sum_{j=0}^{B} \mathbf{1}_{ij}^{\text{obj}} \left[ \left(\sqrt{w_i} - \sqrt{\hat{w}_i}\right)^2 + \left(\sqrt{h_i} - \sqrt{\hat{h}_i}\right)^2 \right] \\ & + \sum_{i=0}^{S^2} \sum_{j=0}^{B} \mathbf{1}_{ij}^{\text{obj}} (C_i - \hat{C}_i)^2 + \lambda_{\text{noobj}} \sum_{i=0}^{S^2} \sum_{j=0}^{B} \mathbf{1}_{ij}^{\text{noobj}} (C_i - \hat{C}_i)^2 + \sum_{i=0}^{S^2} \mathbf{1}_{i}^{\text{obj}} \sum_{c \in \text{classes}} \left(p_i(c) - \hat{p}_i(c)\right)^2 \end{aligned}

Square roots of w,hw, h make small-box errors count more; classification loss applies only in cells containing objects. Trained ~135 epochs on VOC 2007+2012 (batch 64, momentum 0.9, decay 0.0005, dropout 0.5, scale/translation/HSV augmentation).

Results

On Pascal VOC 2007, YOLO reaches 63.4% mAP at 45 FPS and Fast YOLO 52.7% mAP at 155 FPS — versus 30Hz DPM at 26.1% mAP and, among the accurate-but-slow, Fast R-CNN at 70.0% mAP / 0.5 FPS and Faster R-CNN (VGG-16) at 73.2% / 7 FPS; a VGG-16 YOLO scores 66.4% at 21 FPS. Error analysis shows Fast R-CNN is almost 3× more likely to predict background false positives (13.6% of its top detections); rescoring Fast R-CNN with YOLO lifts VOC 2007 mAP from 71.8% to 75.0%. On VOC 2012 test YOLO scores 57.9% mAP, trailing state of the art mainly on small objects (bottle, sheep, tv/monitor 8–10% below R-CNN). Generalizing to artwork, YOLO’s person AP holds up (VOC2007 59.2 → Picasso 53.3, People-Art 45) while R-CNN collapses (54.2 → 10.4, 26).

That 45-FPS operating point made per-frame detection viable inside live perception stacks for the first time, and the family’s evolution kept it there — setting the benchmark that Transformer detectors (DETR → RT-DETR) later had to beat on both axes.

Why it matters for SLAM

Object detection is the cheapest way to inject semantics into a SLAM pipeline. Real-time detectors in the YOLO family are used to mask out dynamic objects (people, vehicles) so they do not corrupt feature tracking, to provide object-level landmarks for object SLAM (e.g., CubeSLAM builds 3D cuboid landmarks from 2D detections), and to label maps for downstream tasks. When a SLAM system needs “what is in the image” at frame rate on a robot’s onboard computer, YOLO is usually the first tool reached for — with the caveat that it is closed-set by design, the limitation that motivated open-vocabulary detectors like Grounding DINO.