DS-SLAM

Yu 2018 · Paper

One-line summary — A semantic visual SLAM for dynamic environments that combines SegNet semantic segmentation with an optical-flow/epipolar moving-consistency check to reject moving objects, built on ORB-SLAM2 and producing a dense semantic octo-tree map.

Problem

Despite decades of progress, two problems remained poorly solved for mobile-robot SLAM: “how to tackle the moving objects in the dynamic environments” and “how to make the robots truly understand the surroundings and accomplish advanced tasks.” A person walking through the scene corrupts pose estimation (their features violate the static-world assumption), and purely geometric maps offer no semantics for higher-level tasks. DS-SLAM addresses both: robust localization in dynamic scenes, plus a semantic map usable for navigation and beyond.

Method & architecture

• Five parallel threads: tracking, semantic segmentation, local mapping, loop closing, and dense semantic map creation; local mapping and loop closing are unchanged ORB-SLAM2. Each RGB frame goes to tracking and segmentation simultaneously — while the tracking thread waits for the SegNet result it runs the moving-consistency check, so the two threads finish at roughly the same time.
• Semantic segmentation: real-time SegNet (Caffe) trained on PASCAL VOC (20 classes); people are treated as the archetypal dynamic class, though the scheme applies to any segmented movable category.
• Moving-consistency check: optical-flow pyramid matches feature points into the current frame (matches near image edges or with large 3×3-block appearance difference are discarded); RANSAC estimates the fundamental matrix $F$; for matched homogeneous points $P_1 = [u_1, v_1, 1]$, $P_2 = [u_2, v_2, 1]$, the epipolar line in the current frame is $\mathbf{I}_1 = [X, Y, Z]^{T} = F P_1$, and a point is declared moving if its distance to that line

$$D = \frac{\lvert P_2^{T} F P_1 \rvert}{\sqrt{\lVert X \rVert^{2} + \lVert Y \rVert^{2}}}$$

exceeds a preset threshold $\varepsilon$.

• Outlier rejection — semantics + geometry combined: contours of whole dynamic regions are too expensive and fragile to extract geometrically, so the two signals form a two-level decision: if enough moving-consistency points fall inside a segmented object’s contour, the object is deemed moving, and all ORB features within its outline are removed before pose estimation. If no person is detected, or detected people are static, all features are used — parked “dynamic” classes are not wastefully discarded.
• Dense semantic 3D octo-tree map: keyframe poses + depth images build local point clouds fused into a global octo-tree, each voxel colored by semantic label. Occupancy is fused probabilistically with the log-odds score $l = \operatorname{logit}(p) = \log\frac{p}{1-p}$, updated per observation as

$$L(n \mid z_{1:t}) = L(n \mid z_{1:t-1}) + L(n \mid z_t),$$

where $L(n \mid z_t) = \tau$ if voxel $n$ is observed occupied at time $t$ and 0 otherwise; only voxels whose probability exceeds a threshold are kept, so transient (dynamic) voxels never stabilize into the map.

Results

• TUM RGB-D (vs RGB-D ORB-SLAM2; Intel i7, P4000 GPU): ATE RMSE on high-dynamic sequences drops by an order of magnitude — fr3_walking_xyz 0.7521 → 0.0247 m (96.71%), fr3_walking_static 0.3900 → 0.0081 m (97.91%), fr3_walking_half 0.4863 → 0.0303 m (93.76%), fr3_walking_rpy 0.8705 → 0.4442 m (48.97%). Improvements reach up to 97.91% (RMSE) and 97.94% (S.D.).
• Drift (RPE): translational drift RMSE on walking_xyz falls 0.4124 → 0.0333, rotational 7.7432 → 0.8266; on low-dynamic fr3_sitting_static gains are small (ATE 25.94%) since ORB-SLAM2 already copes.
• Timing: ORB extraction 9.4 ms + moving-consistency check 29.5 ms in the tracking thread, overlapped with 37.6 ms SegNet; ~59.4 ms per frame average for the full pipeline including octo-tree mapping — near real time, unlike prior dynamic-filtering methods.
• Real robot: integrated with ROS on a TurtleBot2 with Kinect V2 (960×540); the log-odds filtering yields an octo-tree reconstruction unaffected by walking people, plus a 2D cost map for navigation.

Why it matters for SLAM

Classical SLAM assumes a static world, and people walking through a scene can corrupt tracking badly — the TUM RGB-D dynamic sequences exist precisely to expose this. DS-SLAM, contemporaneous with DynaSLAM, is one of the canonical answers: fuse a segmentation network with geometric consistency checks inside an ORB-SLAM2 backbone. The “semantic prior + motion check” recipe it helped establish remains the standard baseline design for dynamic-environment SLAM, and its semantic octo-tree output foreshadowed maps built for high-level tasks rather than localization alone.

Related

• DynaSLAM
• ORB-SLAM2
• DynaSLAM II
• VDO-SLAM
• Epipolar geometry