SVO2

Forster 2017 · Paper

One-line summary — Extends semi-direct visual odometry to multi-camera rigs, fisheye/catadioptric lenses, edgelet features, and motion priors, with an empirical convergence analysis of direct alignment and an optional iSAM2 bundle-adjustment backend.

Problem

The original SVO operated with a single perspective camera, but real robotic platforms — especially micro aerial vehicles — carry multiple cameras with wide-angle or fisheye lenses to maximise field of view. Direct methods were also used largely on faith: there was little analysis of when and why photometric alignment converges, and they lacked the optimality of feature-based joint optimisation. SVO2 (“SVO: Semi-Direct Visual Odometry for Monocular and Multi-Camera Systems”, TRO 2017) generalises the semi-direct approach — direct methods “to track and triangulate pixels that are characterized by high image gradients,” feature-based methods “for joint optimization of structure and motion” — to arbitrary camera systems.

Method & architecture

Sparse image alignment, generalised — incremental motion of a body frame BB (extrinsics TCB\mathbf{T}_{CB} per camera) is estimated by minimising the photometric error of patches around corner and edgelet features with known depth, summed over all cameras C\mathcal{C} of the rig:

Tkk−1⋆=arg⁡min⁡Tkk−1∑C∈C∑u∈Rˉk−1C12∥IkC(π(TCBTkk−1 ρu))−Ik−1C(π(TCB ρu))∥ΣI2\mathbf{T}^{\star}_{kk-1}=\arg\min_{\mathbf{T}_{kk-1}}\sum_{C\in\mathcal{C}}\sum_{\mathbf{u}\in\bar{\mathcal{R}}^{C}_{k-1}}\frac{1}{2}\big\lVert I^{C}_{k}\big(\pi(\mathbf{T}_{CB}\mathbf{T}_{kk-1}\,\rho_{\mathbf{u}})\big)-I^{C}_{k-1}\big(\pi(\mathbf{T}_{CB}\,\rho_{\mathbf{u}})\big)\big\rVert^{2}_{\Sigma_I}

so every camera that sees structure contributes to one body-pose estimate; multi-camera support only changes the Jacobians. Optional motion priors (constant velocity, or gyroscope rotation R~kk−1\tilde{\mathbf{R}}_{kk-1}) add penalty terms 12∥pkk−1−p~kk−1∥Σp2+12∥log⁡(R~kk−1⊤Rkk−1)∨∥ΣR2\frac{1}{2}\lVert\mathbf{p}_{kk-1}-\tilde{\mathbf{p}}_{kk-1}\rVert^{2}_{\Sigma_p}+\frac{1}{2}\lVert\log(\tilde{\mathbf{R}}^{\top}_{kk-1}\mathbf{R}_{kk-1})^{\vee}\rVert^{2}_{\Sigma_R} to the same cost.

Relaxation and refinement — as in SVO, each feature is then re-aligned in 2D against the keyframe where it was first observed (affine-warped patch, inverse compositional Lucas-Kanade). Edgelets suffer the aperture problem, so their correction is restricted to the edge normal n\mathbf{n}: u′⋆=u′+δu⋆ n{\mathbf{u}'}^{\star}=\mathbf{u}'+\delta u^{\star}\,\mathbf{n}. Finally, bundle adjustment minimises reprojection errors over keyframes and landmarks — edge-feature residuals projected along the normal, since the along-edge component is unobservable — solved in real time with the incremental smoother iSAM2 (or, cheaper, pose-and-points-only optimisation for embedded use).

Mapping — a recursive Bayesian filter per feature over inverse depth ρ\rho with inlier probability γ\gamma, measurements modelled as Gaussian + Uniform mixture p(ρ~ik∣ρi,γi)=γi N(ρ~ik∣ρi,τi2)+(1−γi) U(ρ~ik∣ρimin,ρimax)p(\tilde{\rho}_i^k\mid\rho_i,\gamma_i)=\gamma_i\,\mathcal{N}(\tilde{\rho}_i^k\mid\rho_i,\tau_i^2)+(1-\gamma_i)\,U(\tilde{\rho}_i^k\mid\rho_i^{min},\rho_i^{max}) and the posterior approximated as Beta(γ∣ak,bk) N(ρ∣μk,σk2)\mathrm{Beta}(\gamma\mid a_k,b_k)\,\mathcal{N}(\rho\mid\mu_k,\sigma_k^2) — outliers barely shift the mean while γ\gamma reports confidence. Filters are updated from previous and subsequent frames (better for forward motion) and skip measurements where the edge is parallel to the epipolar line.

Wide field of view — projection/unprojection use the polynomial omnidirectional camera model, so fisheye and catadioptric images are used undistorted; epipolar search samples the great circle (epipolar plane ∩ unit sphere) at roughly one-pixel angular resolution to follow the curved epipolar line.

Convergence study — a controlled experiment measures the probability that image-to-model alignment converges as a function of distance to the reference frame: convergence improves with patch size up to 4×4 pixels, dense alignment adds almost nothing over semi-dense (zero-gradient pixels have zero Jacobians), and sparse alignment has a smaller convergence radius than semi-dense — which is why SVO aligns only to the previous frame and relies on feature alignment to the map to kill drift.

Results

On the EuRoC MAV benchmark (11 sequences, averaged over 5 runs, loop closures disabled for competitors), SVO achieves higher accuracy than LSD-SLAM and than real-time ORB-SLAM in most runs, while DSO is consistently very accurate and mostly outperforms SVO in the monocular setting; stereo SVO is generally more accurate than monocular and immune to scale drift, though SVO fails on the Vicon Room sequences with abrupt illumination changes and on-spot rotations. Efficiency is the headline: mean 2.53 ms per frame (SVO Mono) on a laptop i7 with 55% CPU at 20 Hz, versus 29.81 ms/187% for ORB-SLAM and 23.23 ms/236% for LSD-SLAM — “up to ten times faster… and requires only a fourth of the CPU usage.” On an NVIDIA Jetson TX1, sparse image alignment takes 2.54 ms and feature alignment 1.40 ms. Edgelets add little accuracy in corner-rich EuRoC scenes but improve robustness where corners are absent; motion priors cut alignment iterations.

Why it matters for SLAM

SVO2 turned the semi-direct paradigm into a system deployable on real robotic platforms — especially MAVs carrying several wide-angle cameras — and was widely used on commercial and research drones. Its convergence analysis gave direct methods an empirical grounding (and justified coarse-to-fine, patch-size, and frame-rate design choices), and its Beta×Gaussian inverse-depth filter became a common reference for probabilistic per-pixel depth estimation. It remains a go-to design when very fast, lightweight odometry is needed on embedded hardware.