Tightly-coupled vs Loosely-coupled

When fusing a camera and an IMU, the first architectural decision is where the fusion happens.

Loosely-coupled systems run separate estimators — a visual odometry pipeline producing camera poses and an inertial navigation pipeline integrating IMU measurements — and then fuse their outputs (poses, velocities) in a second stage, typically with a Kalman filter. Each subsystem treats the other as a black box:

camera → [ VO pipeline ]  → pose estimate ─┐
                                           ├→ [ fusion filter ] → fused state
IMU    → [ INS integration ] → pose/vel  ──┘

Tightly-coupled systems put the raw measurements of both sensors into a single estimator: image feature observations (reprojection residuals) and IMU readings (preintegrated inertial residuals) are jointly optimized over one shared state containing poses, velocities, and IMU biases. Virtually all modern VIO systems — MSCKF, OKVIS, VINS-Mono, Kimera-VIO, Basalt — are tightly coupled.

What the joint state looks like

A tightly-coupled sliding-window estimator carries, for each keyframe $i$ ,

$\mathbf{x}_i = \left(\mathbf{R}_i,\; \mathbf{p}_i,\; \mathbf{v}_i,\; \mathbf{b}^g_i,\; \mathbf{b}^a_i\right)$

plus landmark parameters (or structureless equivalents), and minimizes one cost of the form

$\min_{\mathcal{X}} \; \|\mathbf{r}_p\|^2 \;+\; \sum_{(i,j)} \|\mathbf{r}_{\text{IMU}}(\mathbf{x}_i, \mathbf{x}_j)\|^2_{\Sigma^{-1}} \;+\; \sum_{k,i} \rho\!\left(\|\mathbf{r}_{\text{reproj}}(\mathbf{l}_k, \mathbf{x}_i)\|^2\right).$

Both sensor modalities constrain the same variables in the same solve — that is the definition of tight coupling. A loosely-coupled design instead feeds the fusion filter a pose measurement $\hat{\mathbf{T}}_i$ whose internal structure (which directions were well-constrained, which features produced it) has already been compressed away.

Comparison

	Loosely-coupled	Tightly-coupled
Fusion level	Pose/velocity estimates	Raw measurements
Accuracy	Lower (information lost at the interface)	Higher (cross-correlations exploited)
Complexity	Low; subsystems reusable	High; joint state and Jacobians
Failure behavior	One subsystem can fail independently	Visual outliers can corrupt the joint state, but IMU also aids vision (e.g., feature prediction)
Bias estimation	IMU biases not observable from fused poses alone	Biases estimated jointly, constrained by vision
Examples	Pose-fusion EKFs (e.g., ethzasl MSF-style frameworks), classical GNSS/INS integration	MSCKF, OKVIS, VINS-Mono, Kimera-VIO, Basalt

Why tight coupling wins on accuracy

The cross-correlations between visual and inertial information are what make the combined system strong:

Vision constrains bias drift. Gyro and accelerometer biases are only weakly observable from IMU data alone; visual constraints on the trajectory pin them down continuously.
The IMU constrains scale, roll, and pitch. The accelerometer senses gravity and true acceleration, making monocular metric scale observable and giving an absolute vertical reference.
The IMU aids the front-end. Integrated rotation between frames predicts where features will reappear, keeping KLT/descriptor tracking alive through fast motion and blur.
Uncertainty stays honest. A loosely-coupled interface must assign the VO pose a covariance; in degenerate geometry (low parallax, pure rotation) the true error distribution is strongly anisotropic and correlated with past states — structure that a compressed pose estimate cannot carry.

The price, and when loose coupling is right

Tight coupling demands consistent time synchronization (camera-IMU offset of even a few ms degrades accuracy), accurate camera-IMU extrinsic calibration, a careful initialization procedure (gravity, velocity, bias, scale), and marginalization machinery to bound the state. Loosely-coupled designs survive where modularity matters more than peak accuracy: quickly integrating an existing odometry source (wheel odometry, a proprietary VO black box, GNSS/INS), building redundant/failover architectures, or prototyping. If one subsystem’s output is already near-optimal and its failure modes must stay contained, loose coupling is a legitimate engineering choice — not just a lesser one.

Common pitfalls

Calling a system “tightly coupled” because it uses both sensors — the test is whether raw measurements share one estimator, not whether both sensors are present.
Fusing a VO pose stream with an aggressive fusion filter while ignoring the strong temporal correlation of VO errors (drift is not white noise); this makes the fused covariance wildly optimistic.
Assuming tight coupling removes the need for good calibration — it increases sensitivity to extrinsic and time-offset errors, which is why modern systems estimate both online.

Why it matters for SLAM

This distinction is the first question to ask about any VIO paper, and the same vocabulary reappears in every multi-sensor fusion context (LiDAR-visual-inertial, GNSS fusion). Understanding why tight coupling wins on accuracy — retained cross-sensor correlations — also explains why the field consistently moves toward joint estimation whenever compute allows.

Filter-based vs Optimization-based — the second key axis for classifying VIO systems.
MSCKF — the classic tightly-coupled filter.
VINS-Mono — the classic tightly-coupled optimizer.
IMU preintegration — the tool that makes tightly-coupled optimization tractable.
Tightly-coupled LiDAR-camera — the same concept applied to LiDAR fusion.
Multi-sensor calibration — the prerequisite tight coupling depends on.