Mobile
Phones are the most widely deployed SLAM platforms in existence: every modern iPhone and most Android devices run visual-inertial tracking as a system service (ARKit and ARCore). Working on “mobile SLAM” therefore means two distinct skills: using the platform tracking stacks from the native app languages, and — for SLAM engineers proper — getting custom C++ perception code to run well on phone hardware.
Android (Java/Kotlin). Applications are written in Kotlin (or Java), but SLAM-grade code runs in C++ via the NDK and is bridged with JNI. The platform pieces that matter: Camera2/CameraX for frame access (with timestamps and exposure metadata), the sensor APIs for IMU data, and ARCore if you prefer to consume Google’s built-in motion tracking instead of running your own. Practical realities of Android SLAM work include camera-IMU time synchronization quality varying wildly across devices, and thermal throttling shaping what “real-time” means.
iOS (Objective-C/Swift). Applications are written in Swift (legacy code in Objective-C), with C++ integrated directly (Objective-C++ or Swift’s C++ interop). AVFoundation provides camera frames, CoreMotion the IMU, and ARKit exposes Apple’s tightly integrated VIO — widely regarded as a benchmark for consumer tracking quality thanks to factory-calibrated, well-synchronized camera-IMU hardware. Metal and CoreML/ANE handle GPU compute and neural inference for learned components.
Sensor plumbing: where mobile VIO lives or dies
- Clock domains. Camera and IMU timestamps must be on the same clock. On Android,
SENSOR_INFO_TIMESTAMP_SOURCEtells you whether camera timestamps share the IMU’s clock base (REALTIME) or live in their own (UNKNOWN) — in the latter case you must estimate the offset yourself, and per device. - Rolling shutter. Nearly all phone cameras read out row by row, so each image row is captured at a slightly different time; at fast motion this warps geometry and must be modeled (or accepted as an error source). Platform trackers handle it internally — a custom pipeline has to as well.
- OIS/EIS. Optical image stabilization physically moves the lens, silently changing the effective intrinsics per frame; electronic stabilization warps the image. Both are great for video and hostile to geometry — disable them for SLAM or model their effect.
- Exposure metadata. Auto-exposure changes image brightness frame to frame; direct/photometric methods need the exposure time and ISO values that Camera2/AVFoundation report.
The engineering checklist
For a SLAM engineer, the mobile-specific checklist looks the same on both platforms:
- Sensor access and synchronization — hardware timestamps for camera and IMU, rolling-shutter awareness, exposure/ISO metadata (see above).
- Compute budget — a few CPU cores (shared with the app), NEON SIMD, mobile GPU, and an NPU; power and thermals constrain sustained load far below peak, so benchmark hot, not cold.
- Language bridging — Kotlin/Swift UI over a shared C++ core, the standard architecture for cross-platform SLAM products.
- Use vs. build — ARKit/ARCore are excellent and free; custom SLAM on phones is justified when you need capabilities they don’t expose (custom sensors, maps you own, cross-device relocalization).
Persistence and shared experiences
The platform stacks also expose map-level features that are pure SLAM under the hood: ARKit can serialize its map (ARWorldMap) so an app can save a session and relocalize into it later, and ARCore’s Cloud Anchors let multiple devices resolve the same anchor — cross-device relocalization as a service. Knowing what these do internally (place recognition, map merging, relocalization) tells you exactly when you can rely on them and when you need your own mapping layer.
Why it matters for SLAM
Mobile AR is where visual-inertial SLAM met mass deployment, and phone constraints (tight power budgets, rolling-shutter cameras, consumer-grade IMUs) have shaped the field’s engineering priorities. Whether you build apps on ARKit/ARCore or compete with them, understanding the mobile stack — sensor plumbing, native/managed language bridging, and thermal-bounded real-time — is essential for shipping SLAM to the largest platform there is.