Event representations

Raw event streams are sparse, asynchronous lists of tuples $e_k = (\mathbf{x}_k, t_k, p_k)$ — a format almost no downstream algorithm consumes directly. Before tracking, mapping, or feeding a neural network, most systems convert a batch of events into an intermediate representation. The choice of representation is one of the central design decisions in event-based vision, trading temporal fidelity against compatibility with existing algorithms.

Event frames / histograms

Accumulate events over a time window (or a fixed event count) into a 2D image:

$H(x, y) = \sum_k p_k \cdot \delta(\mathbf{x}_k - (x, y))$

Simple, compatible with any image-based algorithm (feature detection, CNNs, direct alignment), and the basis of many tracking front-ends. The cost: temporal structure inside the window is discarded, and the “image” appearance depends on motion speed and window length.

Time surfaces

Store at each pixel the time of the most recent event, usually passed through an exponential decay:

$\mathcal{T}(\mathbf{x}, t) = \exp\!\left(-\frac{t - t_{\text{last}}(\mathbf{x})}{\tau}\right)$

This encodes recency of activity — recently active pixels are bright, stale ones fade — producing smooth, gradient-rich maps that are well suited to registration and data association. ESVO’s tracking, for example, aligns against time surfaces. The decay constant $\tau$ is a real tuning knob: too short and the surface is sparse, too long and it smears history.

Voxel grids

Bin events into a 3D tensor over space and discretized time:

$V(x, y, b) = \sum_{k:\, t_k \in \text{bin}_b} p_k \cdot \delta(\mathbf{x}_k - (x, y))$

Voxel grids preserve coarse temporal structure while remaining a dense tensor that standard 2D/3D CNNs can process — the default input for learned event pipelines such as DEVO — at the cost of memory and a fixed binning choice.

Spike tensors / event point clouds

Treat the events themselves as points in $(x, y, t)$ space, processed by PointNet-style networks or spiking neural networks (SNNs) that operate natively on asynchronous spikes. This is the most faithful representation (no aggregation at all) but requires specialized architectures and, for SNNs, neuromorphic hardware to realize the efficiency benefits.

Motion-compensated event images

A fifth family deserves special mention because the motion estimate itself enters the representation. Warp each event along a candidate motion (camera pose trajectory, optical flow) to a common reference time before accumulating:

$H_{\mathbf{w}}(\mathbf{x}) = \sum_k p_k \cdot \delta\!\left(\mathbf{x} - \mathbf{W}(\mathbf{x}_k, t_k;\, \boldsymbol{\theta})\right)$

If the motion hypothesis $\boldsymbol{\theta}$ is correct, events generated by the same scene edge land on the same pixel and the image is sharp; if it is wrong, the image blurs. Contrast maximization turns this into an objective — choose the motion that maximizes the sharpness (e.g., variance) of $H_{\mathbf{w}}$ — and is one of the field’s fundamental model-based tools for ego-motion and flow estimation. The same idea appears as IMU-based motion compensation in ESVIO, where warping events to a reference moment sharpens scene edges before matching.

Choosing a representation

Representation	Temporal fidelity	Compatibility	Typical use
Event frame	Low (window-collapsed)	Highest (any image algorithm)	Tracking front-ends, CNNs
Time surface	Medium (recency only)	High	Registration, data association (ESVO)
Voxel grid	Medium-high (binned)	High (CNNs)	Learned VO (DEVO)
Motion-compensated image	High (uses per-event time)	Medium (needs motion estimate)	Contrast maximization, VIO front-ends
Spikes / points	Full	Low (special architectures)	SNNs, PointNet-style models

Common pitfalls when building one of these:

Fixed time window vs. fixed event count: fixed-time windows produce wildly varying event counts as motion speed changes; fixed-count windows adapt automatically but have variable latency. Many systems prefer fixed-count for this reason.
Polarity handling: summing signed polarities cancels events on edges crossed twice; separate positive/negative channels preserve more information at 2× memory.
Motion dependence: any windowed representation “looks” different at different speeds — a matcher trained or tuned at one speed regime may fail at another.

Why it matters for SLAM

Every event-based SLAM system implicitly answers “what do we do with the raw stream?” first, and the answer shapes everything downstream: event frames enabled frame-like tracking pipelines, time surfaces enabled stereo matching and map registration in ESVO, motion-compensated images underpin contrast-maximization tracking and ESVIO’s front-end, and voxel grids made it possible to reuse deep frame-based VO architectures (DPVO/DROID-style) for events in DEVO. Understanding these representations — and what each one throws away — is the fastest route to reading the event-SLAM literature critically.