Key Concepts
- World model — A learned generative model of environment dynamics, usable for prediction and planning
- VLM vs VLA — Vision-language models reason about images; vision-language-action models additionally output robot actions
- Spatial AI — The convergence of SLAM, scene understanding, and learned world representations (Davison's FutureMapping vision)
World Models
| System | Author/Year | Key Concepts |
|---|---|---|
| GAIA-1 | Wayve 2023 | Driving World Model, action-conditioned future scene generation |
| Sora / DiT | OpenAI 2024 | Diffusion Transformer, spacetime patches, emergent 3D understanding |
| NVIDIA Cosmos | NVIDIA 2025 | World Foundation Model platform for Physical AI, synthetic data for AV/robots |
| World Labs / Marble | Fei-Fei Li 2025 | Generative 3D worlds (persistent Gaussian-splat scenes) from image/video/text prompts |
| WorldVLA | Cen (Alibaba) 2025 | Autoregressive action world model, learns physics for action generation |
| SceneDINO | Jevtić 2025 | Feed-forward unsupervised semantic scene completion |
Generative 3D
| System | Author/Year | Key Concepts |
|---|---|---|
| DreamFusion | Poole 2023 | Text-to-3D via Score Distillation Sampling (SDS) + NeRF |
Vision-Language Models (VLM)
| System | Author/Year | Key Concepts |
|---|---|---|
| CLIP | Radford (OpenAI) 2021 | Contrastive image-text pretraining, 400M pairs, zero-shot |
| SigLIP | Zhai (Google) 2023 | Sigmoid loss CLIP, more efficient, better at small model sizes |
| BLIP-2 | Li (Salesforce) 2023 | Q-Former bridges frozen LLM + image encoder |
| LLaVA | Liu 2023 | LLaMA + vision, conversational VLM |
Vision-Language-Action Models (VLA)
| System | Author/Year | Key Concepts |
|---|---|---|
| RT-2 | Brohan (DeepMind) 2023 | Robot actions as text tokens, emergent generalization |
| OpenVLA | Kim 2024 | Open-source VLA, SigLIP + Llama 7B + Action Head |
| NaVILA | Cheng 2024 | Legged/wheeled-robot vision-language-action model for navigation |
Resources
| Resource | Author/Year | Key Concepts |
|---|---|---|
| Awesome-Transformer-based-SLAM | KwanWaiPang | Curated GitHub list of Transformer-based SLAM methods |