Project Page  •  arXiv  •  Installation  •  Data Preprocessing  •  Training  •  Citation

Ellina Zhang, Madhavan Iyengar, Amir Zadeh, Chuan Li, David Held, Deepak Pathak, Tal Daniel
Carnegie Mellon University  •  Lambda AI  •  ICML 2026

Abstract

We introduce 3D-DLP, a self-supervised object-centric representation learning model that decomposes scene-level RGB-D or voxel observations into a set of 3D latent particles. Building on the Deep Latent Particles (DLP) framework, each particle encodes disentangled attributes — 3D keypoint position, bounding-box dimensions, and appearance features — and represents a distinct entity in the scene. The model learns interpretable per-particle segmentation maps through an end-to-end self-supervised reconstruction objective. On both simulated and real-world datasets, the learned latent space is interpretable and controllable: by manipulating particle positions and decoding, we can generate novel scene configurations. Leveraging these compact 3D latent particles for downstream robotic manipulation improves performance over baselines that either lack explicit 3D information or rely on memory-intensive dense 3D inputs without object-centric structure.

Contents

• Overview
• Installation
• Repository Organization
• Datasets
• Data Preprocessing
• Configuration Files
• Training
• From 3D-DLP to Particle Tokens
• Downstream Policy Training (EC-Diffuser)
• Documentation
• Citation
• Acknowledgements
• License

Overview

3D-DLP encodes a scene-level 3D observation into a fixed set of