LUDVIG: Learning-free Uplifting of 2D Visual features to Gaussian Splatting scenes (ICCV 2025)

This repository contains code for LUDVIG: Learning-free Uplifting of 2D Visual features to Gaussian Splatting scenes, published at ICCV 2025. LUDVIG uses a learning-free approach to uplift visual features from models such as DINOv2, SAM, and CLIP into 3D Gaussian Splatting scenes. It refines 3D features, such as coarse segmentation masks, based on a graph diffusion process that incorporates the 3D geometry of the scene and DINOv2 feature similarities. We evaluate on foreground/background and open-vocabulary object segmentation tasks.

Illustration of the inverse and forward rendering between 2D visual features (produced by DINOv2) and a 3D Gaussian Splatting scene. In the inverse rendering (or uplifting) phase, features are created for each 3D Gaussian by aggregating coarse 2D features over all viewing directions. For forward rendering, the 3D features are projected on any given viewing direction as in regular Gaussian Splatting.

Table of Contents

1. Setup
2. Project Structure
3. Demo
   • Demo for feature uplifting
   • Demo for open-vocabulary object removal
4. Reproducing results
   • Foreground/background segmentation
   • Open-vocabulary object segmentation
   • Open-vocabulary semantic segmentation
5. Citing LUDVIG
6. License

Setup

Clone the repo and cd into it

git clone git@github.com:naver/ludvig.git
cd ludvig

Run the following script to set the paths to your cuda dependencies, e.g. cuda_path=/usr/local/cuda-11.8:

bash script/set_cuda.sh ${cuda_path}

Modify the pytorch-cuda version in environment.yml to match your CUDA version, and create the ludvig environment:

mamba env create -f environment.yml

Our code has been tested on Ubuntu 22, CUDA 11.8 with GPU A6000 ADA (48GB of memory).

Project Structure

The project in ludvig/ is as follows:

• ludvig_*.py: Main scripts for uplifting, graph diffusion and evaluation.
• scripts/: Bash scripts calling ludvig_*.py
• configs/: Configuration files for different models and evaluation tasks.
• diffusion/: Classes for graph diffusion.
• evaluation/: Classes for evaluation, including segmentation on NVOS and SPIn-NeRF with SAM and DINOv2.

Additionally, you should have the following folders in ludvig/ (e.g. as symbolic links to storage locations):

• checkpoints/: Where model checkpoints should be stored. Our experiments use DINOv2 ViT-g with registers, SAM ViT-H, SAM2 Hiera Large.
• dataset/: Where datasets should be stored.
• logs/: Where logs will be stored.

Demo

Data

For this demo, we use the stump and bonsai scenes from Mip-NeRF 360, with the pretrained Gaussian Splatting representation provided by the authors of Gaussian Splatting.
First, download the scene and weights:

bash script/demo_download.sh

This saves the data in dataset/stump,dataset/bonsai and model weights in dataset/stump/gs, dataset/bonsai/gs.

Demo for feature uplifting

The following script will uplift DINOv2 features and save visualizations of the uplifted features:

python demo.py

The script creates an instance of ludvig_uplift.LUDVIGUplift based on paths to the data and on the configuration configs/demo.yaml.
It then runs uplifting through model.uplift() and saves 3D features and visualizations through model.save().

Feature map generation and uplifting

The method model.uplift():

• creates a dataset from a subclass of predictors.base.BaseDataset that generates the feature maps to be uplifted

• calls utils.solver.uplifting to uplift the 2D feature maps generated by the dataset.

  See details on the uplifting function

  The function utils.solver.uplifting takes the following arguments:

  • loader: An iterable (in our case, an instance of DINOv2Dataset) that should yield (feature, camera) pairs.

    • feature: A tensor of shape