Published April 9, 2026 | Version v1
-
1.
Friedrich-Alexander-Universität Erlangen-Nürnberg
-
2.
Centre Inria d'Université Côte d'Azur
Description
3D Gaussian Splatting (3DGS) has transformed novel-view synthesis from RGB images, yet remains restricted to the visible spectrum. Many applications, including agricultural monitoring, rely on multi-spectral imaging, where spectral camera alignment and scalability pose major challenges. We present MS-Splatting—a multi-spectral 3DGS framework enabling unified multi-view consistent reconstruction and rendering across both visible and invisible spectra. Our key component is a neural color representation that encodes per-primitive features shared across spectral bands, decoded through a shallow multi-layer perceptron into spectrum-specific radiance. By leveraging inter-band correlations, this formulation enhances detail while reducing memory consumption compared to independent band modeling via per-channel modeling with spherical harmonics. Our method enables accurate parallax-free novel-view vegetation index rendering for plant monitoring and enhances RGB novel view synthesis quality by exploiting details revealed through multi-spectral bands. Our evaluation demonstrates that MS-Splatting exceeds the current leading methods in both categories. In addition, we introduce a multi-spectral dataset from aerial captures covering outdoor environments, specifically designed for evaluating these applications. We will release our code and dataset to facilitate further research.
Notes
Note: The ms-solar dataset scene contains fewer images due to legal and privacy constraints. Specifically, all side-view images were excluded because the scene was recorded in a residential neighborhood. Consequently, this scene is provided exclusively as a top-down scene.
Code: https://github.com/j-gruen/MS-Splatting
Project Page: https://meyerls.github.io/ms_splatting/
Funding: Lukas Meyer was funded by the 5G innovation program of the German Federal Ministry for Digital and Transport under the funding code 165GU103C. Maximilian Weiherer wasfunded by the German Federal Ministry of Education and Research (BMBF), FKZ: 01IS22082 (IRRW). Linus Franke was supported in part by the ERC Advanced Grant NERPHYS (101141721, https://project.inria.fr/nerphys). The authors are responsible for the content of this publication. The authors gratefully acknowledge the scientific support and HPC resources provided by the Erlangen National High Performance Computing Center (NHR@FAU) of the Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU) under the NHR project b162dc. NHR funding is provided by federal and Bavarian state authorities. NHR@FAU hardware is partially funded by the German Research Foundation (DFG) – 440719683
Files
ms-splatting-dataset.zip