UniGS-SLAM

Monocular Gaussian SLAM
with Enhanced Geometry and
Open-Vocabulary Semantics

Xingyuan Ding1,* Sicheng Yu2,* Kaiyong Zhao3 Hao Wang2,†

1 Shanghai Jiao Tong University

2 The Hong Kong University of Science and Technology (Guangzhou)

3 XGRIDS

* Equal contribution.† Corresponding author.

Paper Not public arXiv Link pending Code Link pending
Manuscript Figure 1: Gaussian map, geometric mesh and semantic voxels; quality and runtime plots; scene understanding examples.
Geometry, appearance, and open-vocabulary semantics from RGB alone. UniGS-SLAM maintains a Gaussian reconstruction and a spatially decoupled semantic voxel map, with semantic evidence revised as geometry evolves.

Abstract

Monocular semantic SLAM aims to reconstruct scene geometry and build semantic maps from RGB observations. Existing Gaussian-based methods often attach semantic labels or features to evolving Gaussian primitives, making semantic fusion difficult to maintain and increasing reconstruction overhead. Hybrid Gaussian–voxel representations provide more stable semantic support, but often rely on RGB-D input or known poses and do not revise historical semantic evidence after geometry updates.

We present UniGS-SLAM, an RGB-only framework that decouples open-vocabulary semantics from Gaussian reconstruction while preserving bidirectional interaction. Foundation priors enhance monocular depth and pose estimation, while semantic observations are fused into an independent sparse voxel map with observation provenance, enabling historical contributions to be revised as reconstruction evolves. The fused semantics further guide Gaussian filtering and local surface refinement.

Experiments on Replica, ScanNet, and KITTI-360 show leading tracking, rendering, and geometric reconstruction performance among representative RGB-only methods, together with strong open-set semantic mapping on Replica and ScanNet. We also achieve a 2.37× reconstruction speedup over the fastest compared semantic SLAM baseline.

Framework

UniGS-SLAM pipeline: foundation geometry enhancement, spatially decoupled reconstruction, semantic fusion and revision, and semantic-guided geometry refinement.
Overview of UniGS-SLAM. Our framework integrates three components: (1) Geometry-enhanced monocular SLAM uses foundation-model priors and confidence-aware fusion for reliable reconstruction; (2) Spatially decoupled open-vocabulary semantic mapping maintains an independent sparse voxel map and revises historical evidence after geometry updates; (3) Semantic-guided Gaussian geometry refinement uses fused semantics to guide floater filtering and local surface refinement.

Tracking Results

Tracking and rendering on Replica and ScanNet; RGB-only and RGB-D methods, as reported in manuscript Table 1.

Rendering Results

Replica room_0 · 20 seconds

Input RGB (left) and reconstructed RGB (right), rendered from the final Gaussian map at the corresponding estimated camera poses.

Paper figures
Figure 3: FreeOcc, ARTDECO, HI-SLAM2, Ours, and ground-truth rendering comparisons on ScanNet and KITTI-360, with magnified detail regions.
Appendix Figure 9: additional RGB rendering comparisons on ScanNet and KITTI-360, with enlarged detail regions; FreeOcc, ARTDECO, HI-SLAM2, Ours, and Ground Truth.

RGB rendering comparisons on ScanNet and KITTI-360.

Geometric Reconstruction

Replica room_0 · 3D reconstruction
Replica office_2 · 3D reconstruction

Progressive visibility replay of the final Gaussian maps with estimated camera trajectories, followed by a short orbit of each scene.

Paper figures
Figure 4: mesh comparisons between FreeOcc, Splat-SLAM, HI-SLAM2, Ours, and ground truth on Replica and ScanNet.
Appendix Figure 7: additional mesh comparisons for FreeOcc, Splat-SLAM, HI-SLAM2, Ours, and Ground Truth.

Mesh comparisons on Replica and ScanNet.

Table 2: geometric reconstruction on Replica and ScanNet using accuracy, L1 depth error, and F-score. Ours F-score: 89.84 on Replica, 52.86 on ScanNet.

Semantic Mapping

Figure 5: RGB, depth, closed-set semantics, open-set semantics, and ground truth on Replica.
Table 4: closed-set 2D segmentation on four Replica scenes, mIoU in percent. Ours average: 95.92.
Closed-set 2D segmentation (mIoU, %), using ground-truth semantic supervision with RGB-only reconstruction.
Table 5: open-set 3D semantic segmentation on a 0–1 scale. Ours mIoU: Replica 0.38, ScanNet 0.50.
Open-set 3D segmentation (0–1 scale), following the RADIO-ViPE protocol without background categories.

Efficiency

UniGS-SLAM reconstructs Replica scenes in 20.40 minutes, achieving a 2.37× speedup over the fastest compared semantic SLAM baseline.

Table 9: reconstruction times in minutes. SNI-SLAM 48.32, SGS-SLAM 118.70, SemGauss-SLAM 132.79, GS3LAM 172.40, Open-set SLAM 90.10, Ours 20.40.
Reconstruction time on Replica. Timed on RTX A6000; Open-set SLAM is a literature-derived estimate on RTX 3090.