OVSplat: Feed-Forward Open-Vocabulary Voxel-Aligned Gaussian Splatting

Open-vocabulary 3D scene understanding aims to associate arbitrary natural-language queries with 3D structure, enabling text-driven scene interpretation beyond a fixed label set. Methods that embed CLIP-aligned language features into 3D Gaussian Splatting, such as LERF and LangSplat, achieve this by optimizing a dedicated representation for every scene, inheriting a minutes-to-hours cost that is impractical at scale. Feed-forward alternatives remove this cost but, to date, either predict only geometry and appearance or build their semantic branch on a pixel-aligned backbone, inheriting its sensitivity to occlusion, low texture, and cross-view misalignment. This thesis introduces OVSplat, a feed-forward, voxel-aligned framework that predicts open-vocabulary, CLIP-aligned language embeddings for every occupied voxel of a 3D Gaussian scene in the same forward pass used for geometry, without any per-scene optimization. Built on the VolSplat reconstruction backbone, OVSplat first fine-tunes it on ScanNet++, addressing view-sampling and catastrophic-forgetting issues specific to indoor, sparsely captured scenes, and then introduces a sparse-voxel language head arrived at through a documented sequence of architectural iterations, each motivated by a specific failure of its predecessor, together with a loss formulation that reconciles photometric and purely directional, cosine-based semantic supervision within a shared sparse backbone. Because no CLIP-annotated version of ScanNet++ existed, this thesis also builds and releases a two-stage SAM-and-CLIP annotation pipeline covering 350 scenes.