FNI V2.0 for Osma Bench Dataset: Authority (A:62), Popularity (P:58), Recency (R:30), Quality (Q:50). Semantic (S) is a query-time baseline scored live at search.
OSMa-Bench (Open Semantic Mapping Benchmark) dataset is a fully automatically generated dataset for evaluating the robustness of open semantic mapping and segmentation systems under varying indoor lighting conditions and robot movement dynamics. This dataset is part of OSMa-Bench pipeline.
Dataset Summary
This dataset provides simulated RGB-D and semantically annotated posed sequences for evaluation of semantic mapping and segmentation, with a particular focus on handling dynamic lightingβa critical but often overlooked factor in existing benchmarks. It also includes a collection of automatically generated questionβanswer pairs across multiple categories to support the evaluation of scene-graphβbased reasoning, offering a task-driven measure of how well a systemβs reconstructed scene captures semantic relationships between objects.
The data is built upon two base datasets:
ReplicaCAD: 22 scenes with 4 lighting configurations and a velocity modifier.
Habitat Matterport 3D (HM3D): 8 scenes with 2 lighting configurations and a velocity modifier.
Installation
We offer two versions of the dataset: one with separate files and one as a single compressed archive. Use the following command to download the separated files (may be slow):
bash
git xet install
git clone https://huggingface.co/datasets/warmhammer/OSMa-Bench_dataset -b main
and this command to download the compressed version (faster one):