Dataset Card for R³ Bayer Dataset

Table of Contents

• Dataset Description
  • Dataset Summary
  • Supported Tasks and Leaderboards
• Dataset Structure
  • Data Instances
  • Data Fields
  • Data Splits
  • File Structure Mapping
• Dataset Creation
  • Curation Rationale
  • Source Data
• Additional Information
  • Licensing Information
  • Citation Information

Dataset Description

• Homepage: https://arxiv.org/abs/2509.24022
• Paper: R³: Reconstruction, Raw, and Rain: Deraining Directly in the Bayer Domain

Dataset Summary

The R³ (Reconstruction, Raw, and Rain) Bayer Dataset is a large-scale, real-world stereo dataset designed for deraining tasks directly in the raw Bayer domain. It was collected using a dual-camera synchronized setup capturing raw Bayer images. Unlike popular synthetic datasets, R³ uses a real-world rain simulation system to generate realistic rain artifacts, including scene depth effects and droplets on windshields. The dataset captures diverse scenarios, including varying lighting conditions (day to low-light evening), scenes behind glass, moving vehicles, and active windshield wipers, enabling the evaluation of model robustness to dynamic occlusions and motion artifacts.

The RAW-RAIN-bayer Dataset is one part of a two part dataset, the other being RAW-RAIN-rgb Dataset.

Both Datasets are released within the context of research done for our paper: https://arxiv.org/abs/2509.24022

This dataset contains the raw .pgm images capturing the Bayer pattern data before ISP processing.

Supported Tasks and Leaderboards

• Raw Image Deraining: Removing rain streaks and droplets directly from raw sensor data.
• Stereo Vision/Depth Estimation in Adverse Weather: utilizing the stereo pairs for reconstruction tasks under rainy conditions.
• Multi-camera low-level vision.

Dataset Structure

Data Instances

A typical data instance represents a single synchronized stereo frame. It includes the raw Bayer image corrupted by rain, its corresponding ground truth raw Bayer image (static scene), and associated metadata.

Data Fields

Under each scene gt/rain you can find the metadata.csv which holds the following fields:

• frame: integer, the sequence number of the frame within the scene.
• camera_id: string, "camera_0" (right camera) or "camera_1" (left camera).
• timestamp: float, capture time from metadata.
• exposure: float, exposure time in ms.
• analog_gain: float, fixed gain value (set to 1 during collection).

Data Splits

The dataset is divided into Train, Identity, Test, and Validation splits.

File Structure Mapping

The underlying data is stored in a nested folder structure. This dataset loader flattens the structure into individual examples. The original structure is as follows:

root/
├── [split_name]/   # e.g., train, test, identity
│      ├── [split_name]_[scene_number]/   # e.g., train_1, test_3
│      ├── metadata.csv               # Contains: frameId, timestamp, exposure, analogGain
│      ├── gt/                        # Ground Truth - Static Scene
│      │   ├── camera_0/              # Right Camera
│      │   │   └── [raw_images].pgm   # Any image here serves as static GT
│      │   └── camera_1/              # Left Camera
│      │       └── [raw_images].pgm
│      └── rain/                      # Rainy Scenario - Video Sequence
│          ├── camera_0/              # Right Camera
│          │   ├── raw_9.pgm
│          │   ├── raw_10.pgm
│          │   └── ...
│          └── camera_1/              # Left Camera
│              ├── raw_9.pgm
│              ├── raw_10.pgm
│              └── ...

Note: For stereo matching, gt/camera_0 corresponds to rain/camera_0, and gt/camera_1 corresponds to rain/camera_1.

Dataset Creation

Curation Rationale

To support studies on deraining directly in raw space, a custom stereo dataset was required that captures real-world rain physics rather than synthetic overlays, while ensuring temporal synchronization and providing ground truth.

Source Data

Data Collection Process: The dataset was collected using a rig with two synchronized FLIR Blackfly S 2.8MP cameras capable of capturing raw Bayer images.

• Protocol: Ground truth images were collected first using a spatially stable rig. Immediately after, a custom rain system was triggered to scatter water-rain-like droplets on the windshield and in the scene depth while recording the "rain" frames.
• Settings: Fixed analog gain of 1. Exposure times varied (300ms - 10,000ms) to capture bright daylight to low-light evening conditions.
• Scenarios: Included recordings through stationary glass windows, from within moving vehicles, and sessions with windshield wipers operating at varying speeds.

Additional Information

Licensing Information

CC-BY-4.0

Citation Information

@misc{rothschild2025mathbfr3reconstructionrawrain,
      title={$\mathbf{R}^3$: Reconstruction, Raw, and Rain: Deraining Directly in the Bayer Domain}, 
      author={Nate Rothschild and Moshe Kimhi and Avi Mendelson and Chaim Baskin},
      year={2025},
      eprint={2509.24022},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={[https://arxiv.org/abs/2509.24022](https://arxiv.org/abs/2509.24022)}, 
}