annotations_creators:

• machine-generated
  language:
• en
  license: cc-by-4.0
  multilinguality:
• monolingual
  size_categories:
• 10K<n<100K
  source_datasets:
• original
  task_categories:
• image-to-image
  tags:
• computer-vision
• raw-image
• bayer-pattern
• stereo-vision
• image-deraining
• rain-removal
• real-world-rain
  pretty_name: R³ (Reconstruction, Raw, and Rain) - RGB Domain

Dataset Card for R³ RGB Dataset

Table of Contents

• Dataset Description
  • Dataset Summary
  • Supported Tasks and Leaderboards
• Dataset Structure
  • Data Instances
  • Data Fields
  • Data Splits
  • File Structure Mapping
• Dataset Creation
  • Curation Rationale
  • Source Data
• Additional Information
  • Licensing Information
  • Citation Information

Dataset Description

• Homepage: https://arxiv.org/abs/2509.24022
• Paper: R³: Reconstruction, Raw, and Rain: Deraining Directly in the Bayer Domain

Dataset Summary

The R³ (Reconstruction, Raw, and Rain) RGB Dataset is a large-scale, real-world stereo dataset designed for deraining tasks directly in the raw sRGB domain. It was collected using a dual-camera synchronized setup capturing raw Bayer images, which under went a software ISP. Unlike popular synthetic datasets, R³ uses a real-world rain simulation system to generate realistic rain artifacts, including scene depth effects and droplets on windshields. The dataset captures diverse scenarios, including varying lighting conditions (day to low-light evening), scenes behind glass, moving vehicles, and active windshield wipers, enabling the evaluation of model robustness to dynamic occlusions and motion artifacts.

The RAW-RAIN-rgb Dataset is one part of a two part dataset, the other being RAW-RAIN-bayer Dataset.

Both Datasets are released within the context of research done for our paper: https://arxiv.org/abs/2509.24022

This dataset contains the raw .pgm images capturing the Bayer pattern data then processing them via a software ISP.

Supported Tasks and Leaderboards

• Raw Image Deraining: Removing rain streaks and droplets directly from raw sensor data.
• Stereo Vision/Depth Estimation in Adverse Weather: utilizing the stereo pairs for reconstruction tasks under rainy conditions.
• Multi-camera low-level vision.

Dataset Structure

Data Instances

A typical data instance represents a single synchronized stereo frame. It includes the raw Bayer image corrupted by rain, its corresponding ground truth raw Bayer image (static scene), and associated metadata.

Data Fields

Under each scene gt/rain you can find the metadata.csv which holds the following fields:

• frame: integer, the sequence number of the frame within the scene.
• camera_id: string, "camera_0" (right camera) or "camera_1" (left camera).
• timestamp: float, capture time from metadata.
• exposure: float, exposure time in ms.
• analog_gain: float, fixed gain value (set to 1 during collection).

Data Splits

The dataset is divided into Train, Identity, Test, and Validation splits.

File Structure Mapping

The underlying data is stored in a nested folder structure. This dataset loader flattens the structure into individual examples. The original structure is as follows:

root/
├── [split_name]/   # e.g., train, test, identity
│      ├── [split_name]_[scene_number]/   # e.g., train_1, test_3
│      ├── metadata.csv               # Contains: frameId, timestamp, exposure, analogGain
│      ├── gt/                        # Ground Truth - Static Scene
│      │   ├── camera_0/              # Right Camera
│      │   │   └── blue_channel   # blue channel folder
│      │   │   │   └── blue_channel_12bit_frame_[frame_num].tiff   # Any image here serves as static GT
│      │   │   └── green_channel   # Green channel folder
│      │   │   │   └── green_channel_12bit_frame_[frame_num].tiff  # To create a rgb image you must concat the same frame across the channels
│      │   │   └── red_channel   # red channel folder
│      │   │   │   └── red_channel_12bit_frame_[frame_num].tiff    # To create a rgb image you must concat the same frame across the channels
│      │   │   └── ccm_info.csv   # contains information about the color correction matrix (ccm) used on the bayer (from RAW-RAIN-bayer) to create the RGB image
│      │   │   └── wb_info.csv   # contains information about the white balance (wb) used on the bayer (from RAW-RAIN-bayer) to create the RGB image
│      │   └── camera_1/              # Left Camera
│      │       └── ...   # same structure as camera_0
│      └── rain/                      # Rainy Scenario - Video Sequence
│          ├── camera_0/              # Right Camera
│             └── blue_channel   # blue channel folder
│      │   │   │   └── blue_channel_12bit_frame_[frame_num].tiff   # These frames are in sequence
│      │   │   └── green_channel   # Green channel folder
│      │   │   │   └── green_channel_12bit_frame_[frame_num].tiff  # To create a rgb image you must concat the same frame across the channels
│      │   │   └── red_channel   # red channel folder
│      │   │   │   └── red_channel_12bit_frame_[frame_num].tiff    # To create a rgb image you must concat the same frame across the channels
│      │   │   └── ccm_info.csv   # contains information about the color correction matrix (ccm) used on the bayer (from RAW-RAIN-bayer) to create the RGB image
│      │   │   └── wb_info.csv   # contains information about the white balance (wb) used on the bayer (from RAW-RAIN-bayer) to create the RGB image
│          └── camera_1/              # Left Camera
│             └── ...   # same structure as camera_0

Note: For stereo matching, gt/camera_0 corresponds to rain/camera_0, and gt/camera_1 corresponds to rain/camera_1.

Dataset Creation

Curation Rationale

To support studies on deraining directly in raw space, a custom stereo dataset was required that captures real-world rain physics rather than synthetic overlays, while ensuring temporal synchronization and providing ground truth.

Source Data

Data Collection Process:
The dataset was collected using a rig with two synchronized FLIR Blackfly S 2.8MP cameras capable of capturing raw Bayer images.

• Protocol: Ground truth images were collected first using a spatially stable rig. Immediately after, a custom rain system was triggered to scatter water-rain-like droplets on the windshield and in the scene depth while recording the "rain" frames.
• Settings: Fixed analog gain of 1. Exposure times varied (300ms - 10,000ms) to capture bright daylight to low-light evening conditions.
• Scenarios: Included recordings through stationary glass windows, from within moving vehicles, and sessions with windshield wipers operating at varying speeds.

Software ISP pipeline:
The RGB data in this dataset was generated from the original raw Bayer data (RAW-RAIN-bayer) via a controlled software Image Signal Processing (ISP) pipeline to ensure high-quality, standard sRGB output. The pipeline steps are applied sequentially as follows:

• Black Level Correction (BLC): A fixed pedestal value, determined by sensor calibration, is subtracted from the raw Bayer pixel values to establish true black.

• Lens Shading: A calibrated correction mask is multiplied with the raw data to compensate for optical vignetting (brightness fall-off towards the image corners) caused by the lens, ensuring uniform brightness across the field of view.

• Demosaicing: The Bayer pattern is interpolated using a high-quality demosaicing algorithm to reconstruct full RGB values for every pixel.

• White Balance (WB): Per-scene specific gains are applied to the Red and Blue Bayer channels to achieve neutral color balance given the scene's illumination. The exact gains used for each frame are recorded in the corresponding wb_info.csv files.

• Color Correction Matrix (CCM): A 3x3 transformation matrix is applied to map the camera-specific sensor colors into the standard sRGB color space. The specific matrix used is stored in the ccm_info.csv files.

• Tone Mapping: A global tone curve is applied to compress the captured high dynamic range data into a usable range, preventing excessive clipping of highlights while preserving shadow details, prior to gamma correction.

• Gamma Correction: A standard sRGB gamma curve (approximately γ=2.2) is applied to convert the linear data into non-linear values suitable for display.

• Channel Splitting: The final RGB image is split into three separate 16-bit TIFF files (Red, Green, and Blue channels) as presented in the dataset structure.

Additional Information

Licensing Information

CC-BY-4.0

Citation Information

@misc{rothschild2025mathbfr3reconstructionrawrain,
      title={$\mathbf{R}^3$: Reconstruction, Raw, and Rain: Deraining Directly in the Bayer Domain}, 
      author={Nate Rothschild and Moshe Kimhi and Avi Mendelson and Chaim Baskin},
      year={2025},
      eprint={2509.24022},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={[https://arxiv.org/abs/2509.24022](https://arxiv.org/abs/2509.24022)}, 
}