video-vec2wav2-tokenizer-2

Version 2 - continuation shard of the video-to-AI-dataset tokenizer project.

Production-ready pipeline (Python package video_vec2wav2_tokenizer, CLI command video2dataset) that turns a folder of videos into clean AI training datasets for speech recognition (ASR) and text-to-speech (TTS).

videos  ──►  audio (16 kHz mono PCM)  ──►  whisper transcript  ──►  clips  ──►  metadata.csv / dataset.jsonl / tts_metadata.csv / report.json

• Video processing — recursive scan of mp4 / mkv / avi / mov / webm, FFmpeg audio extraction to mono · 16 kHz · 16-bit PCM WAV.
• Speech recognition — faster-whisper, CPU & CUDA, automatic language detection, word-level timestamps.
• Segmentation — cut audio by transcript timestamps into dataset/audio/000001.wav ….
• Dataset generation — metadata.csv, dataset.jsonl, tts_metadata.csv.
• Feature extraction (optional) — streaming features/train.bin + train.dat with float32 samples, mel spectrograms, duration and sample rate.
• Statistics — report.json with totals, durations and language distribution.
• Training — train_wav2vec2.py: HuggingFace Wav2Vec2 CTC with resume, multi-GPU, mixed precision and checkpointing.
• Performance — multiprocessing, batch processing, tqdm progress bars and memory-efficient streaming that scales past 1 TB of source media.

Installation

Requires Python 3.11+ and the FFmpeg binary on your PATH.

# 1. FFmpeg (one of):
#    macOS:   brew install ffmpeg
#    Ubuntu:  sudo apt install ffmpeg
#    Windows: winget install Gyan.FFmpeg   (or choco install ffmpeg)

# 2. The package
python -m venv .venv && source .venv/bin/activate     # Windows: .venv\Scripts\activate
pip install -r requirements.txt
pip install -e .                # exposes the `video2dataset` command

# Optional: training extras (torch + transformers)
pip install -e ".[train]"

Verify:

video2dataset --version
f