Fast and memory-efficient webdataset shard reader - Rust core with Python bindings, streaming metadata, and a bucketed dataloader.

bindings cli dataloader machine-learning python pytorch rust shard streaming webdataset
1 Open Issue Need Help Last updated: Sep 10, 2026

Open Issues Need Help

View All on GitHub
enhancement help wanted

Fast and memory-efficient webdataset shard reader - Rust core with Python bindings, streaming metadata, and a bucketed dataloader.

Rust
#bindings#cli#dataloader#machine-learning#python#pytorch#rust#shard#streaming#webdataset