Open Issues Need Help
View All on GitHub Evaluate true out-of-core high-level deduplication after 10M evidence about 2 months ago
help wanted priority: p1 area: architecture status: blocked
Similarity indexing and deduplication for large datasets, with Rust, Python, and CLI interfaces.
Rust
#data-quality#datasets#deduplication#locality-sensitive-hashing#lsh#minhash#python#rust#similarity-search#training-data
Run the dedicated 10M scale campaign on native Linux about 2 months ago
help wanted priority: p1 area: benchmarks area: scale status: blocked
Similarity indexing and deduplication for large datasets, with Rust, Python, and CLI interfaces.
Rust
#data-quality#datasets#deduplication#locality-sensitive-hashing#lsh#minhash#python#rust#similarity-search#training-data