panodata

panodata/omniload

omniload is a polyglot data loader based on dlt. Copy data between any source and any destination.

0 stars 2 forks 0 watchers Python MIT License
data-engineering data-ingestion data-lake data-load data-loaders data-loading data-loading-and-analysis data-loading-and-cleaning data-loading-framework data-pipeline data-warehouse dataloader dlt dlt-destination etl etl-framework warehouse warehouse-management
View on GitHub Website
18 Open Issues Need Help Last updated: Jul 3, 2026

Open Issues Need Help

View All on GitHub
The new canonical `file://` destination about 10 hours ago
help wanted connector refinement needs discussion
panodata/omniload
0

omniload is a polyglot data loader based on dlt. Copy data between any source and any destination.

Python
#data-engineering#data-ingestion#data-lake#data-load#data-loaders#data-loading#data-loading-and-analysis#data-loading-and-cleaning#data-loading-framework#data-pipeline#data-warehouse#dataloader#dlt#dlt-destination#etl#etl-framework#warehouse#warehouse-management
Filesystem: Clarify `filesystem_incremental` option ownership in fsspec sources 1 day ago
help wanted question
panodata/omniload
0

omniload is a polyglot data loader based on dlt. Copy data between any source and any destination.

Python
#data-engineering#data-ingestion#data-lake#data-load#data-loaders#data-loading#data-loading-and-analysis#data-loading-and-cleaning#data-loading-framework#data-pipeline#data-warehouse#dataloader#dlt#dlt-destination#etl#etl-framework#warehouse#warehouse-management
Filesystem: Migrate from `HttpSource` to `fsspec.HTTPFileSystem` 1 day ago
help wanted refinement
panodata/omniload
0

omniload is a polyglot data loader based on dlt. Copy data between any source and any destination.

Python
#data-engineering#data-ingestion#data-lake#data-load#data-loaders#data-loading#data-loading-and-analysis#data-loading-and-cleaning#data-loading-framework#data-pipeline#data-warehouse#dataloader#dlt#dlt-destination#etl#etl-framework#warehouse#warehouse-management
Database: Support 3-component table names (`catalog.schema.table` aka. `project.dataset.table`) 1 day ago
help wanted refinement
panodata/omniload
0

omniload is a polyglot data loader based on dlt. Copy data between any source and any destination.

Python
#data-engineering#data-ingestion#data-lake#data-load#data-loaders#data-loading#data-loading-and-analysis#data-loading-and-cleaning#data-loading-framework#data-pipeline#data-warehouse#dataloader#dlt#dlt-destination#etl#etl-framework#warehouse#warehouse-management
Database: Load file-based databases from remote filesystems 3 days ago
help wanted connector refinement
panodata/omniload
0

omniload is a polyglot data loader based on dlt. Copy data between any source and any destination.

Python
#data-engineering#data-ingestion#data-lake#data-load#data-loaders#data-loading#data-loading-and-analysis#data-loading-and-cleaning#data-loading-framework#data-pipeline#data-warehouse#dataloader#dlt#dlt-destination#etl#etl-framework#warehouse#warehouse-management
[Deployment] Consume managed secrets 11 days ago
help wanted readiness
panodata/omniload
0

omniload is a polyglot data loader based on dlt. Copy data between any source and any destination.

Python
#data-engineering#data-ingestion#data-lake#data-load#data-loaders#data-loading#data-loading-and-analysis#data-loading-and-cleaning#data-loading-framework#data-pipeline#data-warehouse#dataloader#dlt#dlt-destination#etl#etl-framework#warehouse#warehouse-management
Filesystem: Integration tests for remote filesystems 15 days ago
help wanted quality
panodata/omniload
0

omniload is a polyglot data loader based on dlt. Copy data between any source and any destination.

Python
#data-engineering#data-ingestion#data-lake#data-load#data-loaders#data-loading#data-loading-and-analysis#data-loading-and-cleaning#data-loading-framework#data-pipeline#data-warehouse#dataloader#dlt#dlt-destination#etl#etl-framework#warehouse#warehouse-management
Object Storage Service (OSS) `oss://` 19 days ago
help wanted connector
panodata/omniload
0

omniload is a polyglot data loader based on dlt. Copy data between any source and any destination.

Python
#data-engineering#data-ingestion#data-lake#data-load#data-loaders#data-loading#data-loading-and-analysis#data-loading-and-cleaning#data-loading-framework#data-pipeline#data-warehouse#dataloader#dlt#dlt-destination#etl#etl-framework#warehouse#warehouse-management
Incremental loading for filesystem resources 19 days ago
help wanted question refinement
panodata/omniload
0

omniload is a polyglot data loader based on dlt. Copy data between any source and any destination.

Python
#data-engineering#data-ingestion#data-lake#data-load#data-loaders#data-loading#data-loading-and-analysis#data-loading-and-cleaning#data-loading-framework#data-pipeline#data-warehouse#dataloader#dlt#dlt-destination#etl#etl-framework#warehouse#warehouse-management
Filesystem: Migrate from `fsspec.LocalFileSystem` to `pyarrow.fs.LocalFileSystem` 20 days ago
help wanted performance
panodata/omniload
0

omniload is a polyglot data loader based on dlt. Copy data between any source and any destination.

Python
#data-engineering#data-ingestion#data-lake#data-load#data-loaders#data-loading#data-loading-and-analysis#data-loading-and-cleaning#data-loading-framework#data-pipeline#data-warehouse#dataloader#dlt#dlt-destination#etl#etl-framework#warehouse#warehouse-management
Filesystem: Read _multiple_ spreadsheets from workbook files 20 days ago
help wanted refinement
panodata/omniload
0

omniload is a polyglot data loader based on dlt. Copy data between any source and any destination.

Python
#data-engineering#data-ingestion#data-lake#data-load#data-loaders#data-loading#data-loading-and-analysis#data-loading-and-cleaning#data-loading-framework#data-pipeline#data-warehouse#dataloader#dlt#dlt-destination#etl#etl-framework#warehouse#warehouse-management
Filesystem: Migrate remote filesystem adapters from fsspec to pyarrow.fs (Azure Blob Storage, GCS, HDFS, S3) 21 days ago
help wanted performance
panodata/omniload
0

omniload is a polyglot data loader based on dlt. Copy data between any source and any destination.

Python
#data-engineering#data-ingestion#data-lake#data-load#data-loaders#data-loading#data-loading-and-analysis#data-loading-and-cleaning#data-loading-framework#data-pipeline#data-warehouse#dataloader#dlt#dlt-destination#etl#etl-framework#warehouse#warehouse-management
ADBC 24 days ago
help wanted pitch connector
panodata/omniload
0

omniload is a polyglot data loader based on dlt. Copy data between any source and any destination.

Python
#data-engineering#data-ingestion#data-lake#data-load#data-loaders#data-loading#data-loading-and-analysis#data-loading-and-cleaning#data-loading-framework#data-pipeline#data-warehouse#dataloader#dlt#dlt-destination#etl#etl-framework#warehouse#warehouse-management
Filesystem: Document current incremental loading mechanism about 1 month ago
documentation help wanted refinement
panodata/omniload
0

omniload is a polyglot data loader based on dlt. Copy data between any source and any destination.

Python
#data-engineering#data-ingestion#data-lake#data-load#data-loaders#data-loading#data-loading-and-analysis#data-loading-and-cleaning#data-loading-framework#data-pipeline#data-warehouse#dataloader#dlt#dlt-destination#etl#etl-framework#warehouse#warehouse-management
Azure Data Lake Storage Gen2 `adls://` about 1 month ago
help wanted connector
panodata/omniload
0

omniload is a polyglot data loader based on dlt. Copy data between any source and any destination.

Python
#data-engineering#data-ingestion#data-lake#data-load#data-loaders#data-loading#data-loading-and-analysis#data-loading-and-cleaning#data-loading-framework#data-pipeline#data-warehouse#dataloader#dlt#dlt-destination#etl#etl-framework#warehouse#warehouse-management
Azure Blob Storage `az://` about 1 month ago
help wanted connector
panodata/omniload
0

omniload is a polyglot data loader based on dlt. Copy data between any source and any destination.

Python
#data-engineering#data-ingestion#data-lake#data-load#data-loaders#data-loading#data-loading-and-analysis#data-loading-and-cleaning#data-loading-framework#data-pipeline#data-warehouse#dataloader#dlt#dlt-destination#etl#etl-framework#warehouse#warehouse-management
Filesystem: Named hint parameters for URIs about 1 month ago
help wanted connector refinement
panodata/omniload
0

omniload is a polyglot data loader based on dlt. Copy data between any source and any destination.

Python
#data-engineering#data-ingestion#data-lake#data-load#data-loaders#data-loading#data-loading-and-analysis#data-loading-and-cleaning#data-loading-framework#data-pipeline#data-warehouse#dataloader#dlt#dlt-destination#etl#etl-framework#warehouse#warehouse-management
ValueError: Unsupported destination scheme: file about 1 month ago
enhancement help wanted connector
panodata/omniload
0

omniload is a polyglot data loader based on dlt. Copy data between any source and any destination.

Python
#data-engineering#data-ingestion#data-lake#data-load#data-loaders#data-loading#data-loading-and-analysis#data-loading-and-cleaning#data-loading-framework#data-pipeline#data-warehouse#dataloader#dlt#dlt-destination#etl#etl-framework#warehouse#warehouse-management