apache/datafusion-comet

[EPIC] Replace hand-rolled native code with existing arrow-rs kernels 1 day ago

enhancement good first issue EPIC

1.2K

Apache DataFusion Comet Spark Accelerator

Scala

#arrow#datafusion#rust#spark

Nested array comparison does not match Spark for signed zero 1 day ago

good first issue correctness priority:high

apache/datafusion-comet

1.2K

Apache DataFusion Comet Spark Accelerator

Scala

#arrow#datafusion#rust#spark

Wide-decimal overflow reports a different value than Spark 3 days ago

good first issue priority:medium

apache/datafusion-comet

1.2K

Apache DataFusion Comet Spark Accelerator

Scala

#arrow#datafusion#rust#spark

chore: Fix Scala code warnings 5 days ago

enhancement good first issue help wanted build

apache/datafusion-comet

1.2K

Apache DataFusion Comet Spark Accelerator

Scala

#arrow#datafusion#rust#spark

Remove timezone.rs copy of arrow-array Tz (now public in arrow 58) 6 days ago

enhancement good first issue

apache/datafusion-comet

1.2K

Apache DataFusion Comet Spark Accelerator

Scala

#arrow#datafusion#rust#spark

Object store cache key drops the ABFS container, so two containers in one storage account share a store instance 8 days ago

bug help wanted correctness priority:critical area:scan

apache/datafusion-comet

1.2K

Apache DataFusion Comet Spark Accelerator

Scala

#arrow#datafusion#rust#spark

Add `CONTRIBUTING.md` with link to the contributor guide about 1 month ago

documentation enhancement good first issue priority:low

apache/datafusion-comet

1.2K

Apache DataFusion Comet Spark Accelerator

Scala

#arrow#datafusion#rust#spark

Support `MapType` for `ElementAt` about 2 months ago

good first issue priority:medium

apache/datafusion-comet

1.2K

Apache DataFusion Comet Spark Accelerator

Scala

#arrow#datafusion#rust#spark

chore: Improve cast MapType to String about 2 months ago

good first issue priority:medium

apache/datafusion-comet

1.2K

Apache DataFusion Comet Spark Accelerator

Scala

#arrow#datafusion#rust#spark

[Feature] CAST(MapType AS MapType) falls back even though native cast_map_to_map exists about 2 months ago

enhancement good first issue priority:medium area:expressions

apache/datafusion-comet

1.2K

Apache DataFusion Comet Spark Accelerator

Scala

#arrow#datafusion#rust#spark

[Feature] support size() for MapType inputs 2 months ago

enhancement good first issue priority:medium area:expressions

apache/datafusion-comet

1.2K

Apache DataFusion Comet Spark Accelerator

Scala

#arrow#datafusion#rust#spark

[Incompatibility] Verify array_contains null handling matches Spark 2 months ago

enhancement good first issue array expressions

apache/datafusion-comet

1.2K

Apache DataFusion Comet Spark Accelerator

Scala

#arrow#datafusion#rust#spark

Add native implementations of`RegExpExtract` and `RegExpExtractAll` 3 months ago

enhancement good first issue expression

apache/datafusion-comet

1.2K

Apache DataFusion Comet Spark Accelerator

Scala

#arrow#datafusion#rust#spark

`arrays_except` type mismatch 3 months ago

bug good first issue priority:medium area:expressions

apache/datafusion-comet

1.2K

Apache DataFusion Comet Spark Accelerator

Scala

#arrow#datafusion#rust#spark

Use `datafusion-spark` version of `date_add` and `date_sub` 3 months ago

enhancement good first issue

apache/datafusion-comet

1.2K

Apache DataFusion Comet Spark Accelerator

Scala

#arrow#datafusion#rust#spark

Write documentation for working with Comet's spark-4.0 profile in IntelliJ 3 months ago

enhancement good first issue spark 4

apache/datafusion-comet

1.2K

Apache DataFusion Comet Spark Accelerator

Scala

#arrow#datafusion#rust#spark

Reduce Iceberg CI matrix: pin JDK per Spark version 3 months ago

enhancement good first issue priority:low area:ci area:Iceberg

apache/datafusion-comet

1.2K

Apache DataFusion Comet Spark Accelerator

Scala

#arrow#datafusion#rust#spark

Add Java 21 to the supported version matrix (particularly for Spark 4.0) 3 months ago

good first issue spark 4 priority:low area:ci

apache/datafusion-comet

1.2K

Apache DataFusion Comet Spark Accelerator

Scala

#arrow#datafusion#rust#spark

Reduce macOS CI matrix 3 months ago

enhancement good first issue priority:low area:ci

apache/datafusion-comet

1.2K

Apache DataFusion Comet Spark Accelerator

Scala

#arrow#datafusion#rust#spark

[Feature] Support Spark expression: levenshtein 3 months ago

enhancement good first issue

apache/datafusion-comet

1.2K

Apache DataFusion Comet Spark Accelerator

Scala

#arrow#datafusion#rust#spark

Improve documentation for configs related to explain/fallback 3 months ago

documentation enhancement good first issue

apache/datafusion-comet

1.2K

Apache DataFusion Comet Spark Accelerator

Scala

#arrow#datafusion#rust#spark

[EPIC] Add support for Spark 4.0.1 3 months ago

enhancement help wanted spark 4 priority:high

apache/datafusion-comet

1.2K

Apache DataFusion Comet Spark Accelerator

Scala

#arrow#datafusion#rust#spark

[EPIC] Spark-compatible cast / try_cast operations 3 months ago

enhancement help wanted spark 4

apache/datafusion-comet

1.2K

Apache DataFusion Comet Spark Accelerator

Scala

#arrow#datafusion#rust#spark

Add support for `MapFromEntries` 4 months ago

enhancement good first issue expression

apache/datafusion-comet

1.2K

Apache DataFusion Comet Spark Accelerator

Scala

#arrow#datafusion#rust#spark

Implement Spark-compatible CAST from String to Decimal 4 months ago

enhancement good first issue help wanted

apache/datafusion-comet

1.2K

Apache DataFusion Comet Spark Accelerator

Scala

#arrow#datafusion#rust#spark

[Incompatibility] Document array_join null handling differences 4 months ago

documentation enhancement good first issue array expressions

apache/datafusion-comet

1.2K

Apache DataFusion Comet Spark Accelerator

Scala

#arrow#datafusion#rust#spark

We do not respect ignoreNulls in first_value / last_value aggregates 4 months ago

bug good first issue correctness priority:critical area:aggregation

apache/datafusion-comet

1.2K

Apache DataFusion Comet Spark Accelerator

Scala

#arrow#datafusion#rust#spark

`array_overlap` correctness issue 4 months ago

enhancement good first issue correctness

apache/datafusion-comet

1.2K

Apache DataFusion Comet Spark Accelerator

Scala

#arrow#datafusion#rust#spark

Add dictionary support to unhex and test dictionary and scalar cases 4 months ago

bug good first issue priority:low area:expressions

apache/datafusion-comet

1.2K

Apache DataFusion Comet Spark Accelerator

Scala

#arrow#datafusion#rust#spark

`native_datafusion` reports twice more files and bytes scanned 4 months ago

bug good first issue native_datafusion priority:medium

apache/datafusion-comet

1.2K

Apache DataFusion Comet Spark Accelerator

Scala

#arrow#datafusion#rust#spark

[Bug] array_remove returns null when element is null instead of removing null elements 4 months ago

enhancement good first issue correctness array expressions

apache/datafusion-comet

1.2K

Apache DataFusion Comet Spark Accelerator

Scala

#arrow#datafusion#rust#spark

`array_union` correctness issue 4 months ago

enhancement good first issue correctness

apache/datafusion-comet

1.2K

Apache DataFusion Comet Spark Accelerator

Scala

#arrow#datafusion#rust#spark

[EPIC] Migrate to functions in `datafusion-spark` crate 4 months ago

enhancement good first issue help wanted EPIC

apache/datafusion-comet

1.2K

Apache DataFusion Comet Spark Accelerator

Scala

#arrow#datafusion#rust#spark

Improve performance of `translate` expression 5 months ago

enhancement good first issue performance

apache/datafusion-comet

1.2K

Apache DataFusion Comet Spark Accelerator

Scala

#arrow#datafusion#rust#spark

Improve performance of `trim` expressions 5 months ago

enhancement good first issue performance

apache/datafusion-comet

1.2K

Apache DataFusion Comet Spark Accelerator

Scala

#arrow#datafusion#rust#spark

[EPIC] Optimize performance for slow expressions 5 months ago

enhancement good first issue performance

apache/datafusion-comet

1.2K

Apache DataFusion Comet Spark Accelerator

Scala

#arrow#datafusion#rust#spark

Improve performance of `repeat` expression 5 months ago

enhancement good first issue performance

apache/datafusion-comet

1.2K

Apache DataFusion Comet Spark Accelerator

Scala

#arrow#datafusion#rust#spark

Improve performance of `reverse` expression 5 months ago

enhancement good first issue performance

apache/datafusion-comet

1.2K

Apache DataFusion Comet Spark Accelerator

Scala

#arrow#datafusion#rust#spark

Improve performance of `startsWith` and `endsWith` expressions 5 months ago

enhancement good first issue performance

apache/datafusion-comet

1.2K

Apache DataFusion Comet Spark Accelerator

Scala

#arrow#datafusion#rust#spark

Cast string to timestamp remaining work 5 months ago

enhancement good first issue help wanted spark 4

apache/datafusion-comet

1.2K

Apache DataFusion Comet Spark Accelerator

Scala

#arrow#datafusion#rust#spark

[EPIC] Fully support ANSI mode 5 months ago

enhancement good first issue help wanted spark 4

apache/datafusion-comet

1.2K

Apache DataFusion Comet Spark Accelerator

Scala

#arrow#datafusion#rust#spark

[Incompatibility] Document arrays_overlap null handling differences 6 months ago

documentation enhancement good first issue array expressions

apache/datafusion-comet

1.2K

Apache DataFusion Comet Spark Accelerator

Scala

#arrow#datafusion#rust#spark

[Incompatibility] Document array_repeat negative count handling 6 months ago

documentation enhancement good first issue array expressions

apache/datafusion-comet

1.2K

Apache DataFusion Comet Spark Accelerator

Scala

#arrow#datafusion#rust#spark

[Incompatibility] Document array_distinct behavior differences: element ordering 7 months ago

documentation enhancement good first issue array expressions

apache/datafusion-comet

1.2K

Apache DataFusion Comet Spark Accelerator

Scala

#arrow#datafusion#rust#spark

Improve performance of cast from string to integral types 7 months ago

enhancement good first issue performance

apache/datafusion-comet

1.2K

Apache DataFusion Comet Spark Accelerator

Scala

#arrow#datafusion#rust#spark

Improve performance of `contains` expression 7 months ago

enhancement good first issue performance

apache/datafusion-comet

1.2K

Apache DataFusion Comet Spark Accelerator

Scala

#arrow#datafusion#rust#spark

Implement Spark-compatible CAST from String to Floating Point 8 months ago

enhancement good first issue help wanted

apache/datafusion-comet

1.2K

Apache DataFusion Comet Spark Accelerator

Scala

#arrow#datafusion#rust#spark

Add support for Spark SQL `explode` expression 8 months ago

enhancement good first issue expression

apache/datafusion-comet

1.2K

Apache DataFusion Comet Spark Accelerator

Scala

#arrow#datafusion#rust#spark

feat: Plan node `CometTakeOrderedAndProject` has invalid missingInput 9 months ago

good first issue

apache/datafusion-comet

1.2K

Apache DataFusion Comet Spark Accelerator

Scala

#arrow#datafusion#rust#spark

[EPIC] Refactor all expression serde logic out of `QueryPlanSerde` 9 months ago

enhancement good first issue EPIC

apache/datafusion-comet

1.2K

Apache DataFusion Comet Spark Accelerator

Scala

#arrow#datafusion#rust#spark

`Project` / `TakeOrderedAndProject` empty fallback reason 9 months ago

bug good first issue

apache/datafusion-comet

1.2K

Apache DataFusion Comet Spark Accelerator

Scala

#arrow#datafusion#rust#spark

Add support for StringSplit 9 months ago

enhancement good first issue expression

apache/datafusion-comet

1.2K

Apache DataFusion Comet Spark Accelerator

Scala

#arrow#datafusion#rust#spark

Add ANSI support in SUM and AVG 10 months ago

enhancement good first issue help wanted

apache/datafusion-comet

1.2K

Apache DataFusion Comet Spark Accelerator

Scala

#arrow#datafusion#rust#spark

[EPIC] Support try_* functions which always return NULL instead of runtime errors 10 months ago

enhancement help wanted

apache/datafusion-comet

1.2K

Apache DataFusion Comet Spark Accelerator

Scala

#arrow#datafusion#rust#spark

feat: Support ANSI mode for round 10 months ago

enhancement good first issue help wanted

apache/datafusion-comet

1.2K

Apache DataFusion Comet Spark Accelerator

Scala

#arrow#datafusion#rust#spark

Add ANSI support for Divide and IntegralDivide 10 months ago

enhancement good first issue help wanted

apache/datafusion-comet

1.2K

Apache DataFusion Comet Spark Accelerator

Scala

#arrow#datafusion#rust#spark

Add ANSI support for Add 10 months ago

enhancement good first issue help wanted

apache/datafusion-comet

1.2K

Apache DataFusion Comet Spark Accelerator

Scala

#arrow#datafusion#rust#spark

Add ANSI support for Subtract 10 months ago

enhancement good first issue help wanted

apache/datafusion-comet

1.2K

Apache DataFusion Comet Spark Accelerator

Scala

#arrow#datafusion#rust#spark

Add ANSI support for Multiply 10 months ago

enhancement good first issue help wanted

apache/datafusion-comet

1.2K

Apache DataFusion Comet Spark Accelerator

Scala

#arrow#datafusion#rust#spark

Make DiskManager `max_temp_directory_size` configurable 10 months ago

enhancement good first issue

apache/datafusion-comet

1.2K

Apache DataFusion Comet Spark Accelerator

Scala

#arrow#datafusion#rust#spark

Review experiemental status of array functions 12 months ago

enhancement good first issue

apache/datafusion-comet

1.2K

Apache DataFusion Comet Spark Accelerator

Scala

#arrow#datafusion#rust#spark

Improve "unsupported Spark partitioning" fallback message 12 months ago

AI Summary: This GitHub issue requests an improvement to the "unsupported Spark partitioning" log message. Currently, the message doesn't specify *why* a particular partitioning is unsupported, as there are multiple underlying reasons. The proposed solution is to refactor the code to perform individual checks and provide a more detailed explanation in the log message for the specific reason of the unsupported partitioning.

Complexity: 1/5

enhancement good first issue

apache/datafusion-comet

1.2K

Apache DataFusion Comet Spark Accelerator

Scala

#arrow#datafusion#rust#spark

rpad expression panics if length input is not a literal value 12 months ago

AI Summary: The task is to fix a bug in Apache DataFusion Comet where the `rpad` expression panics if the length argument is not a literal value. The solution involves handling non-literal length inputs gracefully, potentially by falling back to Spark's implementation instead of panicking.

Complexity: 4/5

bug good first issue

apache/datafusion-comet

1.2K

Apache DataFusion Comet Spark Accelerator

Scala

#arrow#datafusion#rust#spark

doc: Document supported `Map` methods in `expressions.md` about 1 year ago

AI Summary: Update the Apache DataFusion Comet documentation (expressions.md) to include a list of supported Map methods, addressing a follow-up task from issue #2059 and epic #1044. This involves researching the supported methods and clearly documenting them in the specified markdown file.

Complexity: 2/5

documentation good first issue

apache/datafusion-comet

1.2K

Apache DataFusion Comet Spark Accelerator

Scala

#arrow#datafusion#rust#spark

Upgrade to DataFusion 49.0.0 about 1 year ago

AI Summary: The task involves incrementally upgrading the Apache DataFusion Comet project to use the latest DataFusion codebase (version 49.0.0). This will require updating the project's dependencies, resolving any resulting compilation errors, and repeating this process until the official release is available.

Complexity: 4/5

enhancement good first issue

apache/datafusion-comet

1.2K

Apache DataFusion Comet Spark Accelerator

Scala

#arrow#datafusion#rust#spark

Open Issues Need Help