Show decimal add bug

[kamilk-pal]

Which issue does this PR close?

No linked issue; this PR is a failing regression reproducer.

Rationale for this change

Adding 12345678901234567890123456789012 as DECIMAL(32, 0) to 1.00000000000000000000 as DECIMAL(38, 20) overflows in DataFusion. The inferred result keeps scale 20, so aligning the integer's scale multiplies its underlying value by 10^20 and exceeds Decimal128's capacity.

The expected type and sums in this test follow Spark with spark.sql.decimalOperations.allowPrecisionLoss=true: the result is decimal(38,6) with value 12345678901234567890123456789013.000000.

PostgreSQL uses arbitrary-precision numeric arithmetic and preserves scale 20, returning 12345678901234567890123456789013.00000000000000000000. The expected (38,6) type therefore describes Spark's scale-reduction policy rather than a type shared by all three engines.

What changes are included in this PR?

Regression cases in datafusion/sqllogictest/test_files/decimal.slt check the inferred result type and addition with both literal and column inputs. Comments identify the Spark configuration behind the expected output and PostgreSQL's different representation.

The assertions intentionally expect the desired successful result and are expected to fail until the DataFusion behavior is changed.

What is the testing strategy for this PR?

Run the DataFusion SLT from this PR's checkout, then compare the same operands on PostgreSQL and Spark using the commands below. The PostgreSQL and Spark examples exercise both literals and typed columns.

Observed results for both literal and column inputs:

Engine	Result type	Result
DataFusion 54.0.0	Decimal128(38, 20)	Arithmetic overflow while rescaling the integer
PostgreSQL 17.11	numeric, scale 20	12345678901234567890123456789013.00000000000000000000
Spark 4.2.0, allowPrecisionLoss=true	decimal(38,6)	12345678901234567890123456789013.000000

The DataFusion result above was measured with the released Python binding, not a build of this PR's exact revision. The pinned Rust toolchain was unavailable in the validation workspace. The command below runs the actual SLT against the source checkout when that toolchain is installed.

DataFusion

From the repository root, with the repository's Rust toolchain and test dependencies installed:

cargo test --profile ci --test sqllogictests -- decimal.slt

This executes DataFusion itself. The new type assertion expects Decimal128(38, 6) and is intentionally expected to fail while the engine infers Decimal128(38, 20); the runner can stop at that first failure.

PostgreSQL

Prerequisite: Docker. This starts a temporary database without publishing a host port, runs both forms of the query, and removes the container on exit:

(
set -eu
docker run --detach --rm --name df-decimal-pg \
  -e POSTGRES_HOST_AUTH_METHOD=trust postgres:17-alpine
trap 'docker stop df-decimal-pg >/dev/null' EXIT

until docker exec df-decimal-pg pg_isready -h 127.0.0.1 -U postgres >/dev/null 2>&1; do
  sleep 1
done

docker exec -i df-decimal-pg psql -h 127.0.0.1 -X -v ON_ERROR_STOP=1 -U postgres <<'SQL'
SELECT version();

-- Literal inputs.
WITH calculated AS (
  SELECT CAST('12345678901234567890123456789012' AS DECIMAL(32, 0))
       + CAST('000000000000000001.00000000000000000000' AS DECIMAL(38, 20)) AS result
)
SELECT pg_typeof(result) AS result_type, scale(result) AS result_scale, result
FROM calculated;

-- Column inputs.
CREATE TEMP TABLE decimal_add_wide(lhs DECIMAL(32, 0), rhs DECIMAL(38, 20));
INSERT INTO decimal_add_wide VALUES
  ('12345678901234567890123456789012', '000000000000000001.00000000000000000000');
SELECT pg_typeof(lhs + rhs) AS result_type,
       scale(lhs + rhs) AS result_scale,
       lhs + rhs AS result
FROM decimal_add_wide;
SQL
)

Both queries should report numeric, scale 20, and 12345678901234567890123456789013.00000000000000000000.

PG_COMPAT=true does not run this particular decimal.slt file: the PostgreSQL compatibility runner selects pg_compat_* files. Also, arrow_typeof is specific to DataFusion, so this example uses PostgreSQL's pg_typeof and scale functions.

Spark

Prerequisites: Java 17 or newer and uv. This provisions Python 3.12 and PySpark 4.2.0, then runs a local Apache Spark session:

SPARK_LOCAL_IP=127.0.0.1 uv run --no-project --python 3.12 \
  --with pyspark==4.2.0 python - <<'PY'
from pathlib import Path
from tempfile import TemporaryDirectory
from pyspark.sql import SparkSession

lhs = "12345678901234567890123456789012"
rhs = "000000000000000001.00000000000000000000"
expression = f"CAST('{lhs}' AS DECIMAL(32, 0)) + CAST('{rhs}' AS DECIMAL(38, 20))"

with TemporaryDirectory() as directory:
    spark = (SparkSession.builder.master("local[1]")
             .appName("decimal-add-regression")
             .config("spark.ui.enabled", "false")
             .config("spark.driver.bindAddress", "127.0.0.1")
             .config("spark.sql.warehouse.dir", str(Path(directory) / "warehouse"))
             .config("spark.sql.decimalOperations.allowPrecisionLoss", "true")
             .config("spark.sql.ansi.enabled", "true")
             .getOrCreate())
    try:
        print("Spark version:", spark.version)

        # Literal inputs.
        spark.sql(f"SELECT typeof({expression}) AS result_type, {expression} AS result").show(truncate=False)

        # Typed file-backed columns exercise execution beyond literal folding.
        csv = Path(directory) / "inputs.csv"
        csv.write_text(f"{lhs},{rhs}\n")
        inputs = (spark.read.schema("lhs DECIMAL(32, 0), rhs DECIMAL(38, 20)")
                  .option("mode", "FAILFAST").csv(str(csv)))
        inputs.createOrReplaceTempView("decimal_add_wide")
        spark.sql("SELECT typeof(lhs + rhs) AS result_type, lhs + rhs AS result FROM decimal_add_wide").show(truncate=False)
    finally:
        spark.stop()
PY

Both queries should report decimal(38,6) and 12345678901234567890123456789013.000000.

The precision-loss setting matters: changing spark.sql.decimalOperations.allowPrecisionLoss to false gives decimal(38,20) and an overflow with ANSI mode enabled. With ANSI mode disabled, the result is NULL. The ordinary DataFusion SLT runner, including its Spark-compatibility suite, does not run Apache Spark; the PySpark command above does.

Are there any user-facing changes?

No implementation change. This PR documents and reproduces the decimal-addition behavior with an intentionally failing test.