Adding Gluten (on Velox) to ClickBench

[Iskander14yo]

Hi!

I recently opened a ClickHouse/ClickBench#575 to add Gluten (on Velox) to ClickBench - one of the popular benchmarks for analytical workloads.

I thought it might be useful to let Velox/Gluten community know about this PR. And while results yet to be measured I'd also appreciate any feedback on whether my configuration and setup are correct.

My notes (besides those in README):

• I couldn’t get Gluten to run with pyspark==3.5.5 as the docs recommend; I hit warnings/errors (referenced in the README).

• I didn’t find a reliable, single grep-able pattern to detect operators/functions where Gluten falls back to Spark because warnings vary significantly. For example, below are logs for a last query:

25/08/17 00:42:50 WARN ProjectExecTransformer: 
 - Native validation failed: 
   |- Validation failed due to exception caught at file:SubstraitToVeloxPlanValidator.cc line:1387 function:validate, thrown from file:ExprCompiler.cpp line:472 function:compileRewrittenExpression, reason:Scalar function name not registered: date_trunc, called with arguments: (VARCHAR, TIMESTAMP, VARCHAR).
25/08/17 00:42:50 WARN ProjectExecTransformer: 
 - Native validation failed: 
   |- Validation failed due to exception caught at file:SubstraitToVeloxPlanValidator.cc line:1387 function:validate, thrown from file:ExprCompiler.cpp line:472 function:compileRewrittenExpression, reason:Scalar function name not registered: date_trunc, called with arguments: (VARCHAR, TIMESTAMP, VARCHAR).
25/08/17 00:42:50 WARN GlutenFallbackReporter: Validation failed for plan: Project[QueryId=2], due to: 
 - Native validation failed: 
   |- Validation failed due to exception caught at file:SubstraitToVeloxPlanValidator.cc line:1387 function:validate, thrown from file:ExprCompiler.cpp line:472 function:compileRewrittenExpression, reason:Scalar function name not registered: date_trunc, called with arguments: (VARCHAR, TIMESTAMP, VARCHAR).
25/08/17 00:42:52 WARN ProjectExecTransformer:                                  
 - Validation failed with exception from: ProjectExecTransformer, reason: Not supported to map spark function name to substrait function name: toprettystring(M#433, Some(Europe/Moscow)), class name: ToPrettyString.
25/08/17 00:42:52 WARN ColumnarCollectLimitExec: 
 - Columnar collect-limit is unsupported under the current Spark version
25/08/17 00:42:52 WARN ProjectExecTransformer: 
 - Validation failed with exception from: ProjectExecTransformer, reason: Not supported to map spark function name to substrait function name: toprettystring(M#433, Some(Europe/Moscow)), class name: ToPrettyString.
25/08/17 00:42:52 WARN GlutenFallbackReporter: Validation failed for plan: Project[QueryId=2], due to: 
 - Validation failed with exception from: ProjectExecTransformer, reason: Not supported to map spark function name to substrait function name: toprettystring(M#433, Some(Europe/Moscow)), class name: ToPrettyString.
25/08/17 00:42:52 WARN GlutenFallbackReporter: Validation failed for plan: CollectLimit[QueryId=2], due to: 
 - Columnar collect-limit is unsupported under the current Spark version
E20250817 00:42:52.213160 134615 Exceptions.h:66] Line: /root/src/apache/incubator-gluten/ep/build-velox/build/velox_ep/velox/exec/Task.cpp:2049, Function:terminate, Expression:  Cancelled, Source: RUNTIME, ErrorCode: INVALID_STATE

[philo-he]

@Iskander14yo, toprettystring is not supported by Gluten Velox backend, whereas date_trunc is supported. It's strange why three argumentss are being passed to date_trunc. Though the corresponding scala class has a three-argument constructor, at the SQL level, I think only two arguments are allowed.

[zhztheplayer]

I once looked at the ClickBench workload and yes Gluten had some issues working with that dataset.

ClickBench doesn't check the correctness of query results so we need to be cautious on the result we get now :)

That said, the work so far is much appreciated. And we'll look to improve the results in the future.