[Roadmap] GSOC2026 Rust code quality: QDP pipeline

[0lai0]

GSoC 2026 — QDP Pipeline Cleanup

Scope: qdp-core, qdp-python, qdp-kernels

Pipeline today: read data (synthetic / file / streaming Parquet) → CPU prefetch → GPU encode_batch → yield QuantumTensor to Python (QuantumDataLoader)

Prerequisite: #1310 (single-sample f32 on QuantumEncoder trait) — confirm merged before starting

---

Track A — Parquet / reader f32

Make file/Parquet sources honor dtype=f32 end-to-end (no forced f64 widening)

ID	Issue	What to do
A1	[Refactor] Add FloatElem trait and DataReader #1339	Generalize DataReader::read_batch from Vec<f64> to Vec<T> in reader.rs
A2	[Feature] ParquetReader: f32/f64 columns + Arrow cast #1340	Accept f32 Parquet columns; Arrow cast when dtypes differ
A3	[Feature] Pipeline file load respects PipelineConfig.dtype #1341	Wire A2 into pipeline_runner so dtype=f32 reaches f32 kernels
A4	[Testing] Parquet f32 tests and benchmark #1342	Fidelity tests + optional throughput benchmark

• [Refactor] Add FloatElem trait and DataReader<T> #1339
• [Feature] ParquetReader: f32/f64 columns + Arrow cast #1340
• [Feature] Pipeline file load respects PipelineConfig.dtype #1341
• [Testing] Parquet f32 tests and benchmark #1342

---

Track B — GPU memory estimate

Early OOM detection: fail at loader build, not after 90 min of running

ID	Issue	What to do
B1	[Feature] Add estimate_memory() for pipeline GPU footprint #1429	Pure function estimating CPU + GPU memory from config params
B2	[Feature] Fail fast when estimated memory exceeds VRAM #1430	Call B1 at loader construction; return actionable error

• [Feature] Add estimate_memory() for pipeline GPU footprint #1429
• [Feature] Fail fast when estimated memory exceeds VRAM #1430

---

Track C — Preprocessor Python API

Expose Preprocessor helpers that already exist in Rust to Python users

ID	Issue	What to do
C1	[Feature] PyO3 bindings for Preprocessor #1431	Bind validate_input, calculate_l2_norm, validate_batch, calculate_batch_l2_norms via PyO3
C2	[Docs] Document Preprocessor in qumat_qdp public API #1432	Export from __init__.py, add docstrings and usage example

• [Feature] PyO3 bindings for Preprocessor #1431
• [Docs] Document Preprocessor in qumat_qdp public API #1432

---

Track D — Kernel dedup

Collapse ~270 duplicate f32/f64 stubs in qdp-kernels

ID	Issue	What to do
D1	[Refactor] KernelElem trait to deduplicate f32/f64 stubs #1433	Add KernelElem trait + define_stub! macro to replace copy-paste stubs

• [Refactor] KernelElem trait to deduplicate f32/f64 stubs #1433

---

Track E — pipeline_runner structure

Unify three PipelineIterator constructors and three producers in pipeline_runner.rs

ID	Issue	What to do
E1	[Refactor] Unify PipelineIterator constructors with Source enum #1434	3 new_* methods → 1 PipelineIterator::new(source, config)
E2	[Refactor] Unify batch producers via AdapterBackedProducer #1435	3 producers with 80% identical code → AdapterBackedProducer
E3	[Refactor] StreamingProducer: use VecDeque for O(1) buffer advance #1436	Replace Vec::drain O(n) with VecDeque

• [Refactor] Unify PipelineIterator constructors with Source enum #1434
• [Refactor] Unify batch producers via AdapterBackedProducer #1435
• [Refactor] StreamingProducer: use VecDeque for O(1) buffer advance #1436

---

Others

• [Bug] List<T> reader fails with "Inconsistent sample sizes" on nullable outer rows #1401

---

Done when

• QuantumDataLoader(...).dtype("f32").source_file("x.parquet") uses f32 kernels end-to-end
• Loader build fails early with clear message when VRAM insufficient (partial [Feature] Centralize MAX_QUBITS validation across QDP encoding paths #1262)
• Preprocessor exposed and documented in Python
• Producers / constructors deduplicated; CI green; ≤2% throughput regression

[0lai0]

cc @rich7420 , PTAL thanks. : )

[rich7420]

thanks for ping me!