Add benchmark harness and tensor GEMM benchmark#2
Open
mmshad wants to merge 1 commit into
Open
Conversation
- Add shared infrastructure (common/config.sh, detect_arch.sh, parse_results.py) for centralized SLURM/container configuration, GPU arch detection with precision gating, and unified result parsing across all benchmark types. - Add parameterized launchers (run_*.sh) for HPL, HPL-MxP, HPCG, and STREAM that accept env-var configuration and auto-resubmit with proper SLURM flags. - Add tensor-core GEMM saturation benchmark (p5_tensor_gemm/) with cuBLASLt-based FP16/FP8 microbenchmark, SLURM launcher with smoke/saturation/transformer/custom modes, and example scripts. - Update READMEs with benchmark taxonomy, common infrastructure docs, p5 results, and fix GPU count errors in p1/p2 multi-node descriptions.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Transforms the repo from a collection of one-off SLURM scripts into a parameterized, architecture-aware benchmarking harness and adds a new tensor-core saturation benchmark (FP16/FP8).
Changes
Shared Infrastructure (
common/)config.shdetect_arch.shnvidia-smicompute capability. Gates precision support (FP8 on Hopper, FP4 blocked until Blackwell).parse_results.py.outfiles and produces Markdown summary tables for all 5 benchmark types.Parameterized Launchers
Each benchmark (
p1-p5) now has a singlerun_*.shlauncher that:NODES,GPUS_PER_NODE,N,SLOPPY_TYPE, etc.)--partition,--account,--constraintviaexec sbatch --export=ALLif not already inside a SLURM jobBenchmark-specific features:
HPL.datinline; supports sweep mode (HPL_SWEEP="92160:1:1 190464:2:2")SLOPPY_TYPE=FP8) to numeric flags; auto-generates GPU affinity; gates FP4 by arch + container versionNew: Tensor-Core GEMM Saturation Benchmark (
p5_tensor_gemm/)A cuBLASLt-based microbenchmark that measures raw FP16 and FP8 tensor-core throughput via dense GEMM. This answers: "How close can my GPU get to its theoretical peak tensor-core TFLOPS?"
tensor_gemm_bench.cu-- ~440 lines, supports FP16 and FP8 (E4M3) on Hopper, FP4 gated to Blackwellrun_tensor_gemm.sh-- SLURM launcher withsmoke,saturation,transformer,custommodesexamples/Measured results (single GPU, 1980 MHz):
README Updates
What's NOT Changed
runs/directories are untouchedTesting
All benchmarks verified on H200 node (holygpu8a10302, 4x NVIDIA H200 SXM, 1980 MHz):
common/config.sh-- config loads, container path resolvescommon/detect_arch.sh-- detects SM 90 / Hopperp1HPL -- N=8192, residual PASSEDp2HPL-MxP FP16 -- N=8192, residual PASSEDp3HPCG -- VALID resultp4STREAM -- FP32 and FP64 bandwidth measuredp5Tensor GEMM -- FP16 and FP8 saturation sweeps