Skip to content

feat(benchmark): compare supplied Expressif versions - #657

Open
Seddryck wants to merge 5 commits into
mainfrom
feat/benchmark-version-comparison
Open

feat(benchmark): compare supplied Expressif versions#657
Seddryck wants to merge 5 commits into
mainfrom
feat/benchmark-version-comparison

Conversation

@Seddryck

@Seddryck Seddryck commented Aug 26, 2026

Copy link
Copy Markdown
Owner

Summary

  • add a Windows-only .NET 10 BenchmarkDotNet solution isolated under benchmark/
  • discover manually supplied v1/v2 binary folders and load each version in its own assembly load context
  • compare parse-and-bind plus simple, text, and coercion evaluation workloads with v1 as the ratio baseline
  • validate supplied binaries with isolated parsing, binding, evaluation, and coercion adapter tests

Validation

  • dotnet build benchmark/Expressif.Benchmark.sln -c Release
  • dotnet build Expressif.sln -c Release
  • dotnet test Expressif.sln -c Release --no-build
  • dotnet test benchmark/Expressif.Benchmark.sln -c Release — 3 passed with staged v1.51.0 and next-major binaries
  • all eight benchmark combinations completed in a short smoke run using replace-chars(" ", "-") | first-chars(10)

Close #600

@coderabbitai

coderabbitai Bot commented Aug 26, 2026

Copy link
Copy Markdown
Contributor

Important

  • 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 7fd8cd08-8c87-4643-aed0-3012b96e4b13


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@Seddryck

Copy link
Copy Markdown
Owner Author

Added an isolated benchmark test project in commit 5fb4f52. Validation against the manually staged binaries now confirms: (1) the prescribed complex expression parses successfully on NuGet v1.51.0 and fails specifically during construction/binding on the unavailable
eplace\ function; and (2) the implicit-coercion pipeline returns \€1493.82\ as a string on both v1.51.0 and latest next-major. \dotnet test benchmark/Expressif.Benchmark.sln -c Release: 3 passed.

@Seddryck

Copy link
Copy Markdown
Owner Author

Added parse-only benchmarks for both workload sources. With 3 warmups and 5 measurement iterations: complex text parsing is 32.21 µs / 191.3 KB on v1.51.0 versus 62.46 µs / 8.05 KB on next-major (1.94x time, 0.04x allocation); coercion-source parsing is 30.30 µs / 177.7 KB versus 58.74 µs / 7.6 KB (1.94x time, 0.04x allocation). Benchmark adapter tests now validate that both sources parse on both API generations (5 tests passed).

@Seddryck

Copy link
Copy Markdown
Owner Author

Added native parse+bind benchmarks for both sources (commit 0a84190). Each measurement constructs the expression from source through that version's public API, so it includes parsing and binding together. Results (5 iterations, 3 warmups): complex pipeline — v1 53.14 ms / 2.37 MB, v2 60.78 ms / 2.65 MB (1.14x time, 1.12x allocation); implicit-coercion pipeline — v1 53.30 ms / 2.36 MB, v2 60.99 ms / 2.65 MB (1.14x time, 1.13x allocation). Adapter tests now verify both expressions parse and parse+bind successfully on v1 and v2; 5 tests pass.

@sonarqubecloud

Copy link
Copy Markdown

Quality Gate Failed Quality Gate failed

Failed conditions
C Reliability Rating on New Code (required ≥ A)

See analysis details on SonarQube Cloud

Catch issues before they fail your Quality Gate with our IDE extension SonarQube for IDE

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add benchmarks comparing main and next-major performance

1 participant