diff --git a/README.md b/README.md index 2e0449c..79e102d 100644 --- a/README.md +++ b/README.md @@ -613,9 +613,11 @@ suite.add('Using includes', () => { }); ``` -Here, `%DoNotOptimize` is being called inside the loop for regular benchmarks (assuming V8NeverOptimizePlugin is being used), -ensuring that the operation is not overly optimized within each loop iteration. -This prevents V8 from optimizing away the operation (e.g., skipping certain steps because the result is not used or the function is too trivial). +Here, `DoNotOptimize` is called inside the generated loop for regular benchmarks +(assuming `V8NeverOptimizePlugin` is being used), and the benchmark function is +also marked with `%NeverOptimizeFunction`. +Together, these prevent V8 from optimizing the benchmark function and from +treating the returned value as irrelevant. Managed benchmarks explicitly handle timing through `start()` and `end()` calls around the benchmarked code. This encapsulates the entire set of iterations in one timed block, @@ -637,9 +639,10 @@ suite.add('[Managed] Using includes', (timer) => { }); ``` -In this case, `%DoNotOptimize` is being applied outside the loop, so it does not protect each iteration from -excessive optimization. This can result in higher operation counts because V8 might optimize away repetitive tasks. -That's why an `assert.ok(r)` has been used. To avoid V8 optimizing the entire block as the `r` var was not being used. +In this case, `DoNotOptimize` is applied to the benchmark function's return value, +outside the user-managed loop. It does not consume the local `r` value from each +iteration. That's why `assert.ok(r)` has been used: it makes the per-iteration +result observable inside the timed block. > [!NOTE] > V8 assumptions can change any time soon. Therefore, it's crucial to investigate @@ -889,9 +892,10 @@ and hidden class transformations can all distort your measurements, leading to o claims about performance improvements that might never materialize in production. That’s why **bench-node** was created—to provide a stable and consistent way to compare -small snippets of code. By default, it tells V8 to never optimize your code with a -snippet like `%NeverOptimizeFunction(DoNotOptimize)`, ensuring the JIT compiler doesn’t -remove dead code. However, even this approach can’t fully replicate real-world scenarios +small snippets of code. By default, it tells V8 to never optimize the benchmark +function with `%NeverOptimizeFunction(bench.fn)` and consumes the returned value +with a non-optimized `DoNotOptimize` helper so the JIT compiler doesn’t remove +dead code. However, even this approach can’t fully replicate real-world scenarios in which V8 optimizations and unpredictable workloads impact performance. Think of bench-node as a helpful tool for quick comparisons rather than a guarantee of what you’ll see in production. diff --git a/doc/Plugins.md b/doc/Plugins.md index 4a9c63c..6900e3b 100644 --- a/doc/Plugins.md +++ b/doc/Plugins.md @@ -60,12 +60,13 @@ The wrapped function provides a powerful way to manipulate how the benchmark is run without directly modifying the benchmark logic. ```js -beforeClockTemplate() { +beforeClockTemplate({ bench }) { let code = ''; code += ` function DoNotOptimize(x) {} -// Prevent DoNotOptimize from optimizing or being inlined. +// Prevent the benchmark function and result consumer from optimizing or being inlined. +%NeverOptimizeFunction(${bench}.fn); %NeverOptimizeFunction(DoNotOptimize); ` return [code, 'DoNotOptimize']; @@ -73,7 +74,14 @@ function DoNotOptimize(x) {} ``` In this example, the plugin injects the `DoNotOptimize` function and also -provides it as a wrapper for the benchmark function. +provides it as a wrapper for the benchmark function result. The benchmark +function itself is marked with `%NeverOptimizeFunction(${bench}.fn)`, while the +`DoNotOptimize` wrapper consumes the returned value so the benchmark expression +does not become observationally irrelevant. + +These two protections address different parts of the generated code: +`%NeverOptimizeFunction(${bench}.fn)` targets the function under test, and +`DoNotOptimize(bench.fn())` targets the value returned by each call. ### `afterClockTemplate(varNames)` @@ -160,8 +168,8 @@ benchmarks. ### Class: `V8NeverOptimizePlugin` The `V8NeverOptimizePlugin` prevents the V8 engine from optimizing or inlining -a function, useful when you want to benchmark functions without any -optimization. +the benchmark function. It also wraps the benchmark result in a non-optimized +`DoNotOptimize` helper so V8 cannot treat an unused return value as irrelevant. ### Class: `V8GetOptimizationStatus` diff --git a/lib/plugins/v8-never-opt.js b/lib/plugins/v8-never-opt.js index 216a060..489960b 100644 --- a/lib/plugins/v8-never-opt.js +++ b/lib/plugins/v8-never-opt.js @@ -9,12 +9,13 @@ class V8NeverOptimizePlugin { } } - beforeClockTemplate() { + beforeClockTemplate({ bench }) { let code = ""; code += ` function DoNotOptimize(x) {} -// Prevent DoNotOptimize from optimizing or being inlined. +// Prevent the benchmark function and result consumer from optimizing or being inlined. +%NeverOptimizeFunction(${bench}.fn); %NeverOptimizeFunction(DoNotOptimize); `; return [code, "DoNotOptimize"]; diff --git a/test/plugins.js b/test/plugins.js index c953ed9..f2f9211 100644 --- a/test/plugins.js +++ b/test/plugins.js @@ -101,6 +101,21 @@ describe("Official plugins validation", () => { }); assert.ok(bench); }); + + it("V8NeverOptimizePlugin prevents benchmark function optimization", () => { + const plugin = new V8NeverOptimizePlugin(); + const [code, wrapper] = plugin.beforeClockTemplate({ + bench: "bench", + awaitOrEmpty: "", + context: "context", + timer: "timer", + }); + + assert.strictEqual(wrapper, "DoNotOptimize"); + assert.match(code, /%NeverOptimizeFunction\(bench\.fn\);/); + assert.match(code, /%NeverOptimizeFunction\(DoNotOptimize\);/); + }); + it("V8GetOptimizationStatus validation", () => { const bench = new Suite({ plugins: [new V8GetOptimizationStatus()],