Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 13 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -613,9 +613,11 @@ suite.add('Using includes', () => {
});
```

Here, `%DoNotOptimize` is being called inside the loop for regular benchmarks (assuming V8NeverOptimizePlugin is being used),
ensuring that the operation is not overly optimized within each loop iteration.
This prevents V8 from optimizing away the operation (e.g., skipping certain steps because the result is not used or the function is too trivial).
Here, `DoNotOptimize` is called inside the generated loop for regular benchmarks
(assuming `V8NeverOptimizePlugin` is being used), and the benchmark function is
also marked with `%NeverOptimizeFunction`.
Together, these prevent V8 from optimizing the benchmark function and from
treating the returned value as irrelevant.

Managed benchmarks explicitly handle timing through `start()` and `end()` calls around the benchmarked code.
This encapsulates the entire set of iterations in one timed block,
Expand All @@ -637,9 +639,10 @@ suite.add('[Managed] Using includes', (timer) => {
});
```

In this case, `%DoNotOptimize` is being applied outside the loop, so it does not protect each iteration from
excessive optimization. This can result in higher operation counts because V8 might optimize away repetitive tasks.
That's why an `assert.ok(r)` has been used. To avoid V8 optimizing the entire block as the `r` var was not being used.
In this case, `DoNotOptimize` is applied to the benchmark function's return value,
outside the user-managed loop. It does not consume the local `r` value from each
iteration. That's why `assert.ok(r)` has been used: it makes the per-iteration
result observable inside the timed block.

> [!NOTE]
> V8 assumptions can change any time soon. Therefore, it's crucial to investigate
Expand Down Expand Up @@ -889,9 +892,10 @@ and hidden class transformations can all distort your measurements, leading to o
claims about performance improvements that might never materialize in production.

That’s why **bench-node** was created—to provide a stable and consistent way to compare
small snippets of code. By default, it tells V8 to never optimize your code with a
snippet like `%NeverOptimizeFunction(DoNotOptimize)`, ensuring the JIT compiler doesn’t
remove dead code. However, even this approach can’t fully replicate real-world scenarios
small snippets of code. By default, it tells V8 to never optimize the benchmark
function with `%NeverOptimizeFunction(bench.fn)` and consumes the returned value
with a non-optimized `DoNotOptimize` helper so the JIT compiler doesn’t remove
dead code. However, even this approach can’t fully replicate real-world scenarios
in which V8 optimizations and unpredictable workloads impact performance. Think of
bench-node as a helpful tool for quick comparisons rather than a guarantee of what you’ll
see in production.
Expand Down
18 changes: 13 additions & 5 deletions doc/Plugins.md
Original file line number Diff line number Diff line change
Expand Up @@ -60,20 +60,28 @@ The wrapped function provides a powerful way to manipulate how the benchmark
is run without directly modifying the benchmark logic.

```js
beforeClockTemplate() {
beforeClockTemplate({ bench }) {
let code = '';

code += `
function DoNotOptimize(x) {}
// Prevent DoNotOptimize from optimizing or being inlined.
// Prevent the benchmark function and result consumer from optimizing or being inlined.
%NeverOptimizeFunction(${bench}.fn);
%NeverOptimizeFunction(DoNotOptimize);
`
return [code, 'DoNotOptimize'];
}
```

In this example, the plugin injects the `DoNotOptimize` function and also
provides it as a wrapper for the benchmark function.
provides it as a wrapper for the benchmark function result. The benchmark
function itself is marked with `%NeverOptimizeFunction(${bench}.fn)`, while the
`DoNotOptimize` wrapper consumes the returned value so the benchmark expression
does not become observationally irrelevant.

These two protections address different parts of the generated code:
`%NeverOptimizeFunction(${bench}.fn)` targets the function under test, and
`DoNotOptimize(bench.fn())` targets the value returned by each call.

### `afterClockTemplate(varNames)`

Expand Down Expand Up @@ -160,8 +168,8 @@ benchmarks.
### Class: `V8NeverOptimizePlugin`

The `V8NeverOptimizePlugin` prevents the V8 engine from optimizing or inlining
a function, useful when you want to benchmark functions without any
optimization.
the benchmark function. It also wraps the benchmark result in a non-optimized
`DoNotOptimize` helper so V8 cannot treat an unused return value as irrelevant.

### Class: `V8GetOptimizationStatus`

Expand Down
5 changes: 3 additions & 2 deletions lib/plugins/v8-never-opt.js
Original file line number Diff line number Diff line change
Expand Up @@ -9,12 +9,13 @@ class V8NeverOptimizePlugin {
}
}

beforeClockTemplate() {
beforeClockTemplate({ bench }) {
let code = "";

code += `
function DoNotOptimize(x) {}
// Prevent DoNotOptimize from optimizing or being inlined.
// Prevent the benchmark function and result consumer from optimizing or being inlined.
%NeverOptimizeFunction(${bench}.fn);
%NeverOptimizeFunction(DoNotOptimize);
`;
return [code, "DoNotOptimize"];
Expand Down
15 changes: 15 additions & 0 deletions test/plugins.js
Original file line number Diff line number Diff line change
Expand Up @@ -101,6 +101,21 @@ describe("Official plugins validation", () => {
});
assert.ok(bench);
});

it("V8NeverOptimizePlugin prevents benchmark function optimization", () => {
const plugin = new V8NeverOptimizePlugin();
const [code, wrapper] = plugin.beforeClockTemplate({
bench: "bench",
awaitOrEmpty: "",
context: "context",
timer: "timer",
});

assert.strictEqual(wrapper, "DoNotOptimize");
assert.match(code, /%NeverOptimizeFunction\(bench\.fn\);/);
assert.match(code, /%NeverOptimizeFunction\(DoNotOptimize\);/);
});

it("V8GetOptimizationStatus validation", () => {
const bench = new Suite({
plugins: [new V8GetOptimizationStatus()],
Expand Down
Loading