Skip to content

cnid: sqlite-default residual cleanup; dsi: raise the listen backlog to 128 - #3267

Merged
andylemin merged 1 commit into
mainfrom
perf-cnid-residuals-and-backlog
Aug 26, 2026
Merged

andylemin merged 1 commit into
mainfrom
perf-cnid-residuals-and-backlog

Conversation

@andylemin

@andylemin andylemin commented Aug 24, 2026

Copy link
Copy Markdown
Contributor

CNID: sqlite-default residual cleanup

SQLite has been the default CNID scheme since PR #2265 ("Build and run netatalk without Berkeley DB") made meson select sqlite > mysql > dbd among the compiled backends — and all three compile by default, so DEFAULT_CNID_SCHEME, the runtime cnid scheme default, is already "sqlite" on any default build (afpd -v: "Default CNID backend: sqlite"). What remained was residue that still said dbd:

residue fix
docker entrypoint fallback: cnid scheme = ${AFP_CNID_BACKEND:-dbd} (two sites) :-sqlite — an unset AFP_CNID_BACKEND now matches what a stock source build already does
~30 CI test jobs (the ea:sys/ea:ad/UAM/AFP-version/login matrix) set no AFP_CNID_BACKEND and were unintentionally testing dbd via the entrypoint fallback they now inherit sqlite, the shipped default — only jobs that intentionally test a backend name it. The "AFP spec test (cnid:dbd)" job, which also relied on the fallback, now sets -e AFP_CNID_BACKEND=dbd explicitly
man page never stated which backend is the default the sqlite paragraph now says "This is the default backend when compiled in."
NEWS never recorded the default change entry added

Intentional backend coverage is unchanged: the cnid:mysql legs (Alpine + Debian) and the cnid:sqlite/perf legs set AFP_CNID_BACKEND explicitly, the VM spectest script exports dbd explicitly, and the cnid:dbd job is now pinned. Every other job had no business running dbd; this PR resolves that unintended coverage skew — the broad matrix now tests what ships.

DSI: raise the listen backlog to 128

DSI_TCPMAXPEND has been 20 since the CVS-era initial revision, with no config knob. A reconnect storm (default 200 max connections) overflows a backlog of 20: dropped SYNs and ~1s client retransmit delays. 128 matches the common SOMAXCONN floor, and the kernel silently caps the effective backlog at net.core.somaxconn, so no new option is warranted. No steady-state effect; the #ifndef guard stays for build-time overrides.

Testing

  • meson test (Alpine container, CI build flags): 3/3 suites, zero failures.
  • AFP spectest ea=sys and ea=ad legs run without AFP_CNID_BACKEND — i.e. on the new sqlite fallback end-to-end: both exit 0, zero failures.
  • Entrypoint verification: a container pinned to AFP_CNID_BACKEND=dbd generates cnid scheme = dbd and runs the cnid daemon; an env-less container generates cnid scheme = sqlite.
  • afpd -v in the built image reports sqlite as the default backend.

Outstanding issue

The netatalk master still starts cnid_metad whenever the dbd backend is merely compiled in, so an env-less (sqlite) container runs an idle cnid daemon that nothing talks to. Startup should be conditional on a volume actually using cnid scheme = dbd — tracked in #3268.

@andylemin

Copy link
Copy Markdown
Contributor Author

@rdmark ready for review

Comment thread NEWS.md Outdated
@andylemin
andylemin force-pushed the perf-cnid-residuals-and-backlog branch from 6aa912a to 1f52a34 Compare August 26, 2026 14:24
@andylemin

Copy link
Copy Markdown
Contributor Author

@rdmark ready for final review 🤞

…to 128

SQLite has been the default CNID scheme since PR #2265 made meson
select sqlite > mysql > dbd among the compiled backends. Residue
still said dbd: the docker entrypoint's AFP_CNID_BACKEND fallback,
and the man page and NEWS never stated the default. Align them; no
default changes here. The CI cnid:dbd spectest leg relied on the old
docker fallback for its backend, so it now sets AFP_CNID_BACKEND=dbd
explicitly -- intentional dbd and mysql coverage is unchanged.

Also raise DSI_TCPMAXPEND from 20 to 128. It has been 20 since the
CVS era, with no config knob; a reconnect storm (default 200 max
connections) overflows it: dropped SYNs and ~1s client retransmit
delays. 128 matches the common SOMAXCONN floor; the kernel silently
caps at net.core.somaxconn anyway, so no new option is warranted.
No steady-state effect.
@andylemin
andylemin force-pushed the perf-cnid-residuals-and-backlog branch from 1f52a34 to e04a149 Compare August 26, 2026 16:46
@sonarqubecloud

Copy link
Copy Markdown

@github-actions

Copy link
Copy Markdown
Contributor

📊 Performance Dashboard

Commit: e04a149a97a8c9cf2191eff84c0c8ec686c5c7df

🔥 Spectest (AFP 3.4) - FlameGraph

Netatalk Code-time: 2.7% · Runtime: 60s · Stacks: 1332

🔥 Click the preview to open the interactive flamegraph (zoom + search).

Flamegraph preview

🔝 Top 10 leaf functions
Function Samples
do_syscall_64 351708665
x64_sys_call 197353765
[libsqlite3.so.3.53.4] 127894060
_raw_spin_unlock_irqrestore 119073780
finish_task_switch.isra.0 89305335
__cp_end 78279985
afpd 77177450
srso_alias_safe_ret 52921680
kmem_cache_alloc_noprof 31973515
syscall_trace_enter 28665910

📈 Speedtest (AFP 3.4) - PerfGraph

Speedtest throughput

Peak Read: 6342 MB/s (-11.4% vs hist avg 7160.9 MB/s; min 5963 / max 9526 over 27 PRs)
Peak Write: 1587 MB/s (+24.5% vs hist avg 1274.9 MB/s; min 225 / max 1863 over 27 PRs)

🔝 Throughputs per operation (vs. historical average)
Metric Current (MB/s) Cur Avg Δ% Hist avg Hist min Hist max
Read peak mean 6342 -11.4% 7160.9 5963 9526
Read avg mean 3529 -14.9% 4147.9 3524 5393
Read avg max 4015 -12.3% 4578.9 3812 5932
Write peak mean 1587 +24.5% 1274.9 225 1863
Write avg mean 493 +18.2% 417.1 91 554
Write avg max 813 +8.9% 746.5 224 966
Copy peak mean 2535 -4.5% 2654.1 2294 3353
Copy avg mean 1467 -6.0% 1561.4 1360 1954
Copy avg max 1648 -3.3% 1704.3 1515 2087
ServerCopy peak mean 3375 -8.0% 3669.3 3055 5090
ServerCopy avg mean 1803 -6.0% 1918.3 1604 2501
ServerCopy avg max 1858 -6.7% 1992.0 1675 2658

⏱️ Lantest (AFP 3.4) - LatencyGraph

Lantest latency

Avg total runtime: 4393 ms (+2.9% vs hist avg 4268.1 ms; min 2122 / max 5208 over 27 PRs)
Avg time per AFP op: 89 µs (+3.9% vs hist avg 85.7 µs; min 43 / max 105 over 27 PRs)

🐢 All operations (avg runtime, in test order, vs. historical average)
Metric Current (ms) Cur Avg Δ% Adj Δ% Hist avg Hist min Hist max
Writing one large file 39 +5.2% 37.1 26 47
Reading one large file 14 -9.4% 15.4 12 23
Creating 2000 files 497 -2.8% -15.3% 511.2 211 831
Create 2000 dirs tree (20×9×10) 473 -7.2% -19.7% 509.6 289 796
Open, write 1024 bytes, close 2000 files 420 +12.7% +0.1% 372.7 195 459
Open, read 1024 bytes, close 2000 files 367 +12.6% +0.0% 326.0 177 408
Copying 1000 files client-side (R+W) 573 +2.8% -9.8% 557.6 267 677
Copying 2000 files server-side 480 -7.5% -20.1% 518.9 182 727
Stat (lookup+getparams) 2000 files 252 +13.6% +1.0% 221.9 125 282
Enumerate dir with 2000 files 11 +29.7% +17.1% 8.48 3 14
Lock then unlock 2000 open forks 185 +13.4% +0.9% 163.1 111 199
Deleting 2000 files 333 -3.0% -15.5% 343.1 125 447
Byte-range lock/unlock 2000 ranges in one fork 183 +10.1% -2.4% 166.1 115 196
Directory cache hits (20 dirs x 100 files) 115 +16.5% +3.9% 98.7 59 122
Mixed cache operations (create/stat/enum/delete) on 500 files 222 +2.1% -10.5% 217.4 92 265
Deep path traversal (20 levels x 100 walks) 120 +13.9% +1.4% 105.3 61 131
Cache validation (500 files x 4 lookups) 109 +14.3% +1.7% 95.4 58 116

Run baseline: median op-test delta +12.6%, MAD 3.9%. Adj Δ% shifts each delta by the median; standouts ≥5% in bold. A large MAD means the run did not move uniformly — read the adjusted column with caution.

Performance trend

@andylemin
andylemin merged commit a4a4706 into main Aug 26, 2026
71 checks passed
@andylemin
andylemin deleted the perf-cnid-residuals-and-backlog branch September 6, 2026 13:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants