Skip to content

netatalk: start cnid_metad only when a volume uses the dbd scheme - #3269

Merged
andylemin merged 1 commit into
mainfrom
cnid-metad-on-demand
Aug 26, 2026
Merged

netatalk: start cnid_metad only when a volume uses the dbd scheme#3269
andylemin merged 1 commit into
mainfrom
cnid-metad-on-demand

Conversation

@andylemin

@andylemin andylemin commented Aug 24, 2026

Copy link
Copy Markdown
Contributor

Fixes #3268.

Problem

The netatalk master started cnid_metad whenever the dbd backend was merely compiled in (CNID_BACKEND_DBD), regardless of whether any volume has cnid scheme = dbd. All three backends compile by default and the default scheme is sqlite (in-process), so a stock install ran an idle cnid_metad/cnid_dbd pair that nothing talks to.

Design

The master already has the machinery: it loads volumes before starting services and re-loads them on SIGHUP, and its timer re-launches any service whose pid sentinel is NETATALK_SRV_NEEDED. The change makes the needed/optional decision from the configuration in one small exported check plus three call sites:

  • conf_cnid_scheme_in_use() (netatalk_conf.c): scans the loaded volume list, then the [Homes] section — homes volumes are instantiated per-user at login, so outside AFP sessions they never appear in the master's volume list; the section's scheme is resolved exactly as creatvol() resolves it (section → preset → global → compiled default). It lives in netatalk_conf.c because those resolution helpers are static there.
  • Startup: skip cnid_metad when no volume resolves to dbd.
  • Timer restart guard: the reaper marks a dead cnid_metad as NETATALK_SRV_ERROR, which equals NETATALK_SRV_NEEDED — without a guard the timer would resurrect a daemon a reload just made unnecessary; the guard demotes instead. This one check makes stop transitions converge through the existing machinery with no new state.
  • SIGHUP: after the volume reload, stop a running daemon no dbd volume needs (SIGTERM → reap → guarded demotion) or mark it needed when one now does (next timer tick starts it). Both transitions are logged.

Robustness: if the master's volume load fails, an empty list is indistinguishable from "no volumes", so a volumes_loaded flag makes every decision fail conservative — the daemon runs, exactly as today. Builds without the dbd backend are unchanged: the pid sentinel can never become needed there (the SIGHUP re-evaluation is compile-gated), so the master never attempts to start a binary that isn't installed.

Reload semantics

Start/stop on SIGHUP applies to added or removed volumes. Changing cnid scheme on an existing volume takes effect at restart only — creatvol() does not change options for volumes once loaded, which is netatalk's long-standing reload behaviour for all volume options; this PR matches it rather than changing it.

Testing

All cases run against live containers built from this branch.

build configuration expected result
all backends (default) cnid scheme = sqlite volumes no cnid_metad ✓ 0 processes
all backends cnid scheme = dbd volumes daemon running
all backends only share is [Homes], global cnid scheme = dbd daemon running ✓ — the homes resolution path
all backends SIGHUP: add a new dbd volume started ✓ "Starting 'cnid_metad': a volume now uses the dbd CNID scheme", process appears
all backends SIGHUP: remove that volume stopped, not restarted ✓ "Stopping 'cnid_metad': …", cnid_metad logs "shutting down on SIGTERM", process gone, no restart — the timer guard held
all backends kill -9 a needed cnid_metad restarted by the master ✓ existing behaviour preserved
all backends AFP spectest ea=sys and ea=ad legs green ✓ exit 0, zero failures
all backends meson test green ✓ zero failures
no dbd (-Dwith-cnid-backends=sqlite,mysql) build + unit suite green; cnid_metad not installed
no dbd sqlite volume, live master master + afpd run, no metad
no dbd volume explicitly cnid scheme = dbd inert: no metad, no start-error loop ✓ — the volume fails at client open as on any backend-less build; cnid_open() has no backend fallback (verified)
dbd only (-Dwith-cnid-backends=dbd) stock config, no scheme set anywhere daemon running ✓ — the compiled default is dbd here, exercising the check's default-resolution end to end
dbd only volume explicitly sqlite (unavailable backend) no metad ✓ — masks nothing; the volume fails at open, pre-existing

(The dbd-only build cannot run the unit suite: with-tests uses the sqlite backend because the cnid_metad / cnid_dbd daemon combo needs root privileges to run, so dbd was deliberately left out of the afpd tests.)

@andylemin
andylemin requested a review from a team August 24, 2026 10:40
@andylemin
andylemin requested a review from rdmark as a code owner August 24, 2026 10:40
@andylemin

Copy link
Copy Markdown
Contributor Author

@rdmark ready for review

Comment thread NEWS.md Outdated

@rdmark rdmark left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is a nice improvement! one more step towards completely decoupling from the old dbd structure

@rdmark

rdmark commented Aug 25, 2026

Copy link
Copy Markdown
Member

(The dbd-only build cannot run the unit suite: meson's with-tests requires the sqlite backend — a build-system constraint, not a gap.)

not strictly speaking a build system constraint, rather that root privileges are required to run the cnid_metad / cnid_dbd daemon combo so I decided not to use it for the afpd tests

we could easily add support for the mysql backend in the afpd tests if it's valuable

The master started cnid_metad whenever the dbd backend was compiled
in, regardless of whether any volume has 'cnid scheme = dbd'. All
backends compile by default and the default scheme is sqlite
(in-process), so a stock install ran an idle cnid_metad/cnid_dbd
pair that nothing talks to.

Decide from the configuration instead: start cnid_metad at startup
only when a volume resolves to dbd, re-evaluate on SIGHUP (starting
or stopping it as volumes change), and demote a crash-restart to
no-op when dbd is no longer in use. The check also resolves the
[Homes] section, which the master's volume load skips (homes
volumes are instantiated per-user at login), and falls back to
starting the daemon when the volume load itself failed. Builds
without the dbd backend are unchanged.
@andylemin
andylemin force-pushed the cnid-metad-on-demand branch from c000f3c to 73d230c Compare August 26, 2026 14:42
@sonarqubecloud

Copy link
Copy Markdown

@andylemin

andylemin commented Aug 26, 2026

Copy link
Copy Markdown
Contributor Author

@rdmark feedback resolved

@github-actions

Copy link
Copy Markdown
Contributor

📊 Performance Dashboard

Commit: 73d230cf893f568fd3cf151e3d8d922deaf0b4cc

🔥 Spectest (AFP 3.4) - FlameGraph

Netatalk Code-time: 2.4% · Runtime: 61s · Stacks: 1638

🔥 Click the preview to open the interactive flamegraph (zoom + search).

Flamegraph preview

🔝 Top 10 leaf functions
Function Samples
do_syscall_64 392502460
x64_sys_call 251377980
finish_task_switch.isra.0 174200530
[libsqlite3.so.3.53.4] 163175180
_raw_spin_unlock_irqrestore 145534620
__cp_end 115766175
srso_alias_safe_ret 50716610
syscall_trace_enter 38588725
__syscall_cp_c 38588725
afpd 28665910

📈 Speedtest (AFP 3.4) - PerfGraph

Speedtest throughput

Peak Read: 6864 MB/s (-4.2% vs hist avg 7165.5 MB/s; min 5963 / max 9526 over 27 PRs)
Peak Write: 1863 MB/s (+46.0% vs hist avg 1275.9 MB/s; min 225 / max 1981 over 27 PRs)

🔝 Throughputs per operation (vs. historical average)
Metric Current (MB/s) Cur Avg Δ% Hist avg Hist min Hist max
Read peak mean 6864 -4.2% 7165.5 5963 9526
Read avg mean 4007 -3.5% 4151 3524 5393
Read avg max 4438 -3.0% 4575.4 3812 5932
Write peak mean 1863 +46.0% 1275.9 225 1981
Write avg mean 552 +31.4% 420.0 91 598
Write avg max 966 +29.3% 747.3 224 1032
Copy peak mean 2654 -0.1% 2655.8 2294 3353
Copy avg mean 1532 -2.2% 1566.2 1360 1954
Copy avg max 1600 -6.6% 1712.3 1515 2087
ServerCopy peak mean 3481 -5.3% 3677.3 3055 5090
ServerCopy avg mean 1881 -2.1% 1922 1604 2501
ServerCopy avg max 1932 -3.2% 1995.6 1675 2658

⏱️ Lantest (AFP 3.4) - LatencyGraph

Lantest latency

Avg total runtime: 2906 ms (-31.6% vs hist avg 4249.2 ms; min 2122 / max 5208 over 27 PRs)
Avg time per AFP op: 59 µs (-30.8% vs hist avg 85.3 µs; min 43 / max 105 over 27 PRs)

🐢 All operations (avg runtime, in test order, vs. historical average)
Metric Current (ms) Cur Avg Δ% Adj Δ% Hist avg Hist min Hist max
Writing one large file 47 +28.3% 36.6 26 41
Reading one large file 23 +51.8% 15.1 12 18
Creating 2000 files 327 -35.7% -7.2% 508.8 211 831
Create 2000 dirs tree (20×9×10) 311 -38.9% -10.3% 508.8 289 796
Open, write 1024 bytes, close 2000 files 273 -26.4% +2.2% 370.7 195 459
Open, read 1024 bytes, close 2000 files 236 -27.3% +1.2% 324.7 177 408
Copying 1000 files client-side (R+W) 384 -30.8% -2.2% 554.6 267 677
Copying 2000 files server-side 305 -40.8% -12.3% 515.4 182 727
Stat (lookup+getparams) 2000 files 158 -28.7% -0.1% 221.4 125 282
Enumerate dir with 2000 files 8 -4.8% +23.7% 8.41 3 14
Lock then unlock 2000 open forks 128 -21.4% +7.1% 162.9 111 199
Deleting 2000 files 211 -38.1% -9.6% 340.9 125 447
Byte-range lock/unlock 2000 ranges in one fork 133 -19.9% +8.6% 166.1 115 196
Directory cache hits (20 dirs x 100 files) 75 -23.9% +4.6% 98.6 59 122
Mixed cache operations (create/stat/enum/delete) on 500 files 143 -33.8% -5.3% 216 92 265
Deep path traversal (20 levels x 100 walks) 75 -28.5% +0.0% 105.0 61 131
Cache validation (500 files x 4 lookups) 69 -27.5% +1.1% 95.1 58 116

Run baseline: median op-test delta -28.5%, MAD 5.3%. Adj Δ% shifts each delta by the median; standouts ≥5% in bold. A large MAD means the run did not move uniformly — read the adjusted column with caution.

Performance trend

@andylemin
andylemin merged commit 4eb2ca4 into main Aug 26, 2026
71 checks passed
@rdmark
rdmark deleted the cnid-metad-on-demand branch August 26, 2026 18:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

netatalk: only start cnid_metad when a volume uses the dbd CNID scheme

2 participants