Adding support for new fetchers 2.0 Account fetchers

[karwaankit32]

Adding support for new fetchers 2.0 Account fetchers

Design - #4860

Validation Summary

Validation was performed in a non-production Kubernetes canary environment using both filesystem-backed and HTTP-backed account sources.

Design Reference

#4860

Fetchers 2.0 caches the final typed *config.Account instead of caching raw JSON bytes. This avoids repeated account defaults merge, JSON unmarshal, DSA unpacking, derived GDPR map creation, and IP masking defaults on account cache hits.

Metrics Used

Metric	Meaning
prebid_server_cachekit_cache_result{result="miss",subsystem="account"}	Fetchers 2.0 had to load or reload an account
prebid_server_cachekit_cache_result{result="hit",subsystem="account"}	Fetchers 2.0 served typed cached account
prebid_server_cachekit_cache_result{result="negative",subsystem="account"}	Fetchers 2.0 served cached missing or malformed account verdict
prebid_server_cachekit_backend_fetch{result="ok",subsystem="account"}	source returned valid account data
prebid_server_cachekit_backend_fetch{result="notfound",subsystem="account"}	source returned definitive missing account
prebid_server_cachekit_backend_fetch{result="error",subsystem="account"}	source or transform failed
prebid_server_account_cache_performance{cache_result="hit/miss"}	legacy account byte-cache activity

Compatibility and Filesystem Source Tests

Area	Configuration	Test	User Result	Metrics / Proof	Result
Backward compatibility	Fetchers 2.0 unset	Known account through /cookie_sync	HTTP 200; account config applied	no cachekit metrics; legacy cache active	Passed
Backward compatibility	accounts.v2_enabled=false	Known account through /cookie_sync	HTTP 200	legacy cache miss then hit; cachekit absent	Passed
Filesystem lazy load	lru, refresh=ttl, negative off	known account twice	HTTP 200 both	backend ok +1, miss +1, hit +1	Passed
Filesystem TTL reload	TTL 30s, serve_stale=false	known account, wait past TTL, same account	HTTP 200	backend ok increased again; miss increased	Passed
Filesystem missing, no negative	negative off	same missing account twice	HTTP 200 with defaults	notfound +2, no negative metric	Passed
Filesystem negative cache	negative on	same missing account twice	HTTP 200 with defaults	first notfound; second negative hit	Passed
Filesystem negative TTL	negative TTL 10s	missing, missing, wait, missing	HTTP 200	negative before TTL; backend notfound after TTL	Passed
Filesystem no refresh	refresh=none	known account, wait past TTL, same account	HTTP 200	backend ok stayed 1; second call hit	Passed
Filesystem no positive cache	cache.type=none, negative off	valid twice, missing twice	HTTP 200	valid backend ok +2; missing notfound +2	Passed
Filesystem no positive cache + negative	cache.type=none, negative on	missing account twice	HTTP 200	first notfound; second negative hit	Passed
Filesystem serve stale	serve_stale=true	warm account, wait TTL, same account	HTTP 200 fast	stale hit; background backend refresh	Passed
Filesystem preload	refresh=preload	first known account request after restart	HTTP 200	preload backend ok; first request hit	Passed
Filesystem LRU eviction	max_entries=1	account A, account B, account A	HTTP 200	backend ok +3; all misses	Passed

HTTP Source Tests

Area	Configuration	Test	User Result	Metrics / Proof	Result
HTTP valid account	mock HTTP API, refresh=ttl	known account twice	HTTP 200 both	first backend ok; second hit	Passed
HTTP missing account	account omitted from response, negative on	same missing account twice	HTTP 200 with defaults	first notfound; second negative hit	Passed
HTTP null account	account value is null, negative on	same account twice	HTTP 200 with defaults	first notfound; second negative hit	Passed
HTTP 404	HTTP 404, negative on	same account twice	HTTP 200 with defaults	first notfound; second negative; mock called once	Passed
HTTP 500	HTTP 500	same account three times	HTTP 200 with defaults	backend error +3; not cached	Passed
HTTP bad JSON	invalid JSON body	same account three times	HTTP 200 with defaults	backend error +3; not cached	Passed
HTTP malformed account	invalid account shape	same account twice	HTTP 400	first backend error; second negative	Passed
HTTP bad field type	invalid account field type	same account twice	HTTP 400	first backend error; second negative	Passed
HTTP slow account	API sleeps 2s then returns valid account	same account twice	first slow, second fast	backend ok +1; cache hit +1	Passed
HTTP unavailable endpoint	endpoint on closed port	same account twice	timeout/no body	backend error +2; not cached	Passed
HTTP-only preload	refresh=preload, HTTP source only	startup then known account	startup OK with warning; request OK	no real preload; lazy load worked	Passed

Endpoint Coverage

Endpoint	Account Source	Test	User Result	Metrics / Proof	Result
/cookie_sync	filesystem	known account with account-level cookie sync config	HTTP 200	v2 metrics moved; legacy cache zero	Passed
/cookie_sync	HTTP	known account from mock HTTP API	HTTP 200	v2 metrics moved; legacy cache zero	Passed
/openrtb2/auction	filesystem	auction with parent account	HTTP 200 with seatbid	v2 backend ok/miss; legacy cache zero	Passed
/openrtb2/auction	HTTP	auction with parent account from mock API	HTTP 200 with seatbid	v2 backend ok/miss; legacy cache zero	Passed

Stress and Thundering Herd Tests

Stress Area	Configuration	Scale	Expected	Proof	Result
Invalid ID memory bound	negative max entries 100	1000 unique invalid IDs per pod	memory stays bounded; older negatives evict	memory stayed around 184-188 Mi; old ID queried backend again	Passed
Valid account coalescing	same cold valid account	100 parallel requests	one backend fetch	backend ok +1, cache hit +99	Passed
Invalid account coalescing	same cold invalid account	100 parallel requests	one backend notfound	backend notfound +1, negative +99	Passed
HTTP slow valid herd	mock API sleeps 2s	1000 parallel requests	one upstream call	MOCK_SLOW_DELTA=1, backend ok +1	Passed
HTTP missing herd	same missing account	1000 parallel requests	one upstream call, rest negative	MOCK_ABSENT_DELTA=1, negative +997	Passed
Serve stale protection	warm cache, backend slow 500, serve_stale=true	request after TTL	user gets stale account fast	user returned in ~0.5ms; backend error later	Passed

Notes

Topic	Observation
HTTP-only preload	Does not warm accounts because the HTTP account endpoint is by-ID only; startup logs warning and lazy-loads
Cold cache + dead HTTP endpoint	Request waits for backend timeout, then falls back to account defaults
Warm cache + serve_stale=true	Request returns last good typed account immediately; failed refresh happens in background
HTTP 404	Now classified as notfound so it can be negative-cached
Legacy cache bypass	During Fetchers 2.0 enabled tests, prebid_server_account_cache_performance stayed zero

[karwaankit32]

Will implement this in a follow up PR.