Skip to content

Add pricing experiment suite and cache measurement#61

Open
JaeLeex wants to merge 2 commits into
jl/hosted-metering-upload-20260701from
jl/pricing-experiments-20260701
Open

Add pricing experiment suite and cache measurement#61
JaeLeex wants to merge 2 commits into
jl/hosted-metering-upload-20260701from
jl/pricing-experiments-20260701

Conversation

@JaeLeex

@JaeLeex JaeLeex commented Jul 1, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Add hosted-agent pricing experiment tooling: pricing_loop, pricing_experiment_suite, cache_savings_experiment, and validate_combined_ledger.
  • Improve funded_btcswp_combined_run.py with OpenRouter cache metrics, optional fill polling, and ledger validation hooks.
  • Add modules/openrouter_usage.py for shared cache metric extraction.
  • Add strategies/llm_monitoring.py for monitoring/hedge-heartbeat qualification runs.
  • Fix hl trade --dry-run so it no longer requires a private key when --price is explicit.

Experiment commands

# Full local suite (skips live OpenRouter if no key)
python3 scripts/pricing_experiment_suite.py --suite all --skip-live

# Cache measurement (needs OPENROUTER_API_KEY)
NUNCHI_EXPERIMENT_ID=cache-$(date +%Y%m%d) python3 scripts/cache_savings_experiment.py --rounds 6

# Monitoring / hedge heartbeat qualification
python3 scripts/pricing_loop.py run --scenario pilot_monitoring --agents 1 --duration-seconds 300 --tick 10
python3 scripts/pricing_loop.py run --scenario pilot_hedge_heartbeat --agents 1 --duration-seconds 300 --tick 10

# Combined LLM + trade ledger join (live trades need maker/taker keys + --confirm)
python3 scripts/funded_btcswp_combined_run.py --price 24000 --model openai/gpt-4o-mini --maker-taker --confirm --validate-fills
python3 scripts/validate_combined_ledger.py --input-dir data/funded_combined_run
python3 scripts/pricing_aggregate.py --input-dir data/funded_combined_run

Validation

  • python3 -m pytest tests/test_openrouter_usage.py tests/test_validate_combined_ledger.py tests/test_trade_command.py tests/test_cost_metering.py — pass
  • python3 scripts/pricing_experiment_suite.py --suite combined_dry_run --skip-live — pass

Stack

Made with Cursor

JaeLeex and others added 2 commits July 1, 2026 06:52
Introduce pricing_loop, cache_savings_experiment, validate_combined_ledger, and pricing_experiment_suite; improve funded combined runs with cache metrics and fill polling; allow hl trade --dry-run without a private key.

Co-authored-by: Cursor <cursoragent@cursor.com>
Include market whitelist and pulse webhook modules, HOUSE-Jump preset config, railpack metadata, and web-auth/Hyperliquid action test coverage for the pricing experiment suite.

Co-authored-by: Cursor <cursoragent@cursor.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant