Skip to content

feat(magicc7): concentration-driven mode for CO2 / CH4 / N2O#104

Draft
benmsanderson wants to merge 8 commits into
feat/fair2-ciceroscmpy2-adapters-and-runmode-nonforkfrom
magicc_concdriven
Draft

feat(magicc7): concentration-driven mode for CO2 / CH4 / N2O#104
benmsanderson wants to merge 8 commits into
feat/fair2-ciceroscmpy2-adapters-and-runmode-nonforkfrom
magicc_concdriven

Conversation

@benmsanderson

Copy link
Copy Markdown
Collaborator

What

Brings the MAGICC7 adapter to parity with the FaIR2 / CICEROSCMPY2 adapters' RunMode.CONCENTRATION_DRIVEN support, so a single openscm_runner.run.run call can drive all three models from Atmospheric Concentrations|* inputs.

v1 scope: CO2 / CH4 / N2O only — the three species MAGICC7 exposes via per-gas FILE_<gas>_CONC + <gas>_SWITCHFROMCONC2EMIS_YEAR cfg flags. F-gases / Montreal halocarbons (bundled-array flags) are deferred; user overlay rows for unsupported species fall through to the SCEN7 emissions path. Per-gas mixed mode mirrors FaIR2: a gas is concentration-driven only when every scenario in the batch supplies it.

rcmip3_bundle_path is required for conc-driven runs (matches FaIR2 / CICEROSCMPY2). Baseline + overlay: load the RCMIP3 baseline and overlay user-supplied concentration rows on top (year-by-year, user wins).

Key implementation note

MAGICC reads every *_CONC.IN as a contiguous annual series over its full internal window (its shipped files are ANNUALSTEPS=1, 801 rows, 1700–2500). The writer resamples whatever (possibly sparse) trajectory it has onto that grid before writing, or the Fortran readdata routine hits end-of-file at runtime.

Verification

Verified end-to-end against the real MAGICC v7.5.3 binary (ssp245):

mode GSAT @ 2100 (CS=3, member 1)
emissions-driven ~2.77 K
concentration-driven ~2.88 K

Same ballpark, as expected for the physically-consistent bundle. Tests:

  • tests/unit/adapters/test_magicc7_concentrations.py — pure-Python cfg-key + overlay-merge semantics (no binary).
  • tests/integration/test_magicc7.py — conc-file-writing smoke, rcmip3_bundle_path-required guard, and a binary-driven end-to-end test (test_conc_driven_end_to_end_runs_the_binary) so the annual-grid contract can't regress silently.

Notes

  • No changelog/*.md entry yet — will add with this PR number.
  • Draft: stacked on feat/fair2-ciceroscmpy2-adapters-and-runmode-nonfork.

🤖 Generated with Claude Code

benmsanderson and others added 5 commits June 19, 2026 23:53
Adds RunMode.CONCENTRATION_DRIVEN support to the MAGICC7 adapter,
mirroring the FaIR2 / CICEROSCMPY2 shape: hybrid RCMIP3 baseline +
user overlay (year-by-year merge), per-gas mixed-mode filter
(intersection over batch scenarios), rcmip3_bundle_path required
on the cfg. v1 covers CO2 / CH4 / N2O — the species MAGICC7 exposes
via per-gas FILE_<gas>_CONC + <gas>_SWITCHFROMCONC2EMIS_YEAR cfg
flags; F-gases and Montreal halocarbons (bundled-array
FGAS_FILES_CONC / MHALO_FILES_CONC flags) fall through to the
SCEN7 emissions path until a follow-up adds the bundled-array
machinery.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Captures scope decisions, the v1 scope reduction to CO2 / CH4 / N2O,
files changed, what's verified vs. blocked on this machine
(unsigned-binary restriction), pickup instructions for another
machine, open items to check during the end-to-end run, and the
follow-up work for F-gas / Montreal-halocarbon bundled-array
support.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…grid

The real MAGICC7 binary reads every *_CONC.IN as a contiguous annual
series over its full internal window (its shipped files are
ANNUALSTEPS=1, 801 rows, 1700-2500). The writer previously emitted only
the years present in the source bundle, so a sparse/short trajectory
made the Fortran readdata routine hit end-of-file
(MAGICC7.f90:11903) and the run failed with "No runs to append".

Resample the (possibly sparse) overlay onto the annual grid before
writing: linear interpolation within the supplied range, constant hold
outside. Verified end-to-end against magicc v7.5.3 (ssp245, both modes
land in the same ballpark).

Also:
- add a binary-driven end-to-end conc test so this can't regress
  silently (the file-writing tests bypass the binary).
- fix the conc-file-writing test's cfg-key assertion: the correct key
  is file_co2_conc, not file_co2_concentration (matches
  MAGCFG_DEFAULTALL.CFG).
- record the verification + qemu setup in the handover note.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ode (#104)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@znichollscr

Copy link
Copy Markdown

@benmsanderson for part 2, here's the prompt I would use in case it helps

Adding support for the F-Gases and Montreal Protocol Halogens should be quite straightforward. You need to set the fgas_switchfromconc2emis_year and mhalo_switchfromconc2emis_year to the end of the simulation (or a really big number like 10_000 is also fine). Then you need to write all the concentration files and fill the written files into the FGAS_FILES_CONC and MHALO_FILES_CONC input lists. The order of these files must match the FGAS_NAMES and MHALO_NAMES lists. If you do this for a CMIP6 scenario, e.g. ssp245, you can check your written configuration against the files that are already shipped with the magicc bundle, e.g. run/rcmip/MAGTUNE_SSP245.CFG. When you do this, you will notice that no file is specified for HALON1202. If HALON1202 is not in the RCMIP3 protocol, then this is the correct behaviour: MAGICC will us its default HALON1202 values instead (not ideal as it doesn't vary by scenario, but fine as this is a minor gas). If HALON1202 is in the RCMIP3 protocol, then these concentrations should be written too (and you might get a different result from what is in run/rcmip/MAGTUNE_SSP245.CFG).

You can check that this works by requesting the concentrations as outputs (i.e. requesting the Atmospheric Concentrations|Species variable for each species that is concentration-driven as output) and checking that what is returned as output matches what was passed in.

Per-gas mixed mode mirrors FaIR2: a gas is concentration-driven only when every scenario in the batch supplies it

I thought we removed this?

Same ballpark, as expected for the physically-consistent bundle

If you want to go really deep, you can check that we reproduce the values in AR6 Ch. 7 Table SM.7.2 (I think, from memory) for one of the SSPs - those runs are concentration-driven and should be fully reproducible (I have code that actually reproduces them somewhere, which I can pull out for the SOD if claude goes round in circles).

@benmsanderson

benmsanderson commented Jun 20, 2026 via email

Copy link
Copy Markdown
Collaborator Author

benmsanderson and others added 3 commits June 22, 2026 14:28
Addresses Zeb's review of #104.

- Remove the FaIR2-style "every scenario in the batch must supply the
  gas" intersection filter. MAGICC writes an independent cfg + CONC.IN
  per (scenario, model), so the conc-driven decision is now per-scenario:
  a gas can be conc-driven in one scenario and emissions-driven in
  another within the same batch.
- Add F-gas and Montreal-halocarbon support via the bundled-array flags
  FGAS_FILES_CONC / MHALO_FILES_CONC, positional over FGAS_NAMES /
  MHALO_NAMES, with the shared *_SWITCHFROMCONC2EMIS_YEAR set to 10000.
  HALON1202 (no RCMIP3 trajectory) falls back to MAGICC's default; the
  writer warns on any other unexpected empty slot.
- Add to_magicc_species() to normalise RCMIP3 mixed-case leaf names
  (HFC134a, CCl4, Halon1211) onto MAGICC's upper-case FGAS/MHALO names,
  and classify_conc_species() to route per-gas vs bundled-array paths.
- Fix write_conc_in_file() to label the CONC.IN data variable with
  pymagicc's openscm name (convert_magicc7_to_openscm_variables), not the
  raw MAGICC token. pymagicc cross-checks the filename-derived variable
  against the data variable; the raw token tripped this for every species
  whose openscm name differs in case (e.g. HFC134A vs HFC134a). Caught by
  the new binary end-to-end test; identical for CO2/CH4/N2O.

Tests (verified against the real MAGICC v7.5.3 binary):
- per-scenario / classification / case-normalisation unit tests
- bundled-array writing, FGAS_NAMES/MHALO_NAMES cross-check against the
  binary's MAGCFG_DEFAULTALL.CFG, and a CO2 conc round-trip
- new rcmip3-mini-halo test bundle (kept separate so the binary GSAT
  comparison stays on the WMGHGs)

Docs: add a "Concentration-driven runs" page comparing MAGICC7 / FaIR2 /
CICEROSCMPY2 conc-driven capability and mode scoping.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Opt-in smoke harness that reproduces the concentration-driven
"Future Warming (GSAT)" block of IPCC AR6 WG1 Table 7.SM.4 for MAGICC7:

- _ar6_validation.py: load the AR6 probabilistic drawnset (600 parameter
  sets) into MAGICC7 cfgs (paraset_id -> run_id), load the cleaned
  reference table, and reduce a run to per-period GSAT percentiles
  rebased to the 1995-2014 mean (reusing scmdata's
  relative_to_ref_period_mean and openscm-runner's calculate_quantiles
  conventions).
- test_magicc7_ar6_validation.py: run ssp245 concentration-driven for a
  small ensemble and compare 5/50/95 percentiles to the published
  MAGICC7 column. Gated on a local drawnset (licensed, not vendored,
  via AR6_MAGICC_DRAWNSET) and the MAGICC binary, so skipped by default.
  Point AR6_RCMIP3_BUNDLE at the full RCMIP3 bundle to tighten the check
  (abs=0.15) toward a faithful reproduction; the default mini-bundle
  smoke run asserts only a ballpark (abs=1.0).
- table_7_SM_4.csv: cleaned reference fixture (all five emulator columns
  + assessed ranges) for the key metrics, ERF and SSP GSAT rows.

Smoke run (10 members, WMGHG-only conc) already tracks the AR6 MAGICC7
medians: SSP2-4.5 2081-2100 central 1.71 vs 1.82 published.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Ran the validation against the full RCMIP3 concentration bundle and the
full 600-member AR6 drawnset. The conc-driven GSAT medians reproduce the
published MAGICC7 column to <=0.01 degC across all three SSP2-4.5 periods
(e.g. 2081-2100: 1.82 vs 1.82) — strong end-to-end validation including
the F-gas/MHalo bundled-array path driven through the binary.

Strengthen the value check accordingly: when run with the full bundle and
>=100 members, assert the median lands on the published central for every
period (abs=0.1). The 5th/95th percentiles run wider (95th high) because
the drawnset is the AR6 prior while 7.SM.4 ranges come from the
constrained/weighted distribution, so the test validates the median and
only reports the tails.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@benmsanderson

benmsanderson commented Jun 22, 2026

Copy link
Copy Markdown
Collaborator Author

Alright then @znichollscr

  1. The intersection rule — gone. MAGICC writes its own cfg + CONC.IN per scenario, so it never needed FaIR2's "every scenario must supply the gas" constraint. It's now per-scenario: a gas can be conc-driven in one scenario and emissions-driven in another. (Side effect: MAGICC's now more permissive than FaIR2 here, since FaIR2 still has the rule — I'll keep this PR magicc-only, but we can work to harmonise the approach in Feat/fair2 ciceroscmpy2 adapters and runmode nonfork #97 )

  2. F-gases + Montreal halocarbons — done your way: FGAS_FILES_CONC / MHALO_FILES_CONC ordered to FGAS_NAMES / MHALO_NAMES, shared switch-year at 10000, HALON1202 slot left empty → MAGICC default (matches MAGTUNE_SSP245.CFG). There's a test cross-checking the name lists against the binary's MAGCFG_DEFAULTALL.CFG. Since each group shares one switch-year, conc-driving is all-or-nothing within a group, so I fill the whole group from the baseline and warn on unexpected empty slots.
    Current behaviour: if the group is conc-driven but the bundle is missing a trajectory for some species in it (e.g. in RCMIP - HALON1202 always; potentially others if a bundle is incomplete), the adapter will "warn-and-use-defaults" : leave that species' slot empty → fill it from MAGICC's own built-in default concentration, and we emit a warnign naming the species. So the gas is still concentration-driven, just from MAGICC's default values rather than the scenario's. alternative is to switch to emis driven for the whole group (or just fail, I guess)

Verified against real MAGICC v7.5.3 — which caught a bug: pymagicc checks the CONC.IN variable against the species token in the filename using its own naming (HFC134a, not HFC134A), so anything with a case difference blew up. Fixed the writer to use convert_magicc7_to_openscm_variables. Your round-trip check (conc in == conc out) is now in the end-to-end test.

  1. AR6 reproduction — took your Table 7.SM.x suggestion and built an (opt-in) validation harness: load the AR6 600-member drawnset, run conc-driven, rebase GSAT to 1995–2014, compare percentiles. With the full RCMIP3 bundle + full ensemble the medians reproduce the MAGICC7 column almost exactly:

period SSP245 relative to 1995-2014, 600 members ours (5/50/95) AR6 MAGICC7
2021-2040 0.42 / 0.64 / 1.13 0.45 / 0.64 / 0.89
2041-2060 0.74 / 1.14 / 2.01 0.79 / 1.13 / 1.60
2081-2100 1.15 / 1.82 / 3.30 1.21 / 1.82 / 2.67
Medians match to ≤0.01 °C (validates the conc-driven path incl. the F-gas/MHalo arrays end-to-end). The 95th runs wide because I'm running the raw drawnset unweighted — I think the published ranges are the constrained/weighted distribution, which would explain the heavy upper tail with the median unchanged. This is where your reproduction code would help — mainly to confirm the constraint weights (and that I've got the concentration inputs right).

-Also added a short docs page comparing conc-driven support across the three models — species coverage is at parity across models with this PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants