A computational formalization of Clayton Christensen's "Capitalist's Dilemma" as a cost–benefit arbitration between model-based (analytical) and model-free (intuitive) reinforcement-learning systems — with a runnable simulation, a full statistical-analysis pipeline, and a deployable human-behavioral experiment.
📄 Paper: paper/Diak-2021-Modeling-the-Capitalists-Dilemma-with-Multiple-RL-Systems.pdf
(Diak, 2021 — HBSDOC 4882, Harvard University)
Christensen and van Bever's Capitalist's Dilemma (HBR, 2014) makes a qualitative, influential, but slippery claim: when capital is framed as a scarce resource, managers lean on analytical valuation metrics (NPV, IRR, RONA, ROIC) and away from the long-horizon, balance-sheet-heavy bets that actually produce growth.
This project renders that claim as a mechanism. Drawing on the dual-process literature in cognitive neuroscience, it models the two decision modes as two reinforcement-learning systems:
- Model-based control = System 2 / analytical. Learns the task's transition structure and plans — the RL analogue of NPV-style valuation with an explicit financial model.
- Model-free control = System 1 / intuitive. Caches state–action values from experience via temporal-difference learning — the RL analogue of experience-driven gut calls.
A hybrid agent mixes them by an arbitration weight w:
Q_hybrid(s, a) = w · Q_MB(s, a) + (1 − w) · Q_MF(s, a)
The paper's central move: a framing effect is not (only) a downstream bias of System 1, but an
upstream modulation of the arbitration weight w itself — the parameter that governs which
regime the system operates in. Framing capital as scarce is hypothesized to shift w toward
model-based control. In dynamical-systems terms, w is the order parameter that tips the agent
between two computational regimes.
capitalists_dilemma/
├── environment/ # Two-stage investment task (deterministic-transition MDP)
│ ├── two_stage_task.py # Kool et al. "novel two-step" task, investment-framed
│ └── reward_schedule.py # Drifting (Gaussian random walk) reward schedules
├── agents/
│ ├── model_free.py # SARSA(λ) temporal-difference agent (System 1)
│ ├── model_based.py # Transition-learning planning agent (System 2)
│ └── hybrid.py # Weighted MB/MF arbitration (the `w` parameter)
├── simulation/
│ ├── conditions.py # 2×3 factorial conditions
│ └── run_simulations.py # Batch simulation engine
├── analysis/
│ ├── stay_probability.py # Logistic regression — the canonical MB-vs-MF signature
│ ├── model_fitting.py # Fit the hybrid RL model to behavioral data
│ └── factorial_anova.py # 2×3 ANOVA on estimated control weights
├── visualization/plots.py
├── web_task/task_spec.md # Browser (jsPsych) experiment for the human arm
├── results/ # Figures + summary statistics (see below)
├── tests/ # Environment unit tests
├── config.py # All hyperparameters and experimental constants
└── run_experiment.py # Entry point
pip install -r requirements.txt
# 1. Run the full simulation suite (6 conditions × 1000 agents) — regenerates results/simulation_data.csv
python run_experiment.py --mode simulate
# 2. Analyze (stay-probability regression, model fit, factorial ANOVA)
python run_experiment.py --mode analyze
# 3. Regenerate all figures
python run_experiment.py --mode plot
results/simulation_data.csv(~13 MB) is a regenerable artifact and is git-ignored; step 1 recreates it. The summary outputs (figures,anova_results.txt,w_sweep.csv) are committed so the results are visible without running anything.
The two-stage investment task. Adapted from Kool, Cushman & Gershman's (2016) novel two-step task with an investment framing. Deterministic first-stage transitions and alternating market states are what create a genuine accuracy advantage for model-based planning over model-free caching — the original Daw et al. (2011) 70/30 task does not, so it can't be used to test whether framing modulates arbitration. Stage-2 rewards drift as a bounded Gaussian random walk.
2×3 factorial. Capital framing (scarce / not scarce) × reliability benefit of model-based
control (+ / = / −), the latter set by how correlated and volatile the outcome rewards are.
In the human arm, framing is manipulated by instructions; in simulation, framing is operationalized
as a shift in w, letting us establish the reward-optimal arbitration weight per condition and
validate the analysis pipeline.
Hypotheses (Diak 2021). H1: scarce framing → higher w. H2: scarce × +reliability →
interaction amplifying model-based control. H3: framing shifts w even absent a reliability
benefit (a genuine framing effect, not just reward-chasing).
The w-sweep recovers the expected reward-optimal arbitration weight in each reliability
condition (results/w_sweep.png), the stay-probability analysis reproduces the canonical
model-based vs model-free behavioral signatures (results/figures/stay_probabilities.png), and
the factorial ANOVA over the simulated control weights cleanly separates the conditions
(results/anova_results.txt, results/figures/factorial_results.png).
Honest scope. In these runs framing is imposed as a w-shift by construction, so the
simulation confirms that the model and the statistical pipeline recover and can detect the
hypothesized effects with adequate power — it is a validation of the design, not evidence about
human cognition. Whether framing causally shifts arbitration in people is what the browser
experiment in web_task/ is built to test; that human arm is the forward piece of the program.
Diak, C. J. (2021). Modeling the "Capitalist's Dilemma" Using Multiple Reinforcement Learning Systems. HBSDOC 4882: Micro Topics in Organizational Behavior, Harvard University.
- Christensen, C. M. & van Bever, D. C. M. (2014). The Capitalist's Dilemma. Harvard Business Review.
- Daw, N. D. et al. (2011). Model-based influences on humans' choices and striatal prediction errors. Neuron.
- Kool, W., Cushman, F. A. & Gershman, S. J. (2016). When does model-based control pay off? PLOS Comp. Bio.
- Kool, W., Gershman, S. J. & Cushman, F. A. (2017). Cost-benefit arbitration between multiple RL systems. Psych. Science.
- Lee, S. W., Shimojo, S. & O'Doherty, J. P. (2014). Neural computations underlying arbitration between model-based and model-free learning. Neuron.
- Sutton, R. S. & Barto, A. G. (2018). Reinforcement Learning: An Introduction (2nd ed.). MIT Press.
Code released under the MIT License (see LICENSE). The paper (paper/) is © the author.