Robustness Model Cascade

Accuracy and robustness of model cascades under data perturbations (CIFAR-10 and CIFAR-100).

This project examines the small to large model cascade behaviour on clean data and under ditribution shift. In this repo we have two-model cascade: a small CNN (MS) acts as a fast first-pass classifier and defers uncertain predictions to a large ResNet-18 (ML). The small model is fine-tuned with a Gatekeeper Loss to become more confident when correct and less confident (deferral-friendly) when wrong.

---

Project Structure

Robustness_Model_Cascade/ 
├── src/
│   ├── models.py              # SmallCNN + ResNet-18 for CIFAR-10
│   ├── models_cifar100.py      # SmallCNN + ResNet-18 for CIFAR-100
│   ├── gatekeeper_loss.py     # Gatekeeper loss (alpha trade-off)
│   ├── evaluate_cifar10.py    # Clean-set evaluation for CIFAR-10
│   └── evaluate_cifar100.py   # Clean-set evaluation for CIFAR-100
├── train_cifar10_wandb_cc.py  # Train MS + ML + Gatekeeper on CIFAR-10
├── train_cifar100_wandb_cc.py # Train MS + ML + Gatekeeper on CIFAR-100
├── robustness_cifar10c_wandb_cc.py   # Robustness eval on CIFAR-10-C (corruptions)
├── robustness_cifar10p_wandb_cc.py   # Robustness eval on CIFAR-10-P (perturbations)
├── robustness_cifar100c_wandb_cc.py  # Robustness eval on CIFAR-100-C (corruptions)
└── robustness_cifar100p_wandb_cc.py  # Robustness eval on CIFAR-100-P (perturbation)

Data

Clean Data can be downloaded from

• CIFAR- 10 and CIFAR-100 https://zenodo.org/records/10089977

Step 1 — Train the Cascade

CIFAR-10

python train_cifar10_wandb_cc.py

CIFAR-100

python train_cifar100_wandb_cc.py

What this does:

• Trains models on clean data i.e. without corruption. And saves the metric result.

Checkpoints saved:

model_s_pretrained.pth
model_l_pretrained.pth
model_s_gk_alpha0.1.pth
model_s_gk_alpha0.3.pth
...

---

Step 2 — Evaluate on Clean Data

Move trained checkpoints into models/:

models/model_s_pretrained.pth
models/model_l_pretrained.pth
models/model_s_gk_alpha0.1.pth
...

CIFAR-10

python src/evaluate_cifar10.py

CIFAR-100

python src/evaluate_cifar100.py

Outputs a table with: s_o, s_d, acc_s, casc_acc, gain_%, defer%, mFP-MS, mFP-Casc.

---

Step 3 — Evaluate on Corrupted Data

Download the corruption benchmarks first:

• CIFAR-10-C: https://zenodo.org/record/2535967 → extract to ./data/CIFAR-10-C/
• CIFAR-100-C: https://zenodo.org/record/3555552 → extract to ./data/CIFAR-100-C/
• CIFAR-10-P / CIFAR-100-P: same Zenodo pages

CIFAR-10-C (19 corruptions × 5 severities)

python robustness_cifar10c_wandb_cc.py --data-dir ./data/CIFAR-10-C

CIFAR-100-C

python robustness_cifar100c_wandb_cc.py --data-dir ./data/CIFAR-100-C

CIFAR-10-P / CIFAR-100-P (perturbation sequences)

python robustness_cifar10p_wandb_cc.py
python robustness_cifar100p_wandb_cc.py