Skip to content

CALM

Jump to bottom
Ivan Svetunkov edited this page Oct 22, 2026 · 7 revisions

CALM — Combine Augmented Linear Models

calm() (R) / CALM() (Python) combines the parameters of regressions of the first variable of the data on subsets of the other variables. The models are weighted by an information criterion. A variable missing from a model has a zero parameter in it, so the combined parameter is shrunk towards zero according to the importance of the variable.

Function signatures

R

calm(data, ic=c("AICc","AIC","BIC","BICc"), bruteforce=FALSE, silent=TRUE,
     formula=NULL, subset=NULL,
     distribution=c("dnorm","dlaplace","ds","dgnorm","dlogis","dt","dalaplace",
                    "dlnorm","dllaplace","dls","dlgnorm","dbcnorm",
                    "dinvgauss","dgamma","dexp",
                    "dfnorm","drectnorm",
                    "dpois","dnbinom",
                    "dbeta","dlogitnorm",
                    "plogis","pnorm"),
     parallel=FALSE, ...)

Python

def CALM(
    data: dict | DataFrame,
    ic: Literal["AICc", "AIC", "BIC", "BICc"] = "AICc",
    bruteforce: bool = False,
    silent: bool = True,
    distribution: str = "dnorm",
    **kwargs,
) -> CALMResult | ALM: ...

How it works

  1. Candidate models.
    • bruteforce=TRUE: all 2^k models of the k regressors, including the intercept-only one. With more variables than observations, the function switches to bruteforce=FALSE with a warning; with more than 14 variables it warns that this takes long.
    • bruteforce=FALSE (default): stepwise selects the best model first. If it contains the intercept only, that model is returned (an alm object in R, an ALM in Python). If it has fewer than 14 parameters, all the subsets of its variables are combined (bruteforce on the selected variables). Otherwise the candidates are the best model, the best model plus each excluded variable, and the best model minus each included variable.
  2. Weights. w_i = exp(-0.5 (IC_i - IC_min)) / Σ_j exp(-0.5 (IC_j - IC_min)).
  3. Combination. The parameters are averaged with these weights, with zeros for the variables a model does not include.
  4. Importance. The importance of a variable is the sum of the weights of the models that include it. The number of parameters of the combined model is the sum of the importances plus one (the scale).

Both languages take the data with the response in the first column, not a list of fitted models.

Parameters

Parameter R Python Default Description
Data data data — Data frame / dict / DataFrame: response first, regressors after. Python drops the rows with missing values and expands non-numeric columns into dummies.
Information criterion ic ic "AICc" "AICc", "AIC", "BIC", "BICc".
Brute force bruteforce bruteforce FALSE / False See How it works.
Silent silent silent TRUE / True FALSE prints the progress.
Formula formula — NULL R only: combine over the variables of the formula, after their transformations.
Subset subset — NULL R only: the observations to use.
Distribution distribution distribution "dnorm" Passed to alm() / ALM. R accepts the 23 in the signature; Python also accepts dbinom, dgeom and dchisq. With "dnorm", the models are estimated by least squares.
Parallel parallel — FALSE R only: fit the models in parallel (foreach and doMC or doParallel).
Other ... **kwargs — Passed to alm() / ALM.

Return value

R returns an object of class c("calm", "alm", "greybox"); Python a CALMResult, or the ALM of stepwise when only the intercept is selected.

Field R Python Description
Parameters coefficients, coef() coefficients Combined parameters, intercept first. Python's coef omits the intercept, which is in intercept_.
Parameter names names(coef()) coefficient_names
Covariance matrix vcov vcov() Combined covariance matrix.
Fitted values fitted fitted
Residuals residuals residuals
Location / scale mu, scale mu, scale
Extra parameter other other_ Combined shape, alpha, lambdaBC, nu or size.
Distribution distribution distribution, distribution_
Loss loss loss_, loss_value Always "likelihood".
Log-likelihood logLik log_lik, loglik Combined log-likelihood.
Information criterion IC, ICType IC, IC_type; aic, aicc, bic, bicc
Degrees of freedom df, df.residual, rank df_model, df_residual, df_residual_, rank_
Importance importance importance
Combination combination combination, combination_col_names, combination_row_names, weights One row per model: the included variables (0/1), the IC weight and the IC. Python also has the weights as a vector.
Data data, call, xlevels data, actuals, formula, x_variables, y_variable
Time elapsed timeElapsed time_elapsed

Methods

Method R Python
Summary summary(model, level=0.95) model.summary(level=0.95)
Prediction predict(model, newdata, interval, level, side) model.predict(X, interval, level, side): a PredictionResult (see ALM)
Confidence intervals confint(model, parm, level=0.95) model.confint(parm, level=0.95)
Covariance matrix vcov(model) model.vcov()
Number of parameters nparam(model) model.nparam, model.n_param
Score — model.score(X, y, metric): "likelihood", "MSE", "MAE", "R2"

The other alm methods (plots, rstandard(), …) apply to R's calm object through its class.

Examples

The data are created in R and read by Python from the same file, so both see identical inputs.

R

library(greybox)
set.seed(42)
x1 <- rnorm(100); x2 <- rnorm(100); x3 <- rnorm(100)
y <- 1 + 2*x1 + 0.5*x2 + 0.2*x3 + rnorm(100)
data <- data.frame(y=y, x1=x1, x2=x2, x3=x3)
write.csv(data, "calm.csv", row.names=FALSE)

result <- calm(data, ic="AICc", bruteforce=TRUE)
summary(result)
# The AICc combined model
# Response variable: y
# Distribution used in the estimation: Normal
# Coefficients:
#             Estimate Std. Error Importance Lower 2.5% Upper 97.5%
# (Intercept)   1.0316     0.0671     1.0000     0.8983      1.1649 *
# x1            2.0500     0.0655     1.0000     1.9200      2.1800 *
# x2            0.5134     0.0746     1.0000     0.3654      0.6615 *
# x3            0.1144     0.0796     0.6797    -0.0435      0.2724
#
# Error standard deviation: 0.8916
# Sample size: 100
# Number of estimated parameters: 4.6796
# Number of degrees of freedom: 95.3204
# Approximate combined information criteria:
#      AIC     AICc      BIC     BICc
# 266.1985 266.7621 278.3898 279.6875

head(result$combination, 3)
#        x1 x2 x3   IC weights      ICs
# Model1  0  0  0 1.167341e-42 456.4178
# Model2  1  0  0 3.649225e-06 288.3521
# Model3  0  1  0 6.041710e-42 453.1299

new_data <- data.frame(x1=c(0, 1, 2), x2=c(0, 0, 0), x3=c(0, 0, 0))
predict(result, newdata=new_data, interval="prediction", level=0.95)
#          Mean Lower bound (2.5%) Upper bound (97.5%)
# [1,] 1.031598         -0.7433639            2.806560
# [2,] 3.081600          1.3022147            4.860985
# [3,] 5.131602          3.3383525            6.924852

Python

import pandas as pd
from greybox import CALM

data = pd.read_csv("calm.csv")
result = CALM(data, ic="AICc", bruteforce=True)
print(result.summary())
# The AICc combined model
# Response variable: y
# Distribution used in the estimation: Normal
# Coefficients:
#                  Estimate   Std. Error   Importance   Lower 2.5%  Upper 97.5%
#  (Intercept)       1.0316       0.0671        1.000       0.8983       1.1649 *
#           x1       2.0500       0.0655        1.000       1.9200       2.1800 *
#           x2       0.5134       0.0746        1.000       0.3654       0.6615 *
#           x3       0.1144       0.0796        0.680      -0.0435       0.2724
#
# Error standard deviation: 0.8916
# Sample size: 100
# Number of estimated parameters: 4.680
# Number of degrees of freedom: 95.320
# Approximate combined information criteria:
#          AIC        AICc         BIC        BICc
#     266.1985    266.7621    278.3898    279.6875

result.importance        # array([1., 1., 1., 0.6797])
result.weights           # IC weight of each of the 8 models

new_data = pd.DataFrame({"x1": [0, 1, 2], "x2": [0, 0, 0], "x3": [0, 0, 0]})
result.predict(new_data, interval="prediction", level=0.95)
#        mean     lower     upper
# 0  1.031598 -0.743364  2.806560
# 1  3.081600  1.302215  4.860985
# 2  5.131602  3.338352  6.924852

predict() takes the new regressors by name (a DataFrame or dict), or a design matrix, e.g. the one from formula() (its (Intercept) column is ignored, as R's model.frame() does):

from greybox import formula

X = formula("y ~ x1 + x2 + x3", new_data.assign(y=0.0), return_type="X")
result.predict(X, interval="prediction", level=0.95)    # the same three rows as above

With the default bruteforce=FALSE, stepwise selects all three regressors here, so both languages return the same combination as above.

References

  • Burnham, K.P. and Anderson, D.R. (2002). Model Selection and Multimodel Inference. Springer.
  • Svetunkov, I. (2023). Statistics for Business Analytics. https://openforecast.org/sba/

Clone this wiki locally