| Title: | Bayesian Additive Regression Trees with Gradient-Sampled Parametric Extensions |
|---|---|
| Description: | Fits semiparametric linear and multilevel models with non-parametric additive Bayesian additive regression tree (BART; Chipman, George, and McCulloch (2010) <doi:10.1214/09-AOAS285>) components and gradient-sampled parametric ones. The parametric conditional is drawn with a hand-derived analytic log-posterior gradient and the WALNUTS sampler (<arXiv:2506.18746>), taken from the Walnutpie library (vendored from <https://github.com/flatironinstitute/walnutpie>, commit f3c1833; MIT licensed, Copyright (c) 2025--2026 by the Walnutpie Developers). Multilevel models can be expressed using 'lme4' syntax (Bates, Maechler, Bolker, and Walker (2015) <doi:10.18637/jss.v067.i01>). |
| Authors: | Vincent Dorie [aut, cre] (ORCID: <https://orcid.org/0000-0002-9576-3064>), Ben Goodrich [ctb] (rstanarm_functions.R), Jonah Gabry [ctb] (rstanarm_functions.R), Imad Ali [ctb] (rstanarm_functions.R), Sam Brilleman [ctb] (rstanarm_functions.R), Paul-Christian Burkner [ctb] (rstanarm_functions.R, ORCID: <https://orcid.org/0000-0001-5765-8995>), Bob Carpenter [ctb] (Walnutpie headers (WALNUTS sampler)), Brian Ward [ctb] (Walnutpie headers (WALNUTS sampler)), Walnutpie Developers [cph] (inst/include/walnutpie.hpp and inst/include/walnutpie/, vendored Walnutpie headers, MIT license), Trustees of Columbia University [cph] (rstanarm_functions.R), Douglas Bates [ctb] (lme4_functions.R, ORCID: <https://orcid.org/0000-0001-8316-9503>), Martin Maechler [ctb] (lme4_functions.R, ORCID: <https://orcid.org/0000-0002-8685-9910>), Ben Bolker [ctb] (lme4_functions.R, ORCID: <https://orcid.org/0000-0002-2127-0443>), Steve Walker [ctb] (lme4_functions.R, ORCID: <https://orcid.org/0000-0002-4394-9078>), Armon Dadgar [ctb] (adaptive radix tree), Bothner Per [ctb] (config.guess), Elliston Ben [ctb] (config.guess), Free Software Foundation [cph] (config.sub), Guido U Draheim [ctb] (ax_check_compile_flag.m4), Maarten Bosmans [ctb] (ax_check_compile_flag.m4), Christophe Tournayre [ctb] (ax_ext.m4), Michael Petch [ctb] (ax_ext.m4, ax_gcc_x86_avx_xgetbv.m4, ax_gcc_x86_cpuid.m4), Rafael de Lucena Valle [ctb] (ax_ext.m4), Steven G. Johnson [ctb] (ax_gcc_x86_cpuid.m4, ORCID: <https://orcid.org/0000-0001-7327-4967>), Matteo Frigo [ctb] (ax_gcc_x86_cpuid.m4), Scott Pakin [ctb] (ax_func_posix_memalign.m4, ORCID: <https://orcid.org/0000-0002-5220-1985>) |
| Maintainer: | Vincent Dorie <[email protected]> |
| License: | GPL (>= 3) |
| Version: | 0.0-14 |
| Built: | 2026-11-11 18:56:30 UTC |
| Source: | https://github.com/vdorie/stan4bart |
Jointly fits Bayesian additive regression tree (BART) models to two or more
continuous outcomes that share a predictor set but have correlated residuals.
Each outcome gets its own “sum-of-trees” mean surface; the surfaces are
coupled through a residual covariance matrix that is given a
conjugate inverse-Wishart prior and sampled jointly. This is the
seemingly-unrelated regressions (SUR) analogue of BART: modeling the
cross-outcome residual covariance yields a better-calibrated joint predictive
distribution than fitting an independent BART per outcome whenever the outcomes
are genuinely correlated.
For outcomes fits:
where each is a BART model and is the
residual covariance.
mvbart(formula, data = NULL, subset, na.action = getOption("na.action", "na.omit"), test = NULL, n.samples = 1000L, n.burn = 1000L, n.chains = 4L, n.trees = 75L, prior = NULL, seed = NA_integer_, verbose = FALSE, bart_args = list())mvbart(formula, data = NULL, subset, na.action = getOption("na.action", "na.omit"), test = NULL, n.samples = 1000L, n.burn = 1000L, n.chains = 4L, n.trees = 75L, prior = NULL, seed = NA_integer_, verbose = FALSE, bart_args = list())
formula |
a |
data |
an optional data frame containing the variables in the formula. |
subset, na.action
|
optional components adjusting the model frame. See
|
test |
an optional data frame of held-out predictors. When present, the
per-outcome forest predictions |
n.samples |
positive integer; number of retained posterior draws per chain. |
n.burn |
non-negative integer; number of burn-in sweeps discarded per chain. |
n.chains |
positive integer; number of independent chains, run sequentially. |
n.trees |
positive integer; number of trees in each outcome's forest. |
prior |
an optional named list specifying the inverse-Wishart prior on
|
seed |
optional integer. If supplied, per-chain seeds are drawn deterministically from it and results are reproducible; the caller's RNG state is restored afterwards. |
verbose |
logical or integer; if a positive number, prints progress every
|
bart_args |
an optional named list of further arguments forwarded to each
outcome's BART sampler. Recognized |
Conditional on the other equations' current residuals, updating one outcome's
forest is an ordinary univariate BART step: the working outcome is Gaussian
with a per-observation offset (the conditional mean implied by )
and a scalar conditional variance. These are injected into the outcome's
dbarts sampler through its setOffset and
setSigma methods; the sampler is created with a fixed residual
prior so the engine never draws its own nugget. After the forest sweep,
is drawn from its inverse-Wishart full conditional,
, where is the matrix of structural residuals.
The routine is built entirely on the public dbarts R interface and adds
no compiled code.
Because the response passed to each sampler is fixed (only the offset moves),
the creation-time response rescaling used internally by
BART stays anchored across sweeps. The returned mMax records, per
retained draw, the largest per-sweep ; values well below 1 indicate the fixed scale is
well anchored. If the maximum drift exceeds 1 a warning is issued: for such
heavy-tailed cases the recommended fallback is a longer warmup, or a
warmup-only response rescale (re-anchoring the internal scale to include the
offset range during burn-in only).
When all outcomes share the same predictors, BART's flexibility introduces a
small positive bias in the recovered residual correlation (on the order of
to at moderate to high correlation), because
finite-sample mean-fitting error is correlated across equations. This makes
interval coverage of the true correlation mildly liberal. The bias shrinks
toward zero as grows and is inherent to the nonparametric mean model,
not a defect of the sampler.
mvbart is a pure-SUR sampler with no random-effect component.
Combining the cross-outcome residual covariance modeled here with
stan4bart's multilevel machinery (SUR random
effects, i.e. a multivariate mixed model) is a natural extension that is not
yet implemented.
Returns a list assigned class mvbartFit with components:
Sigma |
a |
train |
an |
test |
when |
mMax |
a |
y |
the response matrix. |
outcome.names |
character vector of outcome names. |
n.samples, n.burn, n.chains
|
sampler settings. |
prior |
the inverse-Wishart prior actually used, a list with
|
mixing |
named vector of the split rank-normalized R-hats (Vehtari
et al., 2021) of each outcome's residual standard deviation, “y sd”,
and of its in-sample fit, “y fit”, the draw-by-draw root mean
squared difference between the forest's fit and the response. A single chain
is checked by comparing its halves. |
call, formula, terms
|
the matched call, formula, and model terms. |
Vincent Dorie: [email protected].
# two correlated outcomes sharing three predictors set.seed(99) n <- 200L x <- matrix(runif(n * 3L), n, 3L) colnames(x) <- paste0("x", 1:3) B <- matrix(c(1.5, -1.0, 0.7, -0.6, 1.1, -0.9), 3L, 2L) rho <- 0.7; s1 <- 1.0; s2 <- 1.3 Sigma <- matrix(c(s1^2, rho * s1 * s2, rho * s1 * s2, s2^2), 2L, 2L) Y <- x %*% B + matrix(rnorm(n * 2L), n, 2L) %*% chol(Sigma) df <- data.frame(y1 = Y[, 1L], y2 = Y[, 2L], x) fit <- mvbart(cbind(y1, y2) ~ x1 + x2 + x3, data = df, n.samples = 200L, n.burn = 100L, n.chains = 1L, n.trees = 50L, seed = 1L) # posterior of the residual correlation Sig <- fit$Sigma rho.draws <- Sig[1L, 2L, , ] / sqrt(Sig[1L, 1L, , ] * Sig[2L, 2L, , ]) c(mean = mean(rho.draws), quantile(rho.draws, c(0.025, 0.975)))# two correlated outcomes sharing three predictors set.seed(99) n <- 200L x <- matrix(runif(n * 3L), n, 3L) colnames(x) <- paste0("x", 1:3) B <- matrix(c(1.5, -1.0, 0.7, -0.6, 1.1, -0.9), 3L, 2L) rho <- 0.7; s1 <- 1.0; s2 <- 1.3 Sigma <- matrix(c(s1^2, rho * s1 * s2, rho * s1 * s2, s2^2), 2L, 2L) Y <- x %*% B + matrix(rnorm(n * 2L), n, 2L) %*% chol(Sigma) df <- data.frame(y1 = Y[, 1L], y2 = Y[, 2L], x) fit <- mvbart(cbind(y1, y2) ~ x1 + x2 + x3, data = df, n.samples = 200L, n.burn = 100L, n.chains = 1L, n.trees = 50L, seed = 1L) # posterior of the residual correlation Sig <- fit$Sigma rho.draws <- Sig[1L, 2L, , ] / sqrt(Sig[1L, 1L, , ] * Sig[2L, 2L, , ]) c(mean = mean(rho.draws), quantile(rho.draws, c(0.025, 0.975)))
Print a brief description of a fitted model, or compute and print a short summary of it. Both check the sampling-phase leapfrog-step diagnostic and warn when it suggests the model was under-warmed.
## S3 method for class 'stan4bartFit' print(x, ...) ## S3 method for class 'stan4bartFit' summary(object, ...) ## S3 method for class 'summary.stan4bartFit' print(x, ...)## S3 method for class 'stan4bartFit' print(x, ...) ## S3 method for class 'stan4bartFit' summary(object, ...) ## S3 method for class 'summary.stan4bartFit' print(x, ...)
x, object
|
a fitted model resulting from a call to
|
... |
unused, but provided to match the signatures of other generics. |
Both print and summary examine
object$adaptation$mean_leapfrog, the mean number of leapfrog
(gradient evaluation) steps per transition during the sampling phase, per
chain (see stan4bart). When the largest per-chain value
exceeds 30, a one-line message is issued: “parametric sampler tuning
looks poor (mean leapfrog steps per transition = ...); consider increasing
warmup.” A well-tuned fit - one run with at least the default warmup, or
with warmup at or above roughly 100 for large fits with many
random-effect levels - should fall well under this value; the message is a
cue to lengthen warmup rather than an error. No message is issued for
a fit with no adaptation record, e.g. one run with warmup = 0.
summary additionally returns an object of class
summary.stan4bartFit, printed by print.summary.stan4bartFit,
giving the formula, family, chain/sample counts, the per-chain warmup
and sampling mean leapfrog counts, and the fit's R-hats, which are the
numbers of the mixing check (see “Mixing check” in
stan4bart): one compact line such as
R-hat: sigma 1.00, g sd 1.01, fit 1.00, each value as it is, with no
star, bar or note, and NA: could not be checked, with the reason
when a computation failed, when a value is NA. print is
unchanged.
R-hat compares each chain's spread with the spread of the chains pooled, so
a value above 1 says the chains disagree, not that pooling them is wrong. At
the defaults BART's chains routinely disagree on sigma, on the
standard deviations of the random effects, on the in-sample error and on a
single row's prediction, so values above 1 there are expected, and no
single bar says whether a fit is usable. Averaged estimands, such as the
mean prediction over a subgroup or a contrast between two subgroups, mix far
better, so read R-hat on the estimand you report. extract with
combine_chains = FALSE gives each chain's draws of the expected
value; the example computes R-hat of a contrast of subgroup means with them
and dbarts::rhat. Call both with the dbarts:: prefix, as
other packages export functions of the same names (see
rhat).
print.stan4bartFit and print.summary.stan4bartFit return
their argument, invisibly.
summary.stan4bartFit returns a list of class
summary.stan4bartFit, printed by print.summary.stan4bartFit.
Vincent Dorie: [email protected].
# R-hat of the estimand reported: the difference between the mean expected # value of two subgroups, one value per draw and chain set.seed(1) n <- 200L df <- data.frame(g = sample(10L, n, replace = TRUE), x1 = rnorm(n), xb = runif(n)) df$y <- 3 * sin(pi * df$xb) + df$x1 + rnorm(df$g[1:10])[df$g] + rnorm(n) fit <- stan4bart(y ~ x1 + bart(xb) + (1 | g), df, chains = 2L, iter = 300L, warmup = 150L, verbose = -1, bart_args = list(n.trees = 25L)) summary(fit) # observations by samples by chains ev <- dbarts::extract(fit, "ev", combine_chains = FALSE) group <- df$xb > 0.5 contrast <- apply(ev[group, , , drop = FALSE], 2:3, mean) - apply(ev[!group, , , drop = FALSE], 2:3, mean) dbarts::rhat(contrast) # rhat() takes iterations by chains# R-hat of the estimand reported: the difference between the mean expected # value of two subgroups, one value per draw and chain set.seed(1) n <- 200L df <- data.frame(g = sample(10L, n, replace = TRUE), x1 = rnorm(n), xb = runif(n)) df$y <- 3 * sin(pi * df$xb) + df$x1 + rnorm(df$g[1:10])[df$g] + rnorm(n) fit <- stan4bart(y ~ x1 + bart(xb) + (1 | g), df, chains = 2L, iter = 300L, warmup = 150L, verbose = -1, bart_args = list(n.trees = 25L)) summary(fit) # observations by samples by chains ev <- dbarts::extract(fit, "ev", combine_chains = FALSE) group <- df$xb > 0.5 contrast <- apply(ev[group, , , drop = FALSE], 2:3, mean) - apply(ev[!group, , , drop = FALSE], 2:3, mean) dbarts::rhat(contrast) # rhat() takes iterations by chains
This function fits semi-parametric linear and probit models that have a
non-parametric, BART component and one or more of a parametric fixed
effect (unmodeled coefficients), or a parametric random effect
(modeled coefficients). If
is a BART “sum-of-trees” model, fits:
For continuous response variables:
For binary response variables:
stan4bart(formula, data = NULL, subset, weights, na.action = getOption("na.action", "na.omit"), offset, contrasts = NULL, test = NULL, treatment = NULL, offset_test = NULL, verbose = FALSE, iter = 2000L, warmup = iter %/% 2L, skip = 1L, chains = 4L, cores = getOption("mc.cores", 1L), refresh = max(iter %/% 10L, 1L), offset_type = c("default", "fixef", "ranef", "bart", "parametric"), seed = NA_integer_, keep_fits = TRUE, save_warmup = FALSE, store = c("fits", "trees"), callback = NULL, stan_args = NULL, bart_args = NULL, bart_range = c("mixed", "lm", "response"), prior = normal(0, 2.5, autoscale = TRUE), prior_aux = exponential(autoscale = TRUE), prior_covariance = decov(1, 1, 1, 1), QR = FALSE, adapt_delta = 0.8, init_r = 2)stan4bart(formula, data = NULL, subset, weights, na.action = getOption("na.action", "na.omit"), offset, contrasts = NULL, test = NULL, treatment = NULL, offset_test = NULL, verbose = FALSE, iter = 2000L, warmup = iter %/% 2L, skip = 1L, chains = 4L, cores = getOption("mc.cores", 1L), refresh = max(iter %/% 10L, 1L), offset_type = c("default", "fixef", "ranef", "bart", "parametric"), seed = NA_integer_, keep_fits = TRUE, save_warmup = FALSE, store = c("fits", "trees"), callback = NULL, stan_args = NULL, bart_args = NULL, bart_range = c("mixed", "lm", "response"), prior = normal(0, 2.5, autoscale = TRUE), prior_aux = exponential(autoscale = TRUE), prior_covariance = decov(1, 1, 1, 1), QR = FALSE, adapt_delta = 0.8, init_r = 2)
formula |
a |
data |
an optional data frame containing the variables in the formula. Its use is strongly encouraged. |
subset, weights, na.action, offset, contrasts
|
optional components
adjusting the constructed model matrices and otherwise changing the linear
predictor. |
test |
an optional data frame to be used as test data. If present, the test predictions will be stored as the sampler runs and can be extracted later. |
treatment |
an optional symbol, that when present and refers to a binary
variable, will be used to create a test data frame with the treatment variable
set to its counterfactual. Only one of |
offset_test |
optional vector which will be added to the test predictions. |
verbose |
a logical or integer. If |
iter |
positive integer indicating the number of posterior samples to draw and return. Includes warmup samples. |
warmup |
non-negative integer indicating number of posterior samples to
draw and throw away. Also controls the number of iterations for which
adaptation is run. The default (half of |
skip |
one or two positive integers. Every |
chains |
positive integer giving the number of Markov Chains to sample. |
cores |
positive integer giving the number of units of parallelization. Computation for each chain will be divide among the cores available. When greater than one, verbose output within chains will not be available. |
refresh |
positive integer giving the frequency with which diagnostic
information should be printed as the sampler runs. Only applies with
|
offset_type |
character; an experimental/testing feature that controls
how |
seed |
Optional integer specifying the desired pRNG seed.
It should not be needed when running single-threaded - calling
|
keep_fits |
Logical that, when false, prevents the sampler from
storing each draw. Intended to be used with |
save_warmup |
Logical; when |
store |
character; how the fitted BART component is retained.
|
callback |
A function that will be called at each iteration, accepting
three arguments: |
stan_args |
deprecated; its entries are now arguments of
|
bart_args |
optional list, specifying further arguments to BART. See details below. |
prior |
the prior on the fixed effect coefficients, as in
|
prior_aux |
the prior on the residual standard deviation (continuous
responses only): |
prior_covariance |
the prior on the random effect covariance matrices:
|
QR |
logical, default |
adapt_delta |
the acceptance rate the parametric sampler's step size is
tuned to, a value in |
init_r |
half-width of the interval, centered at zero, that the
parametric sampler's initial position is drawn from uniformly, on the
unconstrained scale; at least zero, default |
bart_range |
for a continuous response, the range of values the BART
component is calibrated to: a pair of numbers
Under Where a column of the model's own takes one value in each category of an
unordered BART factor, as a region's median income beside region inside
A BART factor with many categories makes the initial fit slow and
memory-hungry: its indicators are a dense block of observations by
categories. A pair or Too narrow a range leaves group-level signal in the random effects that belongs to BART, and too wide a one costs accuracy and mixing in small samples. The linear terms that stand in for BART in the initial fit cannot follow a strongly curved group-level effect, which is a case for giving a pair. A binary response has no range, and a value other than the default is ignored with a warning. |
Fits a Bayesian “mixed effect” model with a non-parametric Bayesian Additive Regression Trees (BART) component. For continuous responses:
where are the “random effects” - random intercepts and
slopes - that correspond to group , is a mapping from
individual to its group index, - a BART sum-of-trees model,
are predictors used in the BART model, are predictors
in a parametric, linear “fixed effect” component, is the
design matrix for the random intercept and slopes, and and
are variance components.
Binary outcome models are obtained by assuming a latent variable that has the above distribution, and that the observed response is 1 when that variable is positive and 0 otherwise. The response variable marginally has the distribution:
where is the cumulative distribution function of the standard
normal distribution.
As stan4bart fits a Bayesian model, essentially all components are
“modeled”. Furthermore, as it has two first-level, non-hierarchical
components, “fixed” effects are ambiguous. Thus we adopt:
“fixed” - refers only to the parametric, linear, individual
level mean component, ; these are “unmodeled
coefficients” in other contexts
“random” - refers only to the parametric, linear,
hierarchical mean component, ; these are “modeled
coefficients” in other contexts
“bart” - refers only to the nonparametric, individual level
mean component,
Model specification occurs in the formula object, according to the
following rules:
variables or terms specified inside a pseudo-call to bart
are used for the “bart” component, e.g. y ~ bart(x_1 + x_2)
variables or terms specified according to lmer
syntax are used for the “random” effect component, e.g.
y ~ (1 | g_1) + (1 + x_3 | g_1)
remaining variables not inside a bart or “bars”
construct are used for the “fixed” effect component; e.g.
y ~ x_4
All three components can be present in a single model, however are
bart part must present. If you wish to fit a model without one, use
stan_glmer in the rstanarm package instead.
Factor variables are handled differently in each component. In the
fixed and random components they use the usual
model.matrix contrasts and lme4 grouping-factor
machinery, respectively. In the bart component a factor is encoded
as dbarts's categorical splits: a single column whose tree
prior chooses level subsets to split on, rather than one indicator column
per level. Consequently extract(type = "varcount") reports one row
per factor, named by the factor's variable name, not one row per level.
A factor level in newdata (to predict) or test that
was not observed when fitting is an error for a bart factor; this
is unlike the random component, where an unseen grouping level is
handled per sample_new_levels in extract.
The parametric part of the model is configured by prior,
prior_aux, prior_covariance, QR, adapt_delta
and init_r, which follow rstanarm's names and meanings, and
the BART part by bart_args. stan4bart no longer uses Stan:
the parametric component is sampled by a gradient-based WALNUTS sampler
driven by a hand-derived analytic log-posterior gradient. The prior
constructors (normal, student_t, cauchy,
exponential, decov) are written inside the argument that
takes them; see the help page for stan_glmer in the rstanarm
package for what each means. There is no prior on an intercept: BART
carries it. bart_args is similar to
bart, but not identical, as stan4bart
constructs its own model internally.
Two settings of the sampler are for measurement and debugging and are not
arguments. The option stan4bart.ridge_move (default TRUE)
switches the exact slice move that draws the random-effect scale once per
sweep along the curve that rescales a grouping factor's standard
deviation; the option stan4bart.save_raw_parameters (default
FALSE), when TRUE, keeps the raw unconstrained rows of the
stored parametric draw and its placeholder diagnostic rows.
Until 0.0-13 these entries were given as one list, stan_args. It is
deprecated and used as the arguments: each entry named prior,
prior_aux, prior_covariance, QR,
adapt_delta or init_r goes to the argument of that name, with
one warning a session naming them. prior_intercept,
adapt_gamma, adapt_kappa, adapt_t0,
adapt_init_buffer, adapt_term_buffer, adapt_window,
stepsize, stepsize_jitter and max_treedepth
configured Stan and have no analog under WALNUTS; they are accepted so that
existing scripts do not error, are ignored, and a warning names each one
supplied. Any other name is an error that names the accepted ones.
BART arguments include:
further arguments to dbartsControl that are not
specified by stan4bart, such as keepTrees; keeping trees can be costly in terms of memory, but is
required to use predict
n.trees, the number of trees, and n.cuts and
useQuantiles, which place the cut points; each is a setting of
the model or the data and not of the control
the prior arguments of dbartsSpec -
leaf.prior and the tree prior's power, base and
split.probs. These are expressions,
not values: dbarts resolves them in its own prior vocabulary
(see dbartsPriors), so a bare
leaf.prior = normal(k = chi(1.25, Inf)) works whether or not
dbarts is attached. They must be written literally in the
stan4bart call, since bart_args is read off it
unevaluated; a prior assembled beforehand has to be built with
dbarts::dbartsPriors$... instead
the remaining model-level arguments of
dbartsSpec - proposal.probs,
monotone, and seed. Unlike the priors these are ordinary
values, evaluated in the caller's frame like any other argument
interactions and blocks, dbarts's forest vocabulary
(see dbartsForests). Written as a call inside
bart_args, such as interactions(max.order = 1), they
resolve to the dbarts constructor whether or not dbarts is
attached, and are evaluated in the caller's own frame, so a local
variable referenced inside the call is found, also where bart_args
is passed on through a wrapper's .... A specification
assembled beforehand has to be built with
dbarts::dbartsForests$... instead
k, shorthand for leaf.prior = normal(k = k). It is a
value too, not an expression, so it does not see dbarts's
prior-evaluation environment; write the prior out in full when that
matters. Supplying both k and leaf.prior is an error
sigma, resid.prior, resid.dist, and variance
are not accepted: the parametric component draws the residual
standard deviation and the BART component conditions on it, so the forest
holds its own error scale fixed and has no residual model to configure.
Neither is family, beyond naming the one the response already
implies ("gaussian" or "probit") - the response family
follows the fitted data, and the remaining dbarts families are not
wired into the two-block sampler.
Before sampling, the model is fitted once without its BART component: for
a continuous response whose range is estimated, "mixed" or
"lm", to fix the range BART is calibrated to (see
bart_range) and the values the first sweep starts from, and for a
binary response to give the first sweep the values it starts from. Where
the model has random terms this is a mixed model, fitted by lme4
when it is installed: that of lmer for a continuous
response and of glmer for a binary one.
lme4 is a suggested package. Without it, a continuous model with random terms takes the response's own range and a binary one starts from a probit regression on its fixed effects alone, each with a warning. The fit is then not the one obtained with lme4 installed, including under a fixed seed; results can also differ between versions of lme4. What that costs was measured on simulated data of 500 rows with a random intercept on groups of 4 or of 25, ten data sets each, fitted with four chains of 1000 warmup and 1000 kept draws. Without lme4 a continuous response was given a range 1.3 to 1.4 times as wide. The error of its fitted mean was 1.5 percent higher with groups of 4 and 0.5 percent higher with groups of 25, where a second seed with lme4 moved it by 1.0 and 0.7 percent, and the fitted mean's effective sample size was 20 and 13 percent lower, where a second seed moved it by 8 and 1 percent. The error of a binary response's fitted probabilities moved by 0.7 percent or less, and by 0.4 percent or less at a second seed with lme4.
A model with no random term never uses lme4. Where lme4's fit stops, or
returns estimates that are not finite, a warning says so and the fit
proceeds as it does without lme4, a binary one from a start of zero. The
initial fits were checked against lmer and glmer of lme4
2.1.0 and 1.1-35.5.
The random effect covariance is sampled in its non-centered form: a scale and a block of standardized effects whose product is the effect itself. When the group effects are well identified - many observations per group - the log scale and the standardized effects are constrained to a product ridge, a direction the sampler's diagonal metric cannot rescale. Every other direction of the parametric block mixes freely; that one does not, and the random effect standard deviations inherit its autocorrelation.
The remedy is the "stan" element of skip: it takes that many
parametric transitions per BART sweep and keeps the last, at that multiple
of the parametric block's cost and no change to the number of draws
returned. On a 20-group, 100-observations-per-group Gaussian random
intercept fit, skip = c(bart = 1, stan = 8) takes the scale's lag-1
autocorrelation from about 0.96 to about 0.77 and stan = 16 to about
0.63, at 1.4x and 2.0x the wall time. How far it has to go tracks how
strongly the groups are identified, so read the autocorrelation of the fit
in hand rather than carrying a number over. The wall-time cost is real - a
parametric transition is roughly a third of a BART sweep - so raise it when
the scale is the quantity of interest, not by default.
Behavior differs when running multi- and single-threaded, as the pseudo random
number generators (pRNG) used by R are not thread safe. When single-threaded,
R's built-in generator is used; if set at the start, .Random.seed
will be used and its value updated as samples are drawn. When multi-threaded,
the default behavior is draw new random seeds for each thread using the clock
and use thread-specific pRNGs.
This behavior can be modified by setting seed. For the single-threaded
case, that seed will be installed and the existing seed replaced at the end,
if applicable. For multi-threaded runs, the seeds for threads are drawn
sequentially using the supplied seed, and will not change the state of
R's built-in generator.
Consequently, the seed argument should not be needed when running
single-threaded - set.seed will suffice. When multi-threaded,
seed can be used to obtain reproducible results.
Callbacks can be used to avoid expensive memory allocation, aggregating results as the sampler proceeds instead of waiting until the end. A callback funtion must accept the arguments:
yhat.train - BART predictions of the expected value of the
training data, conditioned on the Stan parameters
yhat.test - when applicable, the same values as above but
for the test data; NULL otherwise
stan_pars - named vector of parametric-component samples,
including fixed and random effects and variance parameters
It is expected that the callback will return a vector of the same length each time. If not, invalid memory writes will likely result. If the result of the callback has names, they will be added to the result.
At present, stan4bart models cannot be safely saved
and loaded in a way that the sampler can be restarted. This
feature may be added in the future.
After each fit, the split rank-normalized R-hat (Vehtari et al., 2021, as
posterior::rhat) is computed for sigma (not for a binary
response), for the standard deviations of each grouping factor's random
effects, and for fit: each draw's in-sample root mean squared
difference between the expected value and the response (on the probability
scale for a binary response). Fixed effects, individual group effects and
trees are not checked. No warning is issued; summary prints all of
them on one line, as values with no flag (see
summary.stan4bartFit for how to read them). A single chain
is checked by comparing its halves.
Only the saved draws are read, never warmup, and a quantity that cannot be
compared, such as one with fewer than four draws per chain, is not
checked. fit is summed in blocks of rows, as fitted streams,
so its memory stays bounded; if computing it fails, as for want of memory,
it is left unchecked (NA, with the reason in summary) and the
fit is returned.
Returns a list assigned class stan4bartFit. Has components below, some
of which will be NULL if not applicable.
Input values:
y |
response vector |
weights |
weights vector or null |
offset |
offset vector or null |
frame |
joint model frame for all components |
formula |
formula used to specify the model |
na.action |
supplied na.action |
call |
original call |
Stored data:
bartData |
|
X |
fixed effect design matrix or NULL |
X_means |
column means of fixed effect design matrix when appropriate |
reTrms |
random effect “terms” object when applicable, as
used by |
test |
named list when applicable, having components |
treatment |
treatment vector, when applicable |
Results, better accessed using extract:
bart_train |
samples of individual posterior predictions for BART
component; absent (recomputed from the kept trees on access) when
|
bart_test |
predicted test values for BART component, when
applicable; likewise absent under |
bart_varcount |
BART variable counts |
sigma |
samples of residual standard error; not present for binary outcomes |
k |
samples of the leaf sensitivity parameter; only present when it is modeled |
ranef |
samples of random effects, or modeled coefficients; will be a named list, with effects for each grouping factor |
Sigma |
samples of covariance of random effects; also a named list with one element for each grouping factor |
fixef |
samples of the fixed effects, or unmodeled coefficients |
callback |
samples returned by an optional callback function |
Other items:
mixing |
named vector of the split R-hats of the checked
quantities: |
warmup |
a list of warmup samples, containing the same objects in
the results subsection; present only when |
adaptation |
present when warmup was run and |
range.bart |
for a continuous response, the pair
|
range.route |
how that range was asked for: |
range.fallback |
present only where the range is the response's
own although another was asked for: |
range.info |
a list describing the initial fit behind the range:
the number of observations |
sampler.bart |
present when |
state.bart |
present when |
bart_env |
present when |
Vincent Dorie: [email protected].
bart, lmer, and stan_glmer in the
rstanarm package
# simulate data (extension of Friedman MARS paper) # x consists of 10 variables, only first 5 matter # x_4 is linear f <- function(x) 10 * sin(pi * x[,1] * x[,2]) + 20 * (x[,3] - 0.5)^2 + 10 * x[,4] + 5 * x[,5] set.seed(99) sigma <- 1.0 n <- 100 n.g.1 <- 5L n.g.2 <- 8L # sample observation level covariates and calculate marginal mean x <- matrix(runif(n * 10), n, 10) mu.bart <- f(x) - 10 * x[,4] mu.fixef <- 10 * x[,4] # varying intercepts and slopes for first grouping factor g.1 <- sample(n.g.1, n, replace = TRUE) Sigma.b.1 <- matrix(c(1.5^2, .2, .2, 1^2), 2) b.1 <- matrix(rnorm(2 * n.g.1), n.g.1) %*% chol(Sigma.b.1) # varying intercepts for second grouping factor g.2 <- sample(n.g.2, n, replace = TRUE) Sigma.b.2 <- as.matrix(1.2) b.2 <- rnorm(n.g.2, 0, sqrt(Sigma.b.2)) mu.ranef <- b.1[g.1,1] + x[,4] * b.1[g.1,2] + b.2[g.2] y <- mu.bart + mu.fixef + mu.ranef + rnorm(n, 0, sigma) df <- data.frame(y, x, g.1, g.2) fit <- stan4bart( formula = y ~ X4 + # linear component ("fixef") (1 + X4 | g.1) + (1 | g.2) + # multilevel ("ranef") bart(. - g.1 - g.2 - X4), # use bart for other variables verbose = -1, # suppress ALL output # low numbers for illustration data = df, chains = 1, iter = 10, bart_args = list(n.trees = 5)) # posterior means of individual expected values y.hat <- fitted(fit) # posterior means of the random effects ranef.hat <- fitted(fit, type = "ranef")# simulate data (extension of Friedman MARS paper) # x consists of 10 variables, only first 5 matter # x_4 is linear f <- function(x) 10 * sin(pi * x[,1] * x[,2]) + 20 * (x[,3] - 0.5)^2 + 10 * x[,4] + 5 * x[,5] set.seed(99) sigma <- 1.0 n <- 100 n.g.1 <- 5L n.g.2 <- 8L # sample observation level covariates and calculate marginal mean x <- matrix(runif(n * 10), n, 10) mu.bart <- f(x) - 10 * x[,4] mu.fixef <- 10 * x[,4] # varying intercepts and slopes for first grouping factor g.1 <- sample(n.g.1, n, replace = TRUE) Sigma.b.1 <- matrix(c(1.5^2, .2, .2, 1^2), 2) b.1 <- matrix(rnorm(2 * n.g.1), n.g.1) %*% chol(Sigma.b.1) # varying intercepts for second grouping factor g.2 <- sample(n.g.2, n, replace = TRUE) Sigma.b.2 <- as.matrix(1.2) b.2 <- rnorm(n.g.2, 0, sqrt(Sigma.b.2)) mu.ranef <- b.1[g.1,1] + x[,4] * b.1[g.1,2] + b.2[g.2] y <- mu.bart + mu.fixef + mu.ranef + rnorm(n, 0, sigma) df <- data.frame(y, x, g.1, g.2) fit <- stan4bart( formula = y ~ X4 + # linear component ("fixef") (1 + X4 | g.1) + (1 | g.2) + # multilevel ("ranef") bart(. - g.1 - g.2 - X4), # use bart for other variables verbose = -1, # suppress ALL output # low numbers for illustration data = df, chains = 1, iter = 10, bart_args = list(n.trees = 5)) # posterior means of individual expected values y.hat <- fitted(fit) # posterior means of the random effects ranef.hat <- fitted(fit, type = "ranef")
Commonly expected utility functions to derive useful quantities from fitted models.
## S3 method for class 'stan4bartFit' extract( object, type = c("ev", "ppd", "fixef", "indiv.fixef", "ranef", "indiv.ranef", "indiv.bart", "sigma", "Sigma", "k", "varcount", "stan", "trees", "callback"), sample = c("train", "test"), combine_chains = TRUE, sample_new_levels = TRUE, include_warmup = FALSE, ...) ## S3 method for class 'stan4bartFit' fitted( object, type = c("ev", "ppd", "fixef", "indiv.fixef", "ranef", "indiv.ranef", "indiv.bart", "sigma", "Sigma", "k", "varcount", "stan", "callback"), sample = c("train", "test"), sample_new_levels = TRUE, ...) ## S3 method for class 'stan4bartFit' residuals(object, ...) ## S3 method for class 'stan4bartFit' predict( object, newdata, type = c("ev", "ppd", "indiv.fixef", "indiv.ranef", "indiv.bart"), combine_chains = TRUE, sample_new_levels = TRUE, offset, ...)## S3 method for class 'stan4bartFit' extract( object, type = c("ev", "ppd", "fixef", "indiv.fixef", "ranef", "indiv.ranef", "indiv.bart", "sigma", "Sigma", "k", "varcount", "stan", "trees", "callback"), sample = c("train", "test"), combine_chains = TRUE, sample_new_levels = TRUE, include_warmup = FALSE, ...) ## S3 method for class 'stan4bartFit' fitted( object, type = c("ev", "ppd", "fixef", "indiv.fixef", "ranef", "indiv.ranef", "indiv.bart", "sigma", "Sigma", "k", "varcount", "stan", "callback"), sample = c("train", "test"), sample_new_levels = TRUE, ...) ## S3 method for class 'stan4bartFit' residuals(object, ...) ## S3 method for class 'stan4bartFit' predict( object, newdata, type = c("ev", "ppd", "indiv.fixef", "indiv.ranef", "indiv.bart"), combine_chains = TRUE, sample_new_levels = TRUE, offset, ...)
object |
a fitted model resulting from a call to
|
type |
a character vector; one of the options listed below. |
sample |
one of |
combine_chains |
logical controlling if chain information should be discarded and the result returned as a matrix instead of an array. |
sample_new_levels |
logical; if |
include_warmup |
logical or |
newdata |
data frame for making out of sample predictions. |
offset |
optional vector which will be added to test predictors. |
... |
not currently in use, but provided to match signatures of other generics. |
extract is used to obtain raw samples using the training or test data,
fitted averages those samples, and predict operates on data
not available at the time of fitting. Note: predict requires that the
model be fit with bart_args = list(keepTrees = TRUE).
The type argument accepts:
"ev" - the individual level expected value, that is draws
from where the expectation is with respect to the posterior
distribution of the parameters given the data
"ppd" - draws from the individual level posterior predictive
distribution, generally speaking adding noise to the result for
"ev" or simulating new Bernoulli trials. The mean of these draws
estimates the same quantity as the mean of the "ev" draws (times
the weights, for a weighted binomial model), so if only the
posterior-predictive mean is needed, fitted(type = "ev") computes
it exactly and faster; fitted(type = "ppd") notes this once per
session. fitted(type = "ppd") averages freshly drawn samples, so
under a fixed seed it reproduces an average of
extract(type = "ppd") bitwise.
"fixef" - draws from the posterior of the fixed effects
(also known as the “unmodeled” coefficients),
"indiv.fixef" - draws from the posterior distribution of the
individual level mean component deriving from the fixed effects,
"ranef" - the random effects, varying intercepts and slopes,
or “modeled” coefficients, ; has substantial
structure that is represented as the returned value, where coefficients
are reported within their grouping factors
"indiv.ranef" - individual level mean component deriving
from the random effects,
"indiv.bart" - individual level mean component deriving
from the BART model,
"sigma" - for continuous responses, the residual standard
error
"Sigma" - when applicable, the covariance matrices of the
random effects
"stan" - raw matrix or array of the parametric component's
sampled transformed parameters (named "stan" for historical
reasons; the parametric component is now sampled with a gradient-based
WALNUTS sampler rather than Stan's NUTS).
"trees" - a data frame of flatted trees; see the subsection
on extracted trees in bart and note that stan4bart variable
names can be found in the bartData@x element of a fitted stan4bart
model. The chain column is present only for a fit with more than
one chain. For a continuous response a leaf's value is in the
one internal unit every chain shares, set by the fit's
range.bart: at an observation, the values of the leaves it falls
in, summed over a draw's trees, times the width of that range, plus its
middle, are that draw's BART fit. A fit saved by a version that fixed no
range has none, and its leaf values are in each chain's own unit
"callback" - if a callback function was provided while
fitting, the results of that for each sample
extract and predict return either arrays of dimensions equal to
n.observations x n.samples x n.chains when combine_chains is
FALSE, or matrices of dimensions equal to
n.observations x (n.samples * n.chains) when combine_chains is
TRUE.
fitted returns a vector of the appropriate length by averaging the
result of a call to extract.
residuals returns the response residuals on the training rows: the
observed response less fitted(object, type = "ev"), the posterior
mean of its expected value. For a binary response that expected value is
the probability of a 1.
Vincent Dorie: [email protected].