New bcf function fits a Bayesian causal forest, with
separate prognostic and treatment ensembles. It reports both
counterfactual surfaces and each forest's fitted values without a
test matrix, and returns an object of class bartBCF with
print, fitted, extract, residuals and
predict methods.
bartc accepts method.rsp = "bcf", fitting the response
surface with a Bayesian causal forest and estimating treatment
effects as for "bart". crossvalidate is not
available with it.
group.by can be supplied together with parametric;
the varying intercept is added to the parametric equation.
bartc gains weights.rsp, the response model's own
weights, read by dbarts as weights.type says
(precisions or counts of identical rows).
With weights, summary prints the effective sample
size of the weights (Kish) over the rows averaged and, under a
common support rule, the share of the weight the rule cut. A
grouped summary gives each group its own effective sample size, as
an n.eff column beside n, and its own share of the
weight cut, in the table of rows cut.
A treatment that is a factor of two levels, the second being
treated, or logical, is read as binary by every route
(method.trt "bart", "glm", "none" and
given propensity scores). A character treatment is refused, with
an error that writes the factor(arm, levels = c("placebo",
"drug")) call from its values, the untreated value first, or
0/1, as 1.0-10 refused it on every route.
A treatment that is not binary, or has a missing value in a row
subset keeps, is now refused with an error on every route
and call, including given propensity scores; a missing value is
refused with the advice to leave such rows out with
subset, which now works. bartc had fit a treatment
coded 1 and 2, or of three values, without a message when called
through a function with method.trt = "bart", and always
with method.trt = "none" or given scores, the effect then
taking the wrong sign or meaning; recode a 1/2 treatment to 0/1.
bartc called through a function that forwards its
... (function(...) bartc(...)) with the columns in
data no longer fails with object 'z' not found. A
call through such a function without data reads its
variables where they were written, as before.
A bartc fit stores its posterior draws once, in
fit.rsp and fit.trt, rather than again as
mu.hat.obs, mu.hat.cf and samples.p.score, so
it saves at about half the size. Those three are computed when
read, by fit$mu.hat.obs and fit[["mu.hat.obs"]], and
give the same arrays; they no longer appear in names(fit),
str(fit) or with(fit, ...). A fit saved by an earlier
version reads them as stored. A matrix of propensity draws given as
method.trt is still stored.
Requires dbarts >= 1.0-0. Its sampler differs from earlier versions, so fits, and results that depend on them, will not match those from 1.0-10 for the same seed.
group.by with use.ranef = TRUE is now fit by
stan4bart, which adds a (1 | group.by) term to the
model, for the response surface and for the BART propensity model
alike; dbarts no longer fits grouped random intercepts.
stan4bart must be installed for this. The dbarts
sampling arguments (n.samples, n.burn,
n.chains, n.thin, n.threads, n.trees,
k, power, base) are translated to their
stan4bart equivalents rather than left at
stan4bart's defaults. use.ranef = FALSE is unchanged.
Semiparametric response models, those with parametric or
with group.by and use.ranef = TRUE, now refuse a
response with missing values. In 1.0-10, group.by with
use.ranef = TRUE fit such data and estimated counterfactuals
for the missing rows; drop the incomplete rows, or use
use.ranef = FALSE, which enters the grouping factor as a
fixed effect in a dbarts fit and still accepts them.
With weights, method.rsp = "tmle" now uses the
tmle package, version 2.0.0 or later, which is required for
this; a built-in estimator was used in 1.0-10. Weighted TMLE
estimates and standard errors change. Weights that are all equal
give the unweighted fit to rounding, and tmle's own refit of the
treatment mechanism, which it does for the effect on the treated
and on the controls when it trims controls, uses a glm and a gam
only, as dbarts does not fit a probit model under weights
other than 0 and 1. Without weights nothing changes.
weights are population weights: they weight every average
of the individual effects (ATE, ATT and ATC; "pate",
"sate" and "cate" in fitted, extract
and summary, within groups and in their total), and the
response model is now fit without them; weights.rsp gives it
weights. Estimates and intervals do not depend on the weights'
scale. Put the variables the weights were built from among the
confounders; see ‘Weights’ in ?bartc.
The intervals of the PATE and SATE include the response noise of
the weighted average, \sigma^2 \sum_i a_i^2 / w_i, with
a_i a row's share of the population weights and w_i its
response-model weight, where they used \sigma^2 / n; the
SATE adds a row's term once more where its response is missing. Each
drawn counterfactual has the noise the response model gives its
row, \sigma / \sqrt{w_i} for precision weights.
A "glm" treatment model is fit with weights rescaled
to mean 1. BART propensity models are probit and cannot be
weighted; they are fit without weights, with a message only
for the "p.weight" and "tmle" response methods,
whose estimators read the scores.
An args.trt list that gives weights to a BART
propensity model gets the message that the model is fit unweighted
because a probit has no weighted form; an args.rsp list that
gives weights = NULL gives nothing, where other
weights in it are refused.
weights must be numeric, finite and non-negative, with a
positive sum and one value for each row of data; they are
taken for the rows of subset, missing-response rows
included. Each refusal names the rows at fault.
bartc refuses, before it fits anything, a weighted
estimand that has no weight: an effect on the treated whose
treated rows all have weight 0, and likewise for the controls;
a common support rule that leaves no weight refuses at the end of
the fit. A group of a grouped fit with no weight is NA.
bartc warns, once for each model it fits, naming any
argument that did not reach the sampler and why, where it used to
drop the argument unseen: one that dbarts or
stan4bart has no argument for, sigdf and
sigquant for a binary response or a model fit by
stan4bart, which draws the residual standard deviation
itself, and rngKind and rngNormalKind, which
dbarts 1.0-0 no longer has, and rngSeed given with
seed, which governs as it did in 1.0-10.
family and seed are the dbarts 1.0-0
arguments that replace sigdf, sigquant and
rngSeed, which are still taken; see ?bartc for
giving both. power, base and split.probs
are dbarts' own arguments and are passed through.
tree.prior is not an argument of bartc, and is
refused with the power, base and
split.probs it translates to.
bartc(crossvalidate = TRUE) given power,
base or split.probs now cross-validates under
them, where the development builds dropped them from the
cross-validation; its chosen k and draws change.
A weights.rsp of 0 on a row in the averages is refused
before any fit when the response is plainly continuous: no
family and a response that is not 0/1, or a family
that says gaussian. The refusal names the route for a row that is
for the treatment model only, an NA response and a
population weight of 0. A logical response, or a factor of two
levels, is a 0/1 response in these checks, as it is to
dbarts.
A fit saved by 1.0-10 or earlier cannot be used by predict
in this version, whose dbarts cannot replay its saved trees;
predict says so and asks for a refit, where it used to stop
on a missing column. print, summary, extract,
fitted and refit still read such a fit.
weights.rsp stands after weights among the
arguments of bartc, so method.rsp and the arguments
after it are one place further along than in 1.0-10; a call that
gives them by position must name them.
extract and predict with combineChains = FALSE
now keep a leading chain dimension of length one for a
single-chain fit, as dbarts does: scalar quantities are a 1
by n.samples matrix and per-observation quantities a 1 by
n.samples by n.obs array. Previously the dimension was dropped.
Without weights, the "att" and "atc"
"p.weight" estimate now divides by the sum of the propensity
scores rather than by the treated (control) count, as the weighted
estimate always did, so no weights and equal weights agree.
Estimates move when the propensity score is a BART fit, by a
small part of a posterior standard deviation for the pooled
estimate and more for small groups with group.effects. For
a "glm" one they move only through the bounding of extreme
scores, when a common support rule drops rows, as the kept rows'
scores then no longer sum to the kept treated count, and, for group
estimates, when the treatment model has a varying intercept. The
p.weights returned now sum to one in each draw, where
1.0-10 returned values that averaged about one.
bartc(weights = NULL) failed; it now fits without weights.
An argument of the fitter written NULL in bartc's
... or in args.rsp or args.trt, as
sigest = NULL, failed with “subscript out of
bounds”; it now reaches the fitter as NULL, so a fit with
method.rsp = "bart" with such an argument runs, and a flat
prior can be asked of stan4bart with prior = NULL.
This held in 1.0-10.
bartc with a response written as an expression of the
columns, as log(y), failed with “invalid subscript
type 'language'”. This held in 1.0-10.
print of a fit with group.by and
group.effects = TRUE ran the groups' effects together in one
number; it now prints each group's effect, named. This held in
1.0-10.
The summary of a grouped fit under a common support rule printed
a negative n for some groups, with a warning about
recycling, as it took the rows cut from the count a second time.
It now counts the rows its estimate averages. This held in 1.0-10.
summary(target = "cate") with weights reported the
unweighted average.
Weighted "att" and "atc" estimates from extract
and fitted, and any estimate under a common support rule,
normalized the weights over every row and so were scaled by the
ratio of the weights' mean over the rows averaged to their mean over
all rows.
summary under a common support rule cut the weights to the
estimand's rows rather than to the rows kept, recycling them.
With weights and missing responses, the weights were
recycled over the rows; they are now aligned with them.
summary paired the responses of a fit with missing responses
with the wrong rows in the SATE.
With missing responses, extract of "sate",
"ite", "y.0" and "y.1" recycled the response
over the rows, with a warning, and gave wrong values; a missing
response is now drawn from its posterior predictive.
A row with a missing response was predicted twice at the
treatment it did not get and never at the treatment it got, so its
individual effect was exactly 0 in every draw. This held in 1.0-10
and earlier. Such a row is now predicted at both treatments. With
method.rsp = "bart" or "p.weight", the population,
sample and conditional effects change, roughly by the share of
missing responses times those rows' effect. With "tmle",
whose estimate leaves such rows out, only the individual quantities
from extract and the plots change. Fits made with missing
responses in 1.0-10 or earlier should be refit; a saved fit cannot
be corrected, as the prediction at the treatment the row got was
never made.
A weighted extract of a one-chain fit failed.
The total row of a grouped summary combined the groups by
their row counts rather than by their share of the weights, and
failed for target = "cate" with ci.style = "quant".
A "glm" treatment model with weights of survey scale could
fail to converge.
The standard errors of weighted method.rsp = "tmle" fits,
which came from the built-in estimator, were not converted back to
the scale of the response, and weights together with
posteriorOfTMLE = FALSE failed with an error.
The standard errors of "p.weight" fits, and of the
built-in "tmle" estimator used when tmle is not
installed, were on a 0 to 1 scale and are now in the units of the
response, as the estimates are. The sd and the intervals
from summary for those fits are wider than before, by about
40 to 50 percent in a simulation with a response of range 38.
The built-in method.rsp = "tmle" estimator, used when
tmle is not installed and there are no weights,
failed with posteriorOfTMLE = FALSE.
The "p.weight" standard error of an "att" or
"atc" fit divides each draw by the sum of its scores, as the
estimate does, with and without weights. With weights
and several draws it summed over every draw and was too small by
the number of draws.
weights are now taken for the rows of subset in
"p.weight" and "tmle" response fits, where those of
the first rows of the data were used, and with missing responses
the response is paired with its own rows, weights and scores in
both.
refit of an "att" or "atc" fit failed with a
subscript error.
Multi-threaded method.rsp = "tmle" workers are seeded from
the session's random number stream, and the cluster is closed if a
worker fails.
extract and fitted of type = "p.weights"
no longer fail when the propensity score is a "glm" fit;
for "att" and "atc" they no longer return Inf
when the fit stores no propensity-score bounds; and with
weights they weight each observation and sum to one within
each draw, where the weights were recycled across draws.
"ate" with weights returns the weights for every
draw, where it errored.
extract(type = "sigma", combineChains = TRUE) returns a
plain vector, not a matrix, for a single-chain stan4bart
fit.
refit no longer errors on a semiparametric fit.
predict takes the propensity score's column name from the
fit. Previously a confounder whose name began with the score's
(e.g. psps against a score ps) could be mistaken for
it, and predicted scores were written over that confounder.
method.rsp = "tmle" gave a different estimate with
combineChains = TRUE than with chains kept apart, because
the posterior draws were visited in a different order and
tmle::tmle uses R's random number stream. Both now use the
same order.
predict with type = "y" failed, as it sampled the
posterior predictive draws from an undefined object rather than
from the predicted means.
refit on a fit with method.rsp = "p.weight" or
"tmle" errored, as it requested an invalid quantity from
extract; with group.effects = TRUE the
"tmle" branch also referenced undefined variables and
mis-passed its maxIter and n.threads settings.
bartc with a subset and a propensity score fit on
that subset (with data supplied) assigned the score to the
wrong rows of the response model's data, silently when the lengths
divided evenly and with an error otherwise; it is now placed at the
subset's rows.
bartc(..., p.score = <vector>) matched the
p.scoreAsCovariate argument by partial matching and failed
with an unhelpful length error; the message now says so and points
to method.trt = <vector>.
extract on a fit dropped into R's debugger when the
result's dimensions could not be read, rather than signalling an
error.
method.rsp = "tmle" with estimand = "ate" no longer
has tmle also estimate the effects on the treated and on the
controls for each posterior draw. That refit the treatment
mechanism with SuperLearner whenever the propensity scores
were skewed, about 3 seconds a draw, hours at the default number of
draws; the estimates and standard errors are unchanged.
args.rsp and args.trt can be a variable, or any
expression that evaluates to a list, where only a list written out
in the call was accepted and a variable failed with an error that
it was not subsettable.
A prior written in the call as k, such as
bartc(..., k = chi(1.25, Inf)), failed with an error that
chi could not be found, as R matched k to
keepCall and bartc evaluated it, and with
method.rsp = "tmle" or combineChains the fitters
evaluated it again; it now reaches the dbarts or
stan4bart fit as written. k given with another
argument of an equal value, such as n.trees, also failed.
The same prior written in a list written out for
args.rsp or args.trt, such as
args.rsp = list(k = chi(1.25, Inf)), failed in the same
way, and each entry of the list was evaluated twice; each now
reaches the fit as written, and once. A list written out with an
entry that is not named is refused, as one held in a variable is.
subset written as an expression of the columns of
data, such as subset = x1 > 0, is read within
data, as a model frame reads it. It failed with an error
that the column was not found with group.by.
subset with a response that has missing values, fit by
dbarts, failed with an error; the rows modeled are now those
of the subset, the missing ones among them being given
counterfactual estimates as without a subset.
A response with missing values, fit by dbarts without
the propensity score as a covariate (method.trt = "none"
or p.scoreAsCovariate = FALSE), replaced the response
column of data in the calling frame with zeros and
missing values.
bartc with a confounders formula whose deparse spans
more than one line failed with “the condition has length > 1”.
plot_support on a fit with missing responses stopped with
“'x' and 'y' lengths differ”; it now plots every row.
Generics that accept ... arguments but do not use them will
now issue warnings when called with unknown arguments. Suggestion
thanks to Joshua Bon.
Compatible with TMLE >= 2.0.1.
Treatment and propensity score (when applicable) are now added to parametric model. Thanks to Joshua Bon for noticing their absence.
summary now correctly computes SATE.
summary now correctly computes sub-group averages.
Propensity score samples correctly stored for stan4bart models.
bartc now accepts a seed argument that yields
reproducible results, even when running with multiple threads
and multiple chains.
New parametric argument to bartc allows for arbitrary
linear and multilevel models. Requires the stan4bart package
to be installed.
Now works with response models in "tmle" and
"p.weight" when p.scoreAsCovariate is FALSE.
Correctly summarizes the number of samples when n.chains = 1.
sumamry now correctly computes variance expansion for PATE with
weighted observations.
summary correctly drops observations from averages when
common support rules are being used. Bug report thanks to Marcela
Veselkova.
predict now accepts as options "mu" and
"y" for when the user-supplied newdata
object already has a valid treatment assignment.
Default type argument for predict is
now "mu".
bartc argument crossvalidateBinary
renamed to crossvalidate. Accepts one of TRUE,
FALSE, "trt", or "rsp". This feature
is experimental and discouraged from use.
Confounder expressions that can evaluate successfully in the calling environment but yield a warning are now treated as if they threw an error.
summary for models fit with group.effects as
TRUE will use the level names of the grouping factor
when reporting the sub-group estimates.
"ite", "sate", "y.0", and
"y.1" for generics now returns correctly.
Report thanks to Bryan Keller.
Confounders can be passed in as literals when using
data argument. This can be of assistance in
using complicated formulae with many variables.
Passing check with tmle version >= 1.5.0.
Fixed effects should now execute correctly.
Initial public release.