Does the study design allow us to conclude that the intervention program, rather than extraneous variables, was responsible for the results?
The C-Word: Scientific Euphemisms Do Not Improve Causal Inference From Observational Data
Causal inference is a core task of science. However, authors and editors often refrain from explicitly acknowledging the causal goal of research projects; they refer to causal effect estimates as associational estimates. This commentary argues that using the term “causal” is necessary to improve the quality of observational research. Specifically, being explicit about the causal objective of a study reduces ambiguity in the scientific question, errors in the data analysis, and excesses in the interpretation of the results.
mediation: R Package for Causal Mediation Analysis
In this paper, we describe the R package mediation for conducting causal mediation analysis in applied empirical research. In many scientific disciplines, the goal of researchers is not only estimating causal effects of a treatment but also understanding the process in which the treatment causally affects the outcome. Causal mediation analysis is frequently used to assess potential causal mechanisms. The mediation package implements a comprehensive suite of statistical tools for conducting such an analysis. The package is organized into two distinct approaches. Using the model-based approach, researchers can estimate causal mediation effects and conduct sensitivity analysis under the standard research design. Furthermore, the design-based approach provides several analysis tools that are applicable under different experimental designs. This approach requires weaker assumptions than the model-based approach. We also implement a statistical method for dealing with multiple (causally dependent) mediators, which are often encountered in practice. Finally, the package also offers a methodology for assessing causal mediation in the presence of treatment noncompliance, a common problem in randomized trials.
Why Adjusted Regression Coefficients Are Less Descriptive Than They Look | R Psychologist
An interactive article about how regression adjustment works and the pitfalls of conditional associations in descriptive studies.
Estimating Causal Effects of Treatments in Randomized and Nonrandomized Studies
Presents a discussion of matching, randomization, random sampling, and other methods of controlling extraneous variation. The objective was to specify the benefits of randomization in estimating causal effects of treatments. It is concluded that randomization should be employed whenever possible but that the use of carefully controlled nonrandomized data to estimate causal effects is a reasonable and necessary procedure in many cases.
Multiple Inference and Gender Differences in the Effects of Early Intervention: A Reevaluation of the Abecedarian, Perry Preschool, and Early Training Projects
The view that the returns to educational investments are highest for early childhood interventions is widely held and stems primarily from several influential randomized trials—Abecedarian, Perry,...
Intervention Tool | Centre for Homelessness Impact
A single, accessible and trusted entry-point to current body of evidence for homelessness interventions. The tool provides one-page overviews of the available evidence, including simple ratings for strength of evidence, cost-effectiveness and impact, as well as detail on what the studies say about each intervention.
datasetpapers — a public research experiment
An experimental approach to versioned, forkable, machine-readable analyses. A prototype, not a product or service.
datasetpapers — a public research experiment
An experimental approach to versioned, forkable, machine-readable analyses. A prototype, not a product or service.
Impact Dashboard - Impact Counter | Impact Counter
Visualize the human impact of funding changes for aid and support organizations.
Results First Clearinghouse Database
To assist policymakers at all levels of government in identifying evidence-based programs and making data-driven budget decisions, the project has created the Results First Clearinghouse Database. This one-stop online resource provides policymakers with an easy way to find information on the effectiveness of various interventions as rated by eight national research clearinghouses.
The central role of the propensity score in observational studies for causal effects
Abstract. The propensity score is the conditional probability of assignment to a particular treatment given a vector of observed covariates. Both large and
Human Adults and LLMs as Scientists: Who Benefits from Active Exploration?
A long-standing finding in the causal learning literature is that adults struggle to identify conjunctive causal rules, where an effect requires the simultaneous presence of multiple causes, while...
WRKSHP.tools | Cause-Effect Diagram
A cause-effect diagram is a visual tool designed to help you explore and identify possible causes for a problem or an observed phenomenon. This helps you to come up with possible root causes for the problem you observed, that you can then use to build experiments around.
Promises and Perils of Pre-analysis Plans
The purpose of this paper is to help think through the advantages and costs of rigorous pre-specification of statistical analysis plans in economics. A pre-analysis plan pre-specifies in a precise way the analysis to be run before examining the data. A researcher can specify variables, data cleaning procedures, regression specifications, and so on. If the regressions are pre-specified in advance and researchers are required to report all the results they pre-specify, data-mining problems are greatly reduced. I begin by laying out the basics of what a statistical analysis plan actually contains so those researchers unfamiliar with it can better understand how it is done. In so doing, I have drawn both on standards used in clinical trials, which are clearly specified by the Food and Drug Administration, as well as my own practical experience from writing these plans in economics contexts. I then lay out some of the advantages of pre-specified analysis plans, both for the scientific community as a whole and also for the researcher. I also explore some of the limitations and costs of such plans. I then review a few pieces of evidence that suggest that, in many contexts, the benefits of using pre-specified analysis plans may not be as high as one might have expected initially. For the most part, I will focus on the relatively narrow issue of pre-analysis for randomized controlled trials.
Statistical Models Answer the Fundamental Clinical Question and Provide Clinical Trial Estimands – Statistical Thinking
Specific goals and estimation targets for randomized clinical trials have still not been well defined for general outcome variables. Proponents of causal inference calculus have claimed to define goals and estimands, but they have largely done so in a way that is not concordant with the most popular design, the parallel-group randomized trial. Causal inferential methods require the use of counterfactuals that are not informed by any data (outside of crossover studies) and make assumptions that are unverifiable, e.g., about the correlation structure of potential outcomes. Causal inferential structure also leads practitioners to act as if marginal treatment effect estimates are both helpful in decision making and transport to populations when in fact neither is true. Heterogeneity of participants within a treatment arm dictates heterogeneity of outcomes and heterogeneity of treatment effects when quantified on an absolute scale. Statistical models are best poised for estimation and causal inference that is specific to patient types. The increasing generality and robustness of statistical models bolsters the case. In this article I provide a succinct statement of the clinical goal of a parallel-group trial, and statistical estimands for it in the context of a general family of robust and efficient ordinal models that contain virtually all routinely used statistical models and tests as special cases.
