How to Assess Attrition Bias in a Randomized Trial

To assess attrition bias in a randomized trial, do not judge the study by its overall dropout percentage alone. For one outcome and time point, reconstruct how many participants were randomized and analyzed in each group, compare the amount and reasons for missing outcomes, inspect the statistical handling, and ask whether plausible missing values could change the result. Balanced attrition can still bias an estimate when reasons differ; unequal attrition is a warning, not an automatic verdict.

This guide is for university students reading an intervention study, writing a critical appraisal, or preparing a systematic review. It provides a six-step method, a worked example, and assignment-ready language.

What attrition bias means

Attrition is the loss of participants from the analyzed sample after they entered a study. The U.S. Department of Education’s What Works Clearinghouse describes it as people initially included in a study not being included in the final analysis. Attrition bias is the distortion that can follow when missing outcomes are related to the true outcome, the assigned group, or both. The remaining participants may no longer represent the groups created by randomization. What Works Clearinghouse standards brief

The key word is bias, not merely missingness. Lost observations reduce precision because less information remains. They can also shift the effect estimate if, for example, participants doing poorly are more likely to disappear from one group. The amount, balance, reasons, and analysis all help determine whether that shift is plausible.

🔎 Core rule: assess attrition for a particular result, such as reading score at 12 weeks, not once for the whole paper. Cochrane’s Risk of Bias 2 framework is result-based because missingness may be minor early and substantial later. Cochrane RoB 2 guidance

Why a dropout cutoff is not enough

Rules such as “under 10 percent is safe” or “over 20 percent is high risk” are prompts to investigate, not universal cutoffs. Five percent missing can be damaging if nearly all missing participants came from one group for an outcome-related reason. Twenty percent may be less threatening when losses are well explained, similar between groups, handled with defensible methods, and tested under unfavorable assumptions.

A methodological review of 77 randomized controlled trials in four major medical journals found missing outcome data in 73 trials, or 95 percent. Median missingness was 9 percent, with a range from 0 to 70 percent, yet only 27 of the 73 trials with missing data reported a sensitivity analysis. Readers therefore need to inspect what investigators did, not just find a dropout number. Bell et al. review

A six-step method for assessing attrition bias

Step 1: Define the exact result

Write down the outcome, time point, measurement, comparison, and effect estimate. “The trial” is too broad: a participant might provide the primary outcome at 8 weeks but not the follow-up at 12 months. Your unit of appraisal should look like “mean depression score at 12 weeks for intervention versus control” or “proportion passing the mathematics test at the end of term.”

Identify whether the reported result uses intention-to-treat principles, a modified intention-to-treat rule, or complete cases. The label is not proof. Find the actual denominator and the method used for missing values.

Step 2: Reconstruct participant flow by group

Start with the CONSORT flow diagram, then verify the numbers against the Methods, Results, tables, and supplements. The CONSORT 2010 checklist asks authors to report, for each group, numbers randomly assigned, receiving treatment, analyzed for the primary outcome, and lost or excluded after randomization with reasons. Internal consistency matters more than a neat diagram. CONSORT 2010 checklist

• Randomized: the denominator created before outcomes were known.
• Outcome observed: participants with data for your chosen result.
• Included in analysis: which may differ when a model uses partial records.
• Missing and excluded: count these separately when possible.

Step 3: Calculate total and differential attrition

For each group, calculate attrition as: randomized minus participants with the target outcome, divided by randomized, multiplied by 100. Subtract the group percentages to obtain differential attrition in percentage points. “Differential attrition was 5 percentage points higher in the intervention group” is clearer than “there was a 5 percent difference.”

Use an outcome-specific denominator. Do not substitute the number who completed treatment if the outcome analysis includes non-completers, and do not combine groups before checking them separately.

Step 4: Compare reasons and likely outcomes

List reasons for missingness in each group: withdrawal, adverse effects, relocation, administrative error, loss of contact, or another cause. Ask whether each reason could be related to the unobserved outcome. Someone moving for unrelated work may differ from someone leaving because an intervention was ineffective or burdensome.

Do not invent motives. Distinguish evidence from uncertainty: “Reasons were not reported, so the relationship between missingness and outcome cannot be evaluated.” Equal numbers do not neutralize unequal reasons.

Step 5: Inspect the missing-data analysis

Complete-case analysis uses only participants with observed data. It loses precision and can bias the effect unless its assumptions are appropriate. Single-value substitutions can understate uncertainty. Multiple imputation and likelihood-based models may use observed information more efficiently, but they still require assumptions and a correctly specified model.

Ask what information entered the method and whether uncertainty was represented. “Intention-to-treat” does not restore unobserved outcomes. In the Bell et al. review, only 21 of 52 reports using an intention-to-treat or modified intention-to-treat label actually included all randomized participants in that analysis.

Step 6: Test robustness and make a judgment

A sensitivity analysis repeats the estimate under different credible assumptions about missing outcomes. It may assume missing intervention participants did worse than predicted, compare imputation models, or test plausible best- and worst-case ranges. Ask whether the substantive conclusion survives assumptions that challenge the main analysis.

Finish with one of three plain-language judgments: missingness is unlikely to materially bias this result; there are concerns because information or robustness checks are absent; or the result is at high risk because missingness could plausibly change the estimate. State your evidence and acknowledge uncertainty.

Worked example: from flow diagram to judgment

Imagine a hypothetical online tutoring trial with 240 students: 120 assigned to tutoring and 120 to usual support. At the 12-week mathematics test, outcomes are available for 108 tutoring students and 114 controls. Attrition is 12 divided by 120, or 10 percent, in tutoring and 6 divided by 120, or 5 percent, in control. Differential attrition is 5 percentage points, higher in tutoring.

Seven tutoring students withdrew because of scheduling, three said the program was too demanding, and two could not be contacted. In control, four moved schools and two could not be contacted. “Too demanding” could relate to both intervention and achievement. A complete-case estimate favors tutoring by 4.2 test points.

Suppose multiple imputation using baseline score, attendance, school, and prior attainment produces an effect of 3.7 points. A sensitivity analysis assigning missing tutoring students outcomes 2 points worse than predicted reduces the effect to 2.1 points, with an interval including no effect. The study is not automatically invalid, but its conclusion depends on a plausible assumption.

Sample appraisal: “There are concerns about attrition bias for the 12-week mathematics result. Attrition was 10% in intervention and 5% in control, some intervention withdrawals may be outcome-related, and a plausible unfavorable sensitivity analysis changed the conclusion.”

A 10-minute attrition appraisal checklist

1. Name one outcome, time point, measure, and comparison.
2. Record the number randomized to each group.
3. Record the number with the target outcome and the number analyzed.
4. Calculate attrition in each group and the percentage-point difference.
5. List reasons by group; mark unclear reasons explicitly.
6. Identify the actual analysis method, not only its label.
7. Find sensitivity analyses and note which assumptions they change.
8. Judge whether plausible missing outcomes could alter the result.
9. Write a two-sentence conclusion linking evidence to judgment.

For a literature review, keep one checklist per result rather than per paper. If you use Snitchnotes, turn the nine checks into practice questions and attach your calculated denominators to the study note; this helps prevent confusion between treatment completion and outcome availability.

Common mistakes to avoid

Calling all missing data attrition bias

Missing data create a risk; bias is the systematic distortion that may follow. Judge the pattern, reasons, analysis, and robustness instead of declaring bias because one observation is absent.

Looking only at total attrition

Pooling groups can hide differential loss. A study with 10 percent missing overall could have 2 percent missing in one group and 18 percent in the other. Always calculate group-specific rates for the chosen result.

Assuming reviewer practice supplies an easy rule

A study of 10,292 attrition-bias judgments from 729 Cochrane reviews found highly inconsistent explanations. Only 27 judgments, or 0.26 percent, included all four categories tracked: attrition percentage, between-group difference, reasons, and statistical comments. That evidence favors a transparent checklist over a bare label. Kahale et al. study

Assuming advanced statistics erase the problem

An imputation model cannot verify unseen outcomes. It formalizes assumptions using available information. Ask whether those assumptions are credible and whether alternatives were tested.

Frequently Asked Questions

What level of attrition is acceptable in a randomized trial?

There is no universal acceptable percentage. Risk depends on the outcome, imbalance between groups, reasons for missingness, likely outcomes of missing participants, and the analysis. Use a numerical threshold only as a screening prompt. A small, selective loss can matter more than a larger, well-explained and robustly analyzed loss.

Is equal attrition in both groups automatically low risk?

No. Equal percentages can conceal different reasons or likely outcomes. If participants leave the intervention because of burden while controls move away for unrelated reasons, missing outcomes may affect the comparison asymmetrically. Compare reasons and baseline information, then check whether sensitivity analyses challenge the main estimate.

Does intention-to-treat analysis prevent attrition bias?

Intention-to-treat keeps participants in their randomized groups, but missing outcomes still require assumptions or statistical handling. Check who was actually analyzed and how unobserved values were treated. The label is insufficient; denominators, methods, and sensitivity analyses show how closely the analysis follows the principle.

What is differential attrition?

Differential attrition is the difference between group-specific attrition rates, expressed in percentage points. If 10% of intervention participants and 5% of controls lack the target outcome, differential attrition is 5 percentage points. It is a warning signal, but reasons, analysis, and possible outcome differences determine its importance.

Can I assess attrition bias without participant-level data?

Yes, although certainty may be limited. Use the flow diagram, outcome tables, protocol, registry, supplements, and analysis description. Calculate what the report permits and label unreported details as unclear. Do not invent reasons or outcomes. A transparent “some concerns because reasons were not reported” is stronger than an unsupported low-risk judgment.

Conclusion

The reliable way to assess attrition bias in a randomized trial is to follow one result from randomization to analysis. Calculate group-specific loss, compare reasons, inspect the missing-data method, and test whether plausible unseen outcomes could change the conclusion. This replaces a misleading cutoff with an evidence-based judgment.

Use the nine-item checklist on the next trial you read. If you study with Snitchnotes, convert each step into a retrieval prompt so the method becomes something you can apply, not merely define.

Sources

• Cochrane: About Risk of Bias 2 — official result-level risk-of-bias guidance.
• Bell et al.: Handling missing data in RCTs — peer-reviewed methodological review.
• Kahale et al.: Assessments of attrition bias in Cochrane reviews — peer-reviewed study of appraisal consistency.
• What Works Clearinghouse Standards Briefs — U.S. Institute of Education Sciences resource.
• CONSORT 2010 Checklist — participant-flow reporting requirements.