Trial analysis can sound like a fairly dry corner of research methods. But every so often, a small analytical decision can change the story a study appears to tell.
One of those decisions is the analysis population: which animals, herds, farms, or cases are included in the final comparison.
We care about this because the analysis population is not just a data-cleaning detail. It is a design decision. Ideally, it should be written into the protocol before the study starts, not debated once the results are on the table.
Two useful ways to look at the same trial
Two terms come up often: intention-to-treat and per-protocol.
Intention-to-treat, or ITT, means analysing animals or farms according to the group they were originally assigned to, regardless of whether everything went perfectly afterwards.
Per-protocol, or PP, means analysing only those that followed the protocol closely enough to be considered compliant.
Both can be useful. The key is that they answer different questions.
| ITT asks What happens when this intervention is used in the real world? | PP asks What can this intervention do under ideal conditions? |
A simple heifer synchrony example
Imagine a trial of a heifer synchrony programme. Two hundred heifers are randomised: 100 to a CIDR programme and 100 to no synchrony. During the study, 15 CIDRs fall out.
Do we include those heifers in the CIDR group? Or do we exclude them because they did not receive the intervention as intended?
At first glance, excluding them might seem fair. After all, the product did not stay in place, so perhaps those heifers did not really receive the treatment.
But that decision can introduce bias.
If CIDRs are more likely to fall out on farms with poorer handling, lighter animals, more restless heifers, or other management challenges, excluding those animals may remove a disproportionate number of harder-to-get-in-calf heifers from the treated group. The remaining treated group may then look better than it really is in ordinary farm conditions.
In that situation, PP analysis could inflate the apparent benefit.
ITT preserves the original randomisation and gives a better estimate of what a farmer might expect if they adopted the programme in practice. It captures the full intervention package, including the fact that sometimes devices fall out, animals are missed, or implementation is imperfect.
ITT is not automatically better
That does not make ITT automatically “better” in every situation. ITT can dilute the estimate of biological efficacy. It may also be messier because real-world data are messy.
But that messiness is often the point.
This is why many trials use ITT as the primary analysis and PP as a sensitivity analysis. If both reach the same conclusion, confidence in the finding increases. If they differ, the interpretation needs more care.
Where this gets difficult in real trials
A similar issue comes up in real trial work. Take the example of “phantom cows”: cows thought to be pregnant that later turn out not to be. Should they stay in the analysis? Should they be excluded? There may not be a perfect answer, but there does need to be a pre-specified answer.
This is where protocol clarity becomes critical.
Drift can happen when PP analysis is used implicitly during data cleaning, when protocol deviations are not clearly defined, or when the final analysis population is debated after the study has finished. A useful rule of thumb is this: if two analysts would select different animals for the same analysis, the population has not been defined clearly enough.
Missing outcomes need the same discipline
Missing outcomes create another layer of complexity. Animals may be lost to follow-up, farmers may not present them for assessment, or owners may withdraw animals because they think the treatment is not working.
Technically, missing outcomes can sometimes be imputed. But in animal health studies, the reason an outcome is missing is often related to the outcome itself. That means imputation can introduce bias if the assumptions are weak or poorly justified.
Other options include complete case analysis, conservative assumptions such as treating missing outcomes as failures, or planned sensitivity analyses. Again, the most important point is not that one method is always right. It is that the approach should be agreed before the data are analysed.
The EpiVets takeaway
Clear analysis populations protect internal validity, strengthen regulatory credibility, reduce re-analysis and debate, and give clients more confidence in the conclusions.
The practical takeaway is simple: build the analysis population into the protocol.
Define ITT. Define PP. Define what counts as a major protocol deviation. Define how missing outcomes will be handled. Decide which analysis is primary and which is supportive.
Because when the results arrive, the question should not be, “Which animals should we include?”
It should be, “What did we say we would do, and what do the results tell us?”