Archive for April, 2011

It’s A Math, Math World (Randomized Block Designs)

(Note bolded sections and diagrams are from the Research Methods Knowledge Base website at

We saw in out last post that we always want to reduce variability in our data. Stratification is used as a means of controlling sources variation in data as it potentially relates to the outcome.  When we combine stratification with blocking, we get a Randomized Block Design.

They require that the researcher divide the sample into relatively homogeneous subgroups or blocks (analogous to “strata” in stratified sampling). Then, the experimental design you want to implement is implemented within each block or homogeneous subgroup. The key idea is that the variability within each block is less than the variability of the entire sample. Thus each estimate of the treatment effect within a block is more efficient than estimates across the entire sample. And, when we pool these more efficient estimates across blocks, we should get an overall more efficient estimate than we would without blocking.

Here, we can see a simple example. Let’s assume that we originally intended to conduct a simple posttest-only randomized experimental design. But, we recognize that our sample has several intact or homogeneous subgroups. For instance, in a study of college students, we might expect that students are relatively homogeneous with respect to class or year. So, we decide to block the sample into four groups: freshman, sophomore, junior, and senior. If our hunch is correct, that the variability within class is less than the variability for the entire sample, we will probably get more powerful estimates of the treatment effect within each block. Within each of our four blocks, we would implement the simple post-only randomized experiment.

You will only benefit from a blocking design if you are correct that the blocks are more homogeneous than the entire sample. If you are wrong, you will actually be hurt by blocking (you’ll get a less powerful estimate of the treatment effect). How do you know if blocking is a good idea? You need to consider carefully whether the groups are relatively homogeneous.

How Blocking Reduces Noise

So how does blocking work to reduce noise in the data? To see how it works, you have to begin by thinking about the non-blocked study. The figure shows the pretest-posttest distribution for a hypothetical pre-post randomized experimental design. We use the ‘X’ symbol to indicate a program group case and the ‘O’ symbol for a comparison group member. You can see that for any specific pretest value, the program group tends to outscore the comparison group by about 10 points on the posttest. That is, there is about a 10-point posttest mean difference.

Now, let’s consider an example where we divide the sample into three relatively homogeneous blocks. To see what happens graphically, we’ll use the pretest measure to block. This will assure that the groups are very homogeneous. Let’s look at what is happening within the third block. Notice that the mean difference is still the same as it was for the entire sample — about 10 points within each block. But also notice that the variability of the posttest is much less than it was for the entire sample. Remember that the treatment effect estimate is a signal-to-noise ratio. The signal in this case is the mean difference. The noise is the variability. The two figures show that we haven’t changed the signal in moving to blocking — there is still about a 10-point posttest difference. But, we have changed the noise –the variability on the posttest is much smaller within each block that it is for the entire sample. So, the treatment effect will have less noise for the same signal.

Because the blocks are homogeneous, the blocking design yields a stronger treatment effect. If the blocks weren’t homogeneous — their variability was as large as the entire sample’s — we would actually get worse estimates than in the simple randomized experimental case.

Like what you read? Get blogs delivered right to your inbox as I post them so you can start standing out in your job and career. There is not a better way to learn or review biostatistics than by reading, It’s A Math, Math World

Email Marketing You Can Trust