Help the Stat Consulting Group by giving a gift

How can I explain a three-way interaction in ANOVA?

If you are not familiar with three-way interactions in ANOVA, please see our
general FAQ on
understanding three-way interactions in ANOVA. In short, a three-way
interaction means that
there is a two-way interaction that varies across levels of a third variable. Say, for
example, that a b*c interaction differs across various levels of factor **a**.
One way of analyzing the three-way interaction is through the use of tests of simple main-effects, e.g.,
the effect of one variable (or set of variables) across the levels of another variable.

Next, we need to obtain the tests of the simple main-effects for each
level of **a**. For this example, the residual mean-square is the error
term for all of the effects in the model and thus, for all of the tests of simple main-effects.
There are at least three ways to conduct these tests. Perhaps the
easiest way is to simply do some calculations by hand. Another way to
do this is to use the **lmatrix** subcommand and specify the various
contrasts. Once you understand how to code the contrasts on the **
lmatrix** subcommand, this is a simple method with minimal syntax
required. An example illustrating this method can be found
here. A third way to do this is to use OMS (Output Management
System) to capture the necessary values and use **aggregate** to
calculate the necessary values. The advantage of this method is that
you can get the adjusted p-values and critical F values per family error
rate; however, this requires some SPSS syntax that some people find
intimidating. An example illustrating this method can be found
here.

We will use a small artificial dataset called threeway that has a statistically significant three-way interaction to illustrate the process. In our example data set, variables a, b and c are categorical. The techniques shown on this page can be generalized to situations in which one or more variables are continuous, but the more continuous variables that are involved in the interaction, the more complicated things get.

We need to select a two-way interaction to look at more closely. For the
purposes of this example we will examine the b*c interaction. We can use
the **plot** subcommand of the **unianova** command to graph the b*c interaction for each of the two
levels of **a**. We use the **emmeans** subcommands to get the numeric
values that are displayed on the graphs.

unianova y by a b c /plot = profile(c*b*a) /emmeans=tables(a*b*c) /design = a b c a*b a*c b*c a*b*c.

We believe from looking at the two graphs above that the three-way interaction is significant because there appears to be a "strong" two-way interaction at a = 1 and no interaction at a = 2. Now, we just have to show it statistically using tests of simple main-effects.

In SPSS, we need to conduct the tests of simple main-effects in two parts. First, we begin by running the ANOVA for both levels ofsort cases by a. split file by a. unianova y by b c /design = b c b*c. split file off.

Next, we need to obtain the tests of the simple main-effects for each
level of **a**. For this example, the residual mean-square is the error
term for all of the effects in the model and thus, for all of the tests of simple main-effects.
Let's rerun the model from above (with fewer subcommands so as to reduce the
amount of output).

unianova y by a b c /design = a b c a*b a*c b*c a*b*c.

As we can see, the sums of squares error is 16 with 12 degrees of freedom; the mean square error is 1.333.

Now, let's look at the b*c interaction at a=1. The test statistic is
obtained by dividing the mean square of the b*c interaction from the ANOVA with
just **b** and **c** at a=1 by the mean square error from the original ANOVA. To
do this, we need sort the data file by **a**, split the data file by **a**, and then run
the ANOVA with **b**, **c** and the b*c interaction as predictors of **y**.

sort cases by a. split file by a. unianova y by b c.

The mean square of the b*c interaction is 20.333. So we have 20.333/1.333 = 15.25. To test the b*c interaction at a=2, we have .250/1.333 = .1875. The first test, F(2, 12) = 15.25

Clearly, the first F-ratio, F(2, 12) = 15.25, is much larger than the second, F(2, 12) = .1875, but how can we tell which are statistically significant? There are at least four different methods of determining the critical value of tests of simple main-effects. There is a method related to Dunn's multiple comparisons, a method attributed to Marascuilo and Levin, a method called the simultaneous test procedure (very conservative and related to the Scheffé post-hoc test) and a per family error rate method.We will demonstrate the per family error rate method but you should look up
the other methods in a good ANOVA book, like Kirk (1995), to decide which
approach is best for your situation. We divide our alpha level, 0.05, by 2
because we are doing two tests of simple main-effects, so our new value of alpha
is .025. The **idf** function requires us to provide 1 - alpha, so we have 1 -
.025 = .975.

compute p1 = idf.f(.975, 2, 12). exe.

As you can see, p1 is approximately 5.10. This indicates that the b*c interaction is statistically significant at a=1 but not at a=2.

In an ideal world we would be done now, but since we live in the "real" world, there is still more to do because we now need to try to understand the significant two-way interaction at a = 1; first for b = 1 and then for b = 2.

For a = 1 and b = 1:

compute filter1 = 0. if b=1 and a = 1 filter1 = 1. exe. filter by filter1. unianova y by c. filter off.

We will construct this F test in the same way we did above, namely using the
mean square for the effect, in this case, the effect of c, divided by the mean
square error from the original ANOVA. As we can see from the output above,
the mean square for c is .32. Hence, we have 32/1.33 = 24. We can
obtain the critical F-value using the **idf** function.

compute p1 = idf.f(.975, 2, 12). exe.

The critical value is approximately 5.1, so our F(2, 12) = 24 is statistically significant.

For a = 1 and b = 2:

compute filter2 = 0. if b=2 and a = 1 filter2 = 1. exe. filter by filter2. unianova y by c. filter off.

We will construct this F test in the same way we did above, namely using the
mean square for the effect, in this case, the effect of **c**, divided by the mean
square error from the original ANOVA. As we can see from the output above,
the mean square for **c** is .667. Hence, we have .667/1.33 = .5. We can
obtain the critical F-value using the **idf** function.

compute p2 = idf.f(.975, 2, 12). exe.

The critical value is approximately 5.1, so our F(2, 12) = .5 is not statistically significant.

Only the test of simple main-effects of c at b = 1 was significant. But we're not done yet, since there are three levels of c, we don't know where this significant effect lies. We need to test the pairwise comparisons among the three means. We will do this using the Sidak correction for multiple tests.

As shown above, only one of the comparisons is statistically significant. However, the Sidak correction can be conservative. If we had used a different correction, say the Tukey HSD, all three comparisons would be statistically significant. We should note that the error term used in these comparisons is not the error term from the original three-way ANOVA. We would need to use syntax similar to that shown above to save that error term to a new data set and then use it in the comparisons (as shown on http://www.ats.ucla.edu/stat/stata/faq/threeway.htm ). We might want to use the error term from the original three-way ANOVA because we are going post-hoc tests of that analysis.filter by filter1. unianova y by c /emmeans=tables(c) compare(c) adj(sidak) . filter off.

Hopefully, we now have a much better understanding of the three-way a*b*c interaction.

Please note that the process of investigating the three-way interactions would have be similar if we had chosen a different two-way interaction back at the beginning.

**Summary of Steps**

1) Run full model with three-way interaction.

1a) Capture SS and df residual.

2) Run two-way interaction at each level of third variable.

2a) Capture SS and df for interactions.

2b) Compute F-ratios for tests of simple main-effects.

3) Run one-way model at each level of second variable.

3a) Capture SS and df for main effects.

3b) Compute F-ratios for tests of simple main-effects.

4) Run pairwise or other post-hoc comparisons if necessary.

**References**

Kirk, Roger E. (1995) *Experimental Design: Procedures for the Behavioral Sciences,
Third Edition*. Monterey, California: Brooks/Cole Publishing.

The content of this web site should not be construed as an endorsement of any particular web site, book, or software product by the University of California.