UCLA Academic Technology Services HomeServicesClassesContactJobs
Stat Computing > Stata > FAQ
Help the Stat Consulting Group by giving a gift             
Loading

Stata FAQ
How can I get anova type results from xtmixed using the margins command? (Stata 11)

The anova and xtmixed commands present their results rather differently. The anova command displays a single test for each factor in the model including factors that have more than one degree of freedom. The xtmixed command displays estimate for each degree of freedom. Even when you follow the xtmixed command with test the results often don't agree with anova except for the highest order interaction. This page will show you how you can get anova type results from xtmixed by using the margins command.

Example 1

Our first example will use split-plot factorial model with one between subject factor a (2 levels) and one within subject factor b (4 levels). We will begin by loading the data and running the anova model for comparison purposes. 
use http://www.ats.ucla.edu/stat/data/spf24, clear

anova y a / s|a b a#b /

                           Number of obs =      32     R-squared     =  0.9613
                           Root MSE      =    .712     Adj R-squared =  0.9333

                  Source |  Partial SS    df       MS           F     Prob > F
              -----------+----------------------------------------------------
                   Model |     226.375    13  17.4134615      34.35     0.0000
                         |
                       a |       3.125     1       3.125       2.00     0.2070
                     s|a |       9.375     6      1.5625   
              -----------+----------------------------------------------------
                       b |       194.5     3  64.8333333     127.89     0.0000
                     a#b |      19.375     3  6.45833333      12.74     0.0001
                         |
                Residual |       9.125    18  .506944444   
              -----------+----------------------------------------------------
                   Total |       235.5    31  7.59677419
This model has two error terms s and the residual (technically it is the same as the b#s|a interaction). One advantage to using xtmixed for these mixed model anovas is that we do not have to specify which error term is used to test which effect. We do, however, have to be sure to include all of the necessary random effects in our xtmixed model. There is an additional advantage in that you get better estimates using xtmixed when there is missing data for some subjects.

We will now run the xtmixed command followed by tests for each of the effects. The setup for xtmixed has the fixed effects are to the left of the double pipe (||) and the random effects to the right. For each test with more than one degree of freedom we will also show the chi-square value divided by the degrees of freedom so that they are scaled the same as the F-ratio in the above anova. 

xtmixed y a##b || s:

Performing EM optimization: 

Performing gradient-based optimization: 

Iteration 0:   log restricted-likelihood = -34.824381  
Iteration 1:   log restricted-likelihood = -34.824379  

Computing standard errors:

Mixed-effects REML regression                   Number of obs      =        32
Group variable: s                               Number of groups   =         8

                                                Obs per group: min =         4
                                                               avg =       4.0
                                                               max =         4


                                                Wald chi2(7)       =    423.89
Log restricted-likelihood = -34.824379          Prob > chi2        =    0.0000

------------------------------------------------------------------------------
           y |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
         2.a |         -2   .6208193    -3.22   0.001    -3.216783   -.7832165
             |
           b |
          2  |        .25   .5034603     0.50   0.619     -.736764    1.236764
          3  |       3.25   .5034603     6.46   0.000     2.263236    4.236764
          4  |       4.25   .5034603     8.44   0.000     3.263236    5.236764
             |
         a#b |
        2 2  |          1   .7120004     1.40   0.160    -.3954951    2.395495
        2 3  |         .5   .7120004     0.70   0.483    -.8954951    1.895495
        2 4  |          4   .7120004     5.62   0.000     2.604505    5.395495
             |
       _cons |       3.75   .4389855     8.54   0.000     2.889604    4.610396
------------------------------------------------------------------------------

------------------------------------------------------------------------------
  Random-effects Parameters  |   Estimate   Std. Err.     [95% Conf. Interval]
-----------------------------+------------------------------------------------
s: Identity                  |
                   sd(_cons) |    .513701   .2233302      .2191052    1.204393
-----------------------------+------------------------------------------------
                sd(Residual) |   .7120004   .1186667      .5135861    .9870682
------------------------------------------------------------------------------
LR test vs. linear regression: chibar2(01) =     3.30 Prob >= chibar2 = 0.0346

estimates store m1  /* save estimate for use later on */

testparm a#b

 ( 1)  [y]2.a#2.b = 0
 ( 2)  [y]2.a#3.b = 0
 ( 3)  [y]2.a#4.b = 0

           chi2(  3) =   38.22
         Prob > chi2 =    0.0000

display "display as F " r(chi2)/r(df)

display as F 12.739724

testparm i.b

 ( 1)  [y]2.b = 0
 ( 2)  [y]3.b = 0
 ( 3)  [y]4.b = 0

           chi2(  3) =  107.88
         Prob > chi2 =    0.0000

display "display as F " r(chi2)/r(df)

display as F 35.958898

testparm i.a

 ( 1)  [y]2.a = 0

           chi2(  1) =   10.38
         Prob > chi2 =    0.0013
As you can see, the only test that has the same value in both the anova and the xtmixed is the one for the a#b interaction. The reason for this because the factor-variables used in the xtmixed command use indicator (dummy) coding. The tests of the main effects for a and b are really test of simple effects for the reference level of the categorical variable.

We can easily get the "correct" tests for the main effects using the margins command. 

margins b, post asbalanced

Adjusted predictions                              Number of obs   =         32

Expression   : Linear prediction, fixed portion, predict()
at           : a                (asbalanced)
               b                (asbalanced)

------------------------------------------------------------------------------
             |            Delta-method
             |     Margin   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
           b |
          1  |       2.75   .3104096     8.86   0.000     2.141608    3.358392
          2  |        3.5   .3104096    11.28   0.000     2.891608    4.108392
          3  |       6.25   .3104096    20.13   0.000     5.641608    6.858392
          4  |          9   .3104096    28.99   0.000     8.391608    9.608392
------------------------------------------------------------------------------

test (1.b=2.b)(1.b=3.b)(1.b=4.b)

 ( 1)  1bn.b - 2.b = 0
 ( 2)  1bn.b - 3.b = 0
 ( 3)  1bn.b - 4.b = 0

           chi2(  3) =  383.67
         Prob > chi2 =    0.0000

display "display as F " r(chi2)/r(df)

display as F 127.89039

estimates restore m1

(results m1 are active now)

margins a, post asbalanced

Adjusted predictions                              Number of obs   =         32

Expression   : Linear prediction, fixed portion, predict()
at           : a                (asbalanced)
               b                (asbalanced)

------------------------------------------------------------------------------
             |            Delta-method
             |     Margin   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
           a |
          1  |     5.6875   .3124999    18.20   0.000     5.075011    6.299989
          2  |     5.0625   .3124999    16.20   0.000     4.450011    5.674989
------------------------------------------------------------------------------

test (1.a=2.a)

 ( 1)  1bn.a - 2.a = 0

           chi2(  1) =    2.00
         Prob > chi2 =    0.1573
Note that the tests for a and b main effects now agree with the anova results in the first part of this example.

For our second example, let's try a model that is a bit more complex.

Example 2

The second example is a split-plot factorial with one between subject factor a (2 levels) and two within subject factors b and c (2 levels each). The model has four separate error terms but we won't have to divide any of the chi-square values from the xtmixed model by their degrees of freedom because all of the tests have just a single degree of freedom. 
use http://www.ats.ucla.edu/stat/data/spf2-22, clear

anova y a / s|a b a#b / b#s|a c a#c / c#s|a b#c a#b#c /

                           Number of obs =      32     R-squared     =  0.9920
                           Root MSE      = .559017     Adj R-squared =  0.9589

                  Source |  Partial SS    df       MS           F     Prob > F
              -----------+----------------------------------------------------
                   Model |     233.625    25       9.345      29.90     0.0002
                         |
                       a |       3.125     1       3.125       2.00     0.2070
                     s|a |       9.375     6      1.5625   
              -----------+----------------------------------------------------
                       b |         162     1         162     199.38     0.0000
                     a#b |       6.125     1       6.125       7.54     0.0335
                   b#s|a |       4.875     6       .8125   
              -----------+----------------------------------------------------
                       c |        24.5     1        24.5      61.89     0.0002
                     a#c |      10.125     1      10.125      25.58     0.0023
                   c#s|a |       2.375     6  .395833333   
              -----------+----------------------------------------------------
                     b#c |           8     1           8      25.60     0.0023
                   a#b#c |       3.125     1       3.125      10.00     0.0195
                         |
                Residual |       1.875     6       .3125   
              -----------+----------------------------------------------------
                   Total |       235.5    31  7.59677419
We begin this next section by using the egen group command to create combinations of variables to act as our random effects. We will need b by s, c by s and b by c by s along with s for our random effects. We will then run the xtmixed command followed by test of the a#b#c interaction to show that the results for the highest order interaction is the same as the anova results. 
egen bs = group(b s)
egen cs = group(c s)
egen bcs= group(b c s)

xtmixed y a##b##c || _all: R.bs || _all: R.cs || _all: R.bcs || s:, var

Performing EM optimization: 

Performing gradient-based optimization: 

Iteration 0:   log restricted-likelihood = -34.084157  (not concave)
...  
Iteration 5:   log restricted-likelihood = -34.045907  

Computing standard errors:

Mixed-effects REML regression                   Number of obs      =        32

-----------------------------------------------------------
                |   No. of       Observations per Group
 Group Variable |   Groups    Minimum    Average    Maximum
----------------+------------------------------------------
           _all |        1         32       32.0         32
              s |        8          4        4.0          4
-----------------------------------------------------------

                                                Wald chi2(7)       =    332.00
Log restricted-likelihood = -34.045907          Prob > chi2        =    0.0000

------------------------------------------------------------------------------
           y |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
         2.a |         -2   .6208194    -3.22   0.001    -3.216784   -.7832164
         2.b |       3.25   .5303298     6.13   0.000     2.210573    4.289427
             |
         a#b |
        2 2  |         .5   .7499997     0.67   0.505    -.9699723    1.969972
             |
         2.c |        .25   .4208128     0.59   0.552     -.574778    1.074778
             |
         a#c |
        2 2  |          1   .5951192     1.68   0.093    -.1664122    2.166412
             |
         b#c |
        2 2  |        .75   .5590141     1.34   0.180    -.3456475    1.845648
             |
       a#b#c |
      2 2 2  |        2.5   .7905653     3.16   0.002     .9505204     4.04948
             |
       _cons |       3.75   .4389856     8.54   0.000     2.889604    4.610396
------------------------------------------------------------------------------

------------------------------------------------------------------------------
  Random-effects Parameters  |   Estimate   Std. Err.     [95% Conf. Interval]
-----------------------------+------------------------------------------------
_all: Identity               |
                   var(R.bs) |   .2500027   .2512985      .0348603    1.792908
-----------------------------+------------------------------------------------
_all: Identity               |
                   var(R.cs) |   .0416701   .1455802      .0000443    39.22949
-----------------------------+------------------------------------------------
_all: Identity               |
                  var(R.bcs) |   .2496955   .1804157      .0605872    1.029059
-----------------------------+------------------------------------------------
s: Identity                  |
                  var(_cons) |   .1666638   .2644133      .0074371    3.734898
-----------------------------+------------------------------------------------
               var(Residual) |   .0628012          .             .           .
------------------------------------------------------------------------------
LR test vs. linear regression:       chi2(4) =     4.86   Prob > chi2 = 0.3019

Note: LR test is conservative and provided only for reference.

estimates store m1  /* save estimate for use later on */

testparm a#b#c

 ( 1)  [y]2.a#2.b#2.c = 0

           chi2(  1) =   10.00
         Prob > chi2 =    0.0016
Yes indeed, the result for the a#b#c interaction is the same in both the xtmixed and the anova. Next, we will test each of the main effects and two-way interactions to show that these results are not equivalent to those in the anova above. 
testparm i.a

 ( 1)  [y]2.a = 0

           chi2(  1) =   10.38
         Prob > chi2 =    0.0013

testparm i.b

 ( 1)  [y]2.b = 0

           chi2(  1) =   37.56
         Prob > chi2 =    0.0000

testparm a#b

 ( 1)  [y]2.a#2.b = 0

           chi2(  1) =    0.44
         Prob > chi2 =    0.5050

testparm i.c

 ( 1)  [y]2.c = 0

           chi2(  1) =    0.35
         Prob > chi2 =    0.5525

testparm a#c

 ( 1)  [y]2.a#2.c = 0

           chi2(  1) =    2.82
         Prob > chi2 =    0.0929

testparm b#c

 ( 1)  [y]2.b#2.c = 0

           chi2(  1) =    1.80
         Prob > chi2 =    0.1797
Having verified that the tests for the main effects and two-way interactions differ between xtmixed and anova, we will finish up by running a series of margin commands testing the main effects and two-way interactions correctly. 
margins a, post asbalanced

Adjusted predictions                              Number of obs   =         32

Expression   : Linear prediction, fixed portion, predict()
at           : a                (asbalanced)
               b                (asbalanced)
               c                (asbalanced)

------------------------------------------------------------------------------
             |            Delta-method
             |     Margin   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
           a |
          1  |     5.6875   .3124997    18.20   0.000     5.075012    6.299988
          2  |     5.0625   .3124997    16.20   0.000     4.450012    5.674988
------------------------------------------------------------------------------

test (1.a = 2.a)    /* a main effect */

 ( 1)  1bn.a - 2.a = 0

           chi2(  1) =    2.00
         Prob > chi2 =    0.1573

estimates restore m1

margins b, post asbalanced

Adjusted predictions                              Number of obs   =         32

Expression   : Linear prediction, fixed portion, predict()
at           : a                (asbalanced)
               b                (asbalanced)
               c                (asbalanced)

------------------------------------------------------------------------------
             |            Delta-method
             |     Margin   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
           b |
          1  |      3.125   .2724312    11.47   0.000     2.591045    3.658955
          2  |      7.625   .2724312    27.99   0.000     7.091045    8.158955
------------------------------------------------------------------------------

test (1.b = 2.b)    /* b main effect */

 ( 1)  1bn.b - 2.b = 0

           chi2(  1) =  199.38
         Prob > chi2 =    0.0000

estimates restore m1

margins c, post asbalanced

Adjusted predictions                              Number of obs   =         32

Expression   : Linear prediction, fixed portion, predict()
at           : a                (asbalanced)
               b                (asbalanced)
               c                (asbalanced)

------------------------------------------------------------------------------
             |            Delta-method
             |     Margin   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
           c |
          1  |        4.5   .2473822    18.19   0.000      4.01514     4.98486
          2  |       6.25   .2473822    25.26   0.000      5.76514     6.73486
------------------------------------------------------------------------------

test (1.c = 2.c)    /* c main effect */

 ( 1)  1bn.c - 2.c = 0

           chi2(  1) =   61.89
         Prob > chi2 =    0.0000

estimates restore m1

margins a#b, post asbalanced

Adjusted predictions                              Number of obs   =         32

Expression   : Linear prediction, fixed portion, predict()
at           : a                (asbalanced)
               b                (asbalanced)
               c                (asbalanced)

------------------------------------------------------------------------------
             |            Delta-method
             |     Margin   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
         a#b |
        1 1  |      3.875   .3852758    10.06   0.000     3.119873    4.630127
        1 2  |        7.5   .3852758    19.47   0.000     6.744873    8.255127
        2 1  |      2.375   .3852758     6.16   0.000     1.619873    3.130127
        2 2  |       7.75   .3852758    20.12   0.000     6.994873    8.505127
------------------------------------------------------------------------------

test (1.a#1.b - 1.a#2.b - 2.a#1.b + 2.a#2.b = 0)   /* a#b interaction */

 ( 1)  1bn.a#1bn.b - 1bn.a#2.b - 2.a#1bn.b + 2.a#2.b = 0

           chi2(  1) =    7.54
         Prob > chi2 =    0.0060

estimates restore m1

margins a#c, post asbalanced

Adjusted predictions                              Number of obs   =         32

Expression   : Linear prediction, fixed portion, predict()
at           : a                (asbalanced)
               b                (asbalanced)
               c                (asbalanced)

------------------------------------------------------------------------------
             |            Delta-method
             |     Margin   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
         a#c |
        1 1  |      5.375   .3498513    15.36   0.000     4.689304    6.060696
        1 2  |          6   .3498513    17.15   0.000     5.314304    6.685696
        2 1  |      3.625   .3498513    10.36   0.000     2.939304    4.310696
        2 2  |        6.5   .3498513    18.58   0.000     5.814304    7.185696
------------------------------------------------------------------------------

test (1.a#1.c - 1.a#2.c - 2.a#1.c + 2.a#2.c = 0)   /* a#c interaction */

 ( 1)  1bn.a#1bn.c - 1bn.a#2.c - 2.a#1bn.c + 2.a#2.c = 0

           chi2(  1) =   25.58
         Prob > chi2 =    0.0000

estimates restore m1

margins b#c, post asbalanced

Adjusted predictions                              Number of obs   =         32

Expression   : Linear prediction, fixed portion, predict()
at           : a                (asbalanced)
               b                (asbalanced)
               c                (asbalanced)

------------------------------------------------------------------------------
             |            Delta-method
             |     Margin   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
         b#c |
        1 1  |       2.75   .3104097     8.86   0.000     2.141608    3.358392
        1 2  |        3.5   .3104097    11.28   0.000     2.891608    4.108392
        2 1  |       6.25   .3104097    20.13   0.000     5.641608    6.858392
        2 2  |          9   .3104097    28.99   0.000     8.391608    9.608392
------------------------------------------------------------------------------

test (1.b#1.c - 1.b#2.c - 2.b#1.c + 2.b#2.c = 0)   /* b#c interaction */

 ( 1)  1bn.b#1bn.c - 1bn.b#2.c - 2.b#1bn.c + 2.b#2.c = 0

           chi2(  1) =   25.60
         Prob > chi2 =    0.0000
These two examples should give you a good feel for how to use the margins command to get anova type results from xtmixed.

How to cite this page

Report an error on this page or leave a comment

UCLA Researchers are invited to our Statistical Consulting Services
We recommend others to our list of Other Resources for Statistical Computing Help
These pages are Copyrighted (c) by UCLA Academic Technology Services

The content of this web site should not be construed as an endorsement of any particular web site, book, or software product by the University of California.