Help the Stat Consulting Group by giving a gift

How can I compute a chi-squared test for nested models with the MLMV or WLSMV estimators (difftest)?

Chi-squared difference tests are frequently used to test differences between nested models in confirmatory factor analysis, path analysis, and structural equation modeling. Nested models are two models (or more if one is fitting a series of models) that are identical except that one of the models constrains some of the parameters (the null model) and one does not have those constraints (the alternative model). Examples of this include overall tests for a set of dichotomous predictors representing a single nominal (categorical), or a test for differences across groups in a multiple group model. Typically performing a chi-squared difference test involves calculating the difference between the chi-squared statistic for the null and alternative models, the resulting test statistic is distributed chi-squared with degrees of freedom equal to the difference in the degrees of freedom between the two models. However, when a model is run in Mplus using the MLMV or WLSMV estimators, the following warning message is displayed, warning the user that the standard chi-squared difference test is not valid:

* The chi-square value for MLM, MLMV, MLR, ULSMV, WLSM and WLSMV cannot be used for chi-square difference tests. MLM, MLR and WLSM chi-square difference testing is described in the Mplus Technical Appendices at www.statmodel.com. See chi-square difference testing in the index of the Mplus User's Guide.

If you are using the MLMV or WLSMV tesimators you can use the **difftest** command to
test for differences in model fit. Alternatively, you could use the **model test**
command to calculate a Wald test for an equivalent null hypothesis. For more information
on Wald tests you can see our FAQ page:
How are the likelihood ratio, Wald, and Lagrange
multiplier (score) tests different and/or similar? Note that while the **difftest**
option is only
available for the MLMV and WLSMV estimators, the **model test** command can be used with a variety of models.
If you are unsure of what estimator is being used in your model, you can find the estimator listed towards
the top of the output file. Also keep in mind that this type of test is only valid if the models are nested, that is, the models
must be the same except that one of the models includes additional constraints
on the parameters.

Below we show an example of how to use the **difftest** command to test
nested models. In this simple example, the model contains four variables. The outcome variable
**y** has three ordered categories, and the three predictor variables, **x1**, **x2**,
and **x3** could be either continuous or binary (0/1). For this example, we
will test whether that including the variables **x2** and **x3** as
predictors of **y** significantly improves the fit of the model (i.e. that
the two variables together are statistically significant). When the outcome
variable is an ordered category, the default estimator is WLSMV so we will need
to use **difftest** to test this hypothesis.

The first step is to run the full or unconstrained model (also denoted H1).
Below is the Mplus input file where **x1**, **x2**, and **x3** are used to predict **
y**.
We have used the categorical option of the variables command to inform Mplus
that the variable **y** is ordinal (the same option is used to identify
binary outcome variables). We have used the **difftest**
option of the **savedata** command to save the information necessary for our
chi-squared difference test to the file **mydiff.dat**.

data: file is d:\data\hsb2.dat; variable: names are id y x1 x2 x3;categorical are y;model: y on x1 x2 x3;savedata: difftest is mydiff.dat;

Once we have run the unconstrained model (above), we can then run the constrained
or null model model (also denoted H0).
In the model below we have used the **difftest** option, this time in the **
analysis** command, to tell Mplus we wish to compare the current model to the
model saved in the file **mydiff.dat**. Recall that we want to test the null
hypothesis that **x2**, and **x3** used together to predict **y** do
not significantly improve the fit of the model. To test this hypothesis we fix
the coefficients for the variables **x2**, and **x3** to zero using **@0**.

data: file is d:\data\hsb2.dat; variable: names are y x1 x2 x3; categorical are y;analysis: difftest = mydiff.dat;model:y on x1 x2@0 x3@0;

In the output, immediately after the chi-square test of model fit, Mplus
prints the results of the chi-square difference test (i.e. **difftest**).
The resulting chi-square value of 1.852 with 2 degrees of freedom results in
a p-value of 0.3961, thus we cannot reject the null hypothesis that
including **x2**, and **x3** as predictors of **y** does not
significantly improve model fit.

Chi-Square Test for Difference Testing Value 1.852 Degrees of Freedom 2** P-Value 0.3961 * The chi-square value for MLM, MLMV, MLR, ULSMV, WLSM and WLSMV cannot be used for chi-square difference tests. MLM, MLR and WLSM chi-square difference testing is described in the Mplus Technical Appendices at www.statmodel.com. See chi-square difference testing in the index of the Mplus User's Guide. ** The degrees of freedom for MLMV, ULSMV and WLSMV are estimated according to a formula given in the Mplus Technical Appendices at www.statmodel.com. See degrees of freedom in the index of the Mplus User's Guide.

The content of this web site should not be construed as an endorsement of any particular web site, book, or software product by the University of California.