1 Global fit

Measures of global fit in SEM provide information about how well the model fits the data. Importantly, these statistics attempt to quantify the overall recovery of the observed data without typically considering specific components of fit or misfit in each element of the mean and covariance structure.

Global fit statistics can be divided into absolute and comparative fit indices. Absolute indices are often a function of the test statistic \(T\), which quantifies global fit to the population covariance structure (see model chi-square goodness of fit below). Absolute fit indices can also be a function of the model residuals.

Comparative fit indices compare a candidate model (specified by you) against a baseline model, which is a minimal model containing only variances for observed endogenous variables, but not covariances among them. Thus, the baseline model represents the view that there are no meaningful relationships among variables. Comparative fit indices describe how much better your model fits the data compared to this independence representation.

Global fit statistics can also be divided into tests of goodness versus badness of model fit, so it’s important to remember what are acceptable ranges for each.

1.1 Matrix expression of CFA

\[ \boldsymbol{\Sigma} = \boldsymbol{\Lambda}_y \boldsymbol{\Psi} \boldsymbol{\Lambda}_y' + \boldsymbol{\Theta}_\varepsilon \]

Reminders about SEM parameterization:

  1. The number of observed variables (sometimes called ‘observations’) is \(k\).
  2. The sample covariance matrix, estimated in a sample of \(N\) individuals, is \(\boldsymbol{S_{XX}}\).
  3. The population covariance matrix (unknown) is denoted \(\boldsymbol{\Sigma}\).
  4. The model-implied covariance matrix, given \(q\) free (estimated) parameters, is \(\boldsymbol{\Sigma}({\hat{\boldsymbol{\theta}}})\).
  5. The number of unique elements in a covariance matrix is: \(p = k (k+1) / 2\).
  6. SEM tries to minimize the discrepancy between \(\boldsymbol{S_{XX}}\) and \(\boldsymbol{\Sigma}(\hat{\boldsymbol{\theta}})\) according to an objective (fit) function: \(F = (\textbf{s} - \hat{\boldsymbol{\sigma}}(\theta))' \boldsymbol{W}^{-1} (\textbf{s} - \hat{\boldsymbol{\sigma}}(\theta))\).
  7. In standard ML estimation, the objective function is: \(\hat{F} = \textrm{log} | \boldsymbol{\Sigma}(\hat{\boldsymbol{\theta}}) | + \textrm{tr} | \boldsymbol{S_{XX}} \boldsymbol{\Sigma}(\hat{\boldsymbol{\theta}})^{-1} | - \textrm{log} | \boldsymbol{S_{XX}} | - k\).

1.2 Model chi-square goodness of fit

The model \(\chi^2\) test is the most common global fit index in SEM and is a component of several other fit indices. It is a test of the null hypothesis that the model-implied covariance matrix \(\boldsymbol{\Sigma}(\hat{\boldsymbol{\theta}})\) equals the population covariance \(\boldsymbol{\Sigma}\). Therefore, we would prefer not to reject the null hypothesis since doing so would be indicative of misfit.

In standard ML, model \(\chi^2\) is computed as the product of \(N - 1\) with the minimum value of \(\hat{F}\) estimated in the optimization algorithm (often expectation-maximization [EM]). The minimum value, denoted \(f\), represents the fit of the model to the data given a vector of parameters with the greatest sample likelihood. The model \(\chi^2\) is then computed:

\[ T = (N-1) f \] which is (asymptotically) distributed as \(\chi^2\) with \(df = p - q\). Thus, we can test the null hypothesis of perfect model fit in the population given value \(T\) and \(df\). People often test this hypothesis at \(\alpha = .05\), but as we will discuss below, the \(\chi^2\) test can be ‘significant’ in cases where there is only trivial misfit, especially in large samples.

1.2.1 Assumptions of conventional model chi-square test

  1. Observed variables are distributed as multivariate normal.
  2. \(N\) is sufficiently large (approx. 150+ observations).
  3. None of the tested parameters is at a boundary or invalid (e.g., variance of zero).

1.2.2 Relationship to likelihood ratio test (LRT)

The model \(\chi^2\) is a specific case of a likelihood ratio test (LRT) in which two models are compared according to differences in their log likelihoods (LL) and degrees of freedom. In the case of the model \(\chi^2\) test, we are comparing the fit of our proposed SEM against the perfect fit of a saturated model, which estimates all variances and covariances individually (i.e., using \(p\) parameters).

More generally, LRTs can be used to test fit differences in nested models, where model A is considered nested in model B if the free parameters of A are a subset of the parameters in B.

In this situation, the difference in model fit can be computed as the difference in model \(\chi^2\) values, which is also distributed according to a \(\chi^2\) distribution. The degrees of freedom for a nested LRT is the difference in the individual degrees of freedom for each model:

\[ \begin{align*} \Delta \chi^2 &= \chi^2_\textrm{Fewer} - \chi^2_\textrm{More} \\ \Delta df &= df_\textrm{Fewer} - df_\textrm{More} \end{align*} \] where \(Fewer\) refers to the model with fewer parameters that is nested in the \(More\) model.

1.3 Comparing candidate models against a saturated model

Let’s walk through an example that highlights model nesting and chi-square model comparisons.

Say that we are interested in how well fear of global warming (FoGWar) and concern for animal welfare (CoAWe) predict donations to charities involved in environmental protection. We measure FoGWar and CoAWe using two self-report scales with four items each. We measure donations in terms of total dollars to five large charities most involved in environmental protection (i.e., observed variable). We predict that latent standing (factor scores) on both FoGWar and CoAWe will be associated with greater donations.

1.3.1 Observed correlations

#simulate data along the hypothesized lines above
demo.model <- '

l_FoGWar =~ .8*FoGWar1 + .8*FoGWar2 + .8*FoGWar3 + .8*FoGWar4  #definition of factor FoGWar with loadings on 4 items
l_CoAWe =~ .8*CoAWe5 + .8*CoAWe6 + .8*CoAWe7 + .8*CoAWe8  #definition of factor CoAWe with loadings on 4 items

l_FoGWar =~ .2*CoAWe5 #small cross-loading

donations ~ 0.3*l_FoGWar + 0.3*l_CoAWe

l_FoGWar ~~ 0.4*l_CoAWe

#equal (unit) residual variance
FoGWar1 ~~ 1*FoGWar1
FoGWar2 ~~ 1*FoGWar2
FoGWar3 ~~ 1*FoGWar3
FoGWar4 ~~ 1*FoGWar4
CoAWe5 ~~ 1*CoAWe5
CoAWe6 ~~ 1*CoAWe6
CoAWe7 ~~ 1*CoAWe7
CoAWe8 ~~ 1*CoAWe8
'

# generate data; note, standardized lv is default
simData <- lavaan::simulateData(demo.model, sample.nobs=500)

#round(cor(simData), 2)
ggcorrplot::ggcorrplot(cor(simData), type="lower")

1.3.2 Fit a saturated model

Here is the lavaan model syntax for the saturated model.

#pairwise combinations of observed variables
pairwise <- combn(x = names(simData), m = 2)
msat.syntax <- paste(apply(pairwise, 2, function(pair) { paste(pair, collapse="~~")}), collapse="\n")

msat <- sem(msat.syntax, simData)
cat(msat.syntax)
## FoGWar1~~FoGWar2
## FoGWar1~~FoGWar3
## FoGWar1~~FoGWar4
## FoGWar1~~CoAWe5
## FoGWar1~~CoAWe6
## FoGWar1~~CoAWe7
## FoGWar1~~CoAWe8
## FoGWar1~~donations
## FoGWar2~~FoGWar3
## FoGWar2~~FoGWar4
## FoGWar2~~CoAWe5
## FoGWar2~~CoAWe6
## FoGWar2~~CoAWe7
## FoGWar2~~CoAWe8
## FoGWar2~~donations
## FoGWar3~~FoGWar4
## FoGWar3~~CoAWe5
## FoGWar3~~CoAWe6
## FoGWar3~~CoAWe7
## FoGWar3~~CoAWe8
## FoGWar3~~donations
## FoGWar4~~CoAWe5
## FoGWar4~~CoAWe6
## FoGWar4~~CoAWe7
## FoGWar4~~CoAWe8
## FoGWar4~~donations
## CoAWe5~~CoAWe6
## CoAWe5~~CoAWe7
## CoAWe5~~CoAWe8
## CoAWe5~~donations
## CoAWe6~~CoAWe7
## CoAWe6~~CoAWe8
## CoAWe6~~donations
## CoAWe7~~CoAWe8
## CoAWe7~~donations
## CoAWe8~~donations

First, let’s check the parameterization of the saturated model. Is it identified? How many degrees of freedom?

inspect(msat)
## $lambda
##          
## FoGWar1  
## FoGWar2  
## FoGWar3  
## FoGWar4  
## CoAWe5   
## CoAWe6   
## CoAWe7   
## CoAWe8   
## donations
## 
## $theta
##           FoGWr1 FoGWr2 FoGWr3 FoGWr4 CoAWe5 CoAWe6 CoAWe7 CoAWe8 dontns
## FoGWar1   37                                                            
## FoGWar2    1     38                                                     
## FoGWar3    2      9     39                                              
## FoGWar4    3     10     16     40                                       
## CoAWe5     4     11     17     22     41                                
## CoAWe6     5     12     18     23     27     42                         
## CoAWe7     6     13     19     24     28     31     43                  
## CoAWe8     7     14     20     25     29     32     34     44           
## donations  8     15     21     26     30     33     35     36     45    
## 
## $psi
## <0 x 0 matrix>

Next, let’s examine fit and parameter values:

summary(msat, fit.measures=TRUE, standardized=TRUE)
## lavaan 0.6-3 ended normally after 41 iterations
## 
##   Optimization method                           NLMINB
##   Number of free parameters                         45
## 
##   Number of observations                           500
## 
##   Estimator                                         ML
##   Model Fit Test Statistic                       0.000
##   Degrees of freedom                                 0
##   Minimum Function Value               0.0000000000000
## 
## Model test baseline model:
## 
##   Minimum Function Test Statistic              945.926
##   Degrees of freedom                                36
##   P-value                                        0.000
## 
## User model versus baseline model:
## 
##   Comparative Fit Index (CFI)                    1.000
##   Tucker-Lewis Index (TLI)                       1.000
## 
## Loglikelihood and Information Criteria:
## 
##   Loglikelihood user model (H0)              -7025.937
##   Loglikelihood unrestricted model (H1)      -7025.937
## 
##   Number of free parameters                         45
##   Akaike (AIC)                               14141.875
##   Bayesian (BIC)                             14331.532
##   Sample-size adjusted Bayesian (BIC)        14188.700
## 
## Root Mean Square Error of Approximation:
## 
##   RMSEA                                          0.000
##   90 Percent Confidence Interval          0.000  0.000
##   P-value RMSEA <= 0.05                             NA
## 
## Standardized Root Mean Square Residual:
## 
##   SRMR                                           0.000
## 
## Parameter Estimates:
## 
##   Information                                 Expected
##   Information saturated (h1) model          Structured
##   Standard Errors                             Standard
## 
## Covariances:
##                    Estimate  Std.Err  z-value  P(>|z|)   Std.lv  Std.all
##   FoGWar1 ~~                                                            
##     FoGWar2           0.581    0.080    7.291    0.000    0.581    0.345
##     FoGWar3           0.559    0.079    7.079    0.000    0.559    0.334
##     FoGWar4           0.588    0.079    7.419    0.000    0.588    0.352
##     CoAWe5            0.299    0.079    3.765    0.000    0.299    0.171
##     CoAWe6            0.263    0.073    3.580    0.000    0.263    0.162
##     CoAWe7            0.212    0.080    2.646    0.008    0.212    0.119
##     CoAWe8            0.240    0.075    3.195    0.001    0.240    0.144
##     donations         0.231    0.064    3.613    0.000    0.231    0.164
##   FoGWar2 ~~                                                            
##     FoGWar3           0.727    0.082    8.835    0.000    0.727    0.430
##     FoGWar4           0.668    0.081    8.233    0.000    0.668    0.396
##     CoAWe5            0.468    0.082    5.732    0.000    0.468    0.265
##     CoAWe6            0.380    0.075    5.065    0.000    0.380    0.233
##     CoAWe7            0.319    0.082    3.905    0.000    0.319    0.177
##     CoAWe8            0.416    0.077    5.393    0.000    0.416    0.249
##     donations         0.368    0.066    5.596    0.000    0.368    0.258
##   FoGWar3 ~~                                                            
##     FoGWar4           0.669    0.081    8.283    0.000    0.669    0.399
##     CoAWe5            0.386    0.080    4.816    0.000    0.386    0.221
##     CoAWe6            0.118    0.073    1.613    0.107    0.118    0.072
##     CoAWe7            0.269    0.081    3.323    0.001    0.269    0.150
##     CoAWe8            0.289    0.075    3.825    0.000    0.289    0.174
##     donations         0.224    0.064    3.499    0.000    0.224    0.158
##   FoGWar4 ~~                                                            
##     CoAWe5            0.308    0.079    3.871    0.000    0.308    0.176
##     CoAWe6            0.236    0.073    3.212    0.001    0.236    0.145
##     CoAWe7            0.129    0.080    1.614    0.106    0.129    0.072
##     CoAWe8            0.274    0.075    3.634    0.000    0.274    0.165
##     donations         0.312    0.065    4.822    0.000    0.312    0.221
##   CoAWe5 ~~                                                             
##     CoAWe6            0.748    0.083    9.025    0.000    0.748    0.441
##     CoAWe7            0.873    0.092    9.475    0.000    0.873    0.468
##     CoAWe8            0.795    0.085    9.308    0.000    0.795    0.458
##     donations         0.349    0.068    5.144    0.000    0.349    0.236
##   CoAWe6 ~~                                                             
##     CoAWe7            0.749    0.084    8.885    0.000    0.749    0.433
##     CoAWe8            0.616    0.077    7.987    0.000    0.616    0.382
##     donations         0.405    0.064    6.339    0.000    0.405    0.296
##   CoAWe7 ~~                                                             
##     CoAWe8            0.760    0.086    8.826    0.000    0.760    0.430
##     donations         0.389    0.070    5.583    0.000    0.389    0.258
##   CoAWe8 ~~                                                             
##     donations         0.365    0.065    5.632    0.000    0.365    0.260
## 
## Variances:
##                    Estimate  Std.Err  z-value  P(>|z|)   Std.lv  Std.all
##     FoGWar1           1.671    0.106   15.811    0.000    1.671    1.000
##     FoGWar2           1.701    0.108   15.811    0.000    1.701    1.000
##     FoGWar3           1.678    0.106   15.811    0.000    1.678    1.000
##     FoGWar4           1.675    0.106   15.811    0.000    1.675    1.000
##     CoAWe5            1.829    0.116   15.811    0.000    1.829    1.000
##     CoAWe6            1.573    0.099   15.811    0.000    1.573    1.000
##     CoAWe7            1.902    0.120   15.811    0.000    1.902    1.000
##     CoAWe8            1.647    0.104   15.811    0.000    1.647    1.000
##     donations         1.195    0.076   15.811    0.000    1.195    1.000
semPaths_default(msat, layout="circle")