About this website
This website provides an easy-to-use tool to derive flexible cutoffs for evaluation of absolute model fit in Covariance-Based Structural Equation Modeling (CBSEM).
Package
Until February 2022, the tool was integrated into the homepage. From now on, the tool is available as a separate R package called FCO on CRAN. Please use the following links to go directly to the resources:
Problem
For CBSEM, a multitude of fit indices have been proposed, roughly classified as Goodness-of-Fit (GoF), such as CFI (Comparative Fit Index), or Badness-of-Fit (BoF), such as SRMR (Standardized Root Mean Square Residual). Despite its initial appeal, all fit indices are distorted by characteristics of the model (e.g., size of the model) and sample characteristics (e.g., sample size) to some degree. Hence, their ability to determine whether a model has truly a ‚good‘ fit or not is often masked by an unwanted sensitivity to model and sample characteristics.
Consequentially, widely acknowledged recommendations and guidelines (e.g., CFI >= .95 indicates ‚good‘ fit) struggle with these distortions because their ‚fixed‘ cutoffs remain constant, irrespective of the model and sample characteristics investigated. For example, CFI tends to be sensitive to sample size. A rather small sample (e.g., N = 200) will almost always produce smaller CFIs than a rather large sample (e.g., N = 1,000). Hence, fixed cutoffs for CFI, such as .95 will likely reject more correct models for small sample sizes and likely accept more incorrect models for larger sample sizes.
Flexible cutoffs
Flexible cutoffs aim to overcome this deficit by providing customized cutoff values for a given model with specific model and sample characteristics. For the above example, a flexible cutoff will decrease (to e.g., .94 for N = 200) or increase (to e.g., .98 for N = 1,000) depending on the model and its characteristics.
Principle
Flexible cutoffs are derived from simulated distributions of correctly specified Confirmatory Factor Analysis (CFA) models for a wide range of latent variables (e.g., 2 to 10), indicators per latent variable (e.g., 2 to 10), sample sizes (e.g., 100 to 1,000), factor loadings (e.g., .7 to .9), and normal as well as non-normal data (normal, moderate, and severe non-normal data). The package allows to specify even more models and conditions.
Flexible cutoffs can be understood as the empirical quantile of a given index for a predefined uncertainty. If an uncertainty of 5 percent (or .05) is accepted a-priori, the 5 percent quantile of the simulated distribution for correctly specified CFA models with the given model and sample characteristics determines the flexible cutoff. Depending on the nature of the underlying fit index, the appropriate lower (GoF) or upper (BoF) width of the respective confidence interval as defined by the quantile is used to derive the flexible cutoff.
Flexibility in the recommendations
Thereby, flexible cutoffs are also flexible in the recommendations they provide. If users are more uncertain about their model, they can adjust the uncertainty to 10 percent (or .10). When being more certain about the underlying model, the uncertainty can be adjusted to .1, or even .001. Note that this uncertainty is inverse to the understanding of a p-value. A researcher admits how uncertain s/he is about a given model and thus .10 indicates very conservative cutoffs, while .001 determines very lenient cutoffs.
Further information
Details on the procedure and methods can be found in the paper „Flexible Cutoff Values for Fit Indices in the Evaluation of Structural Equation Models“ (Niemand and Mai, 2018). Improved recommendations have been published as well (Mai et al., 2021) and are available in the package as well.
The tool is available as a package for R, see above. Please feel free to share the idea and links with other researchers and provide ideas as well as questions via email to webmaster@flexiblecutoffs.org or by using the contact form.
Papers
Niemand, T. & Mai, R. (2018): Flexible Cutoff Values for Fit Indices in the Evaluation of Structural Equation Models. Journal of the Academy of Marketing Science, 46(6), 1148-1172. doi:10.1007/s11747-018-0602-9