Which criterion is explicitly designed to balance fit and complexity, with a lower score indicating a better model?

Which criterion is explicitly designed to balance fit and complexity, with a lower score indicating a better model?

• A. Bayesian Information Criterion (BIC)
• B. Adjusted R-Squared
• C. Akaike Information Criterion (AIC)
• D. Mean Absolute Error

Answer: (C)

Balancing fit and complexity is achieved by an information criterion that adds a penalty for each extra parameter while measuring how well the model explains the data. The Akaike Information Criterion does this by combining the model’s likelihood with a penalty for complexity: AIC = 2k − 2 ln(L), where k is the number of estimated parameters and L is the likelihood of the model given the data. The idea is to minimize information loss by choosing a model that fits reasonably well but isn’t overfitted with unnecessary parameters. Because lower AIC values indicate a better trade-off, you compare candidate models and select the one with the smallest AIC. This differs from measures like Mean Absolute Error, which focus solely on prediction error without considering model complexity, or Adjusted R-squared, which adjusts fit for degrees of freedom but doesn’t constitute a formal information-criterion penalty. Bayesian Information Criterion also balances fit and complexity (with a different penalty term), but the explicit design described here is characteristic of AIC.

Balancing fit and complexity is achieved by an information criterion that adds a penalty for each extra parameter while measuring how well the model explains the data. The Akaike Information Criterion does this by combining the model’s likelihood with a penalty for complexity: AIC = 2k − 2 ln(L), where k is the number of estimated parameters and L is the likelihood of the model given the data. The idea is to minimize information loss by choosing a model that fits reasonably well but isn’t overfitted with unnecessary parameters. Because lower AIC values indicate a better trade-off, you compare candidate models and select the one with the smallest AIC. This differs from measures like Mean Absolute Error, which focus solely on prediction error without considering model complexity, or Adjusted R-squared, which adjusts fit for degrees of freedom but doesn’t constitute a formal information-criterion penalty. Bayesian Information Criterion also balances fit and complexity (with a different penalty term), but the explicit design described here is characteristic of AIC.