Which method gives less weight to noisy observations and more weight to precise ones?

Which method gives less weight to noisy observations and more weight to precise ones?

• A. Principal Component Analysis
• B. Robust Regression
• C. Weighted Least Squares
• D. Ordinary Least Squares

Answer: (C)

The main idea is to let the reliability of each measurement determine how much it influences the fit. Weighted Least Squares does this by assigning a weight to every observation, with more precise measurements getting larger weights and noisier ones getting smaller weights. When an observation has high variance (is noisy), its weight is reduced, so it has less impact on the estimated parameters. Conversely, a precise observation with small variance pulls the estimate more strongly. In practice, weights are often chosen as the inverse of the observation variance, so the minimization downweights the less reliable data and upweights the reliable data, making the estimator more efficient when the error variances differ across observations. Principal Component Analysis isn’t about fitting a regression with per-observation precision, Robust Regression focuses on downweighting outliers rather than general precision differences, and Ordinary Least Squares assumes equal variance and treats all observations equally, which can let noisy data distort the results.

The main idea is to let the reliability of each measurement determine how much it influences the fit. Weighted Least Squares does this by assigning a weight to every observation, with more precise measurements getting larger weights and noisier ones getting smaller weights. When an observation has high variance (is noisy), its weight is reduced, so it has less impact on the estimated parameters. Conversely, a precise observation with small variance pulls the estimate more strongly. In practice, weights are often chosen as the inverse of the observation variance, so the minimization downweights the less reliable data and upweights the reliable data, making the estimator more efficient when the error variances differ across observations. Principal Component Analysis isn’t about fitting a regression with per-observation precision, Robust Regression focuses on downweighting outliers rather than general precision differences, and Ordinary Least Squares assumes equal variance and treats all observations equally, which can let noisy data distort the results.