非均一性的誤差變異數 and SERIAL CORRELATION Variance stabilize transformation Weighted LS correlated errors collinear data

Heteroskedasticity 變異數不齊一性 For regressions with cross-section data it is usually safe to assume the errors are uncorrelated, but often their variances are not constant across individuals. This is known as the problem of heteroskedasticity (for "unequal scatter"). Unequal Error Variances the usual assumption of constant error variance is referred to as homoskedasticity. Although the mean of the dependent variable might be a linear function of the regressors, the variance of the error terms might also depend on those same regressors, so that the observations might "fan out" in a scatter diagram, as illustrated in the following diagrams.

Heteroskedasticity

Heteroscadasticity 1.Examples: Variance (large firm’s sales) > Variance (small firm’s sales) Variance (high income family’s expenditure) > Variance (low income family’s) 2.Error Terms: …the error variance is not constant for each obs.

unweighted model Call:lm(formula = Y ~ X1 + X2 + X3, data = education.table) Coefficients: Estimate Std. Error t value Pr(>|t|) (Intercept) -5.566e+02 1.232e+02 -4.518 4.34e-05 *** X1 7.239e-02 1.160e-02 6.239 1.27e-07 *** X2 1.552e+00 3.147e-01 4.932 1.10e-05 *** X3 -4.269e-03 5.139e-02 -0.083 0.934 --- Residual standard error: 40.47 on 46 degrees of freedom Multiple R-Squared: 0.5913, Adjusted R-squared: 0.5647 F-statistic: 22.19 on 3 and 46 DF, p-value: 4.945e-09

boxplot of residuals from the unweighted model boxplot(rstandard(education.lm) ~ Region) Remove Alaska: 1975 was a big oil year

unweighted model after removing Alaska Coefficients: Estimate Std. Error t value Pr(>|t|) (Intercept) -277.57731 132.42286 -2.096 0.041724 * X1 0.04829 0.01215 3.976 0.000252 *** X2 0.88693 0.33114 2.678 0.010291 * X3 0.06679 0.04934 1.354 0.182591 Residual standard error: 35.81 on 45 degrees of freedom Multiple R-Squared: 0.4967, Adjusted R-squared: 0.4631 F-statistic: 14.8 on 3 and 45 DF, p-value: 7.653e-07

Properties of Classical Least Squares Under Heteroskedasticity Least squares estimators of and still unbiased and consistent; a) unbiased b) consistency (2) Not efficient (Best? Min variance?) 即 的variance並非minimum

Inherent Heteroscedasticity If the response variable follows a distribution in which the variance is functionally related to the mean, heteroscedasticity is inherent. When the residuals from OLS Regression were plotted against the predicted values, the characteristic funnel shape was observed. This would suggest that a transformation on the dependent variable Y might stabilize the variance. If Y is a Poisson random variable, counting the number of occurrences per unit of time or space, then the variance increases with the mean.

Some Types of Nonnormal Data and Their Variance-Stabilizing Transformations Type of Distribution Relationship of Mean & Variance Type of Transformation Poisson Variance = Mean Square Root Binomial Proportions Mean = p Variance = p(1–p)/n Arcsine of Exponential SD = Mean Log and Rank

Variance Stabilizing Transformations We investigate 3 different transformations: Square Root Logarithm Inverse • An examination of the following residual plots shows that only the inverse transformation improves the heterscedasticity, but not as well as the WLS regression. • Also, the OLS regression using the inverse(Y) as the response is harder to interpret than the WLS regression using Y as the response.

Another remedial measure in such cases is to use weighted least squares.

廣義的複迴歸模型函數 假設誤差項 ，對應不同的誤差項則變異數為 ，則廣義的複迴歸模型函數為： (M1.1) 其中 為參數 假設誤差項 ，對應不同的誤差項則變異數為 ，則廣義的複迴歸模型函數為： (M1.1) 其中 為參數 Xi1, …, Xi,p–1為已知的常數 為獨立且服從 i = 1,…, n

對於廣義複迴歸模型之誤差項的變異－共變異矩陣比之前所討論的更為複雜：

Weighted Least-Squares Regression The logic of WLS Regression In WLS, values of a and bk are estimated which minimize This process has the effect of minimizing the influence of a case with a large error on the estimation of a and bk and maximizing the influence of a case with a small error on the estimation of a and bk

表示模型(M1.1)之迴歸參數的最大概似及加權最小平方估計量，其最簡單的方法是採用矩陣表示。令矩陣W為由加權量wi構成的對角矩陣： 則標準的函數如下所示：

而迴歸參數之加權最小平方及最大概似估計量為： 其中bw為已加權最小平方法得到之估計的迴歸參數向量。此迴歸參數之加權最小平方估計量的共變異矩陣為： 此共變異矩陣為已知，因為變異數　 均假設已知。

Find a weight: 誤差項變異數已知 Observations with small variances provide more reliable information about the regression function than do those with large variances. Each observation is weighted differently. inversely proportionally to its variance:

Find a weight -Grouped data the observed yi‘ s are actually averages of several (say ni) observations.

Find a weight: function of a predictor

Ex:Age and BP OLS

Residual plots

Find a weight

Ex: WLS Age and BP

revisit: Education example Suppose we have a hypothesis about the weights, i.e. they are constant within Region, Fit model using OLS (Ordinary Least Squares) to get initial estimate b_{OLS} Use predicted values from this model to estimate wi . Refit model using WLS (Weighted Least Squares). If needed, iterate previous two steps.

weighted model after removing Alaska lm(formula = Y ~ X1 + X2 + X3, weights = weights) Coefficients: Estimate Std. Error t value Pr(>|t|) (Intercept) -3.181e+02 7.833e+01 -4.060 0.000193 *** X1 6.245e-02 7.867e-03 7.938 4.24e-10 *** X2 8.791e-01 2.003e-01 4.388 6.83e-05 *** X3 2.981e-02 3.421e-02 0.871 0.388178 --- Residual standard error: 0.984 on 45 degrees of freedom Multiple R-Squared: 0.7566, Adjusted R-squared: 0.7404 F-statistic: 46.63 on 3 and 45 DF, p-value: 7.41e-14

WLS in matrix form

Consequences of the Errors Being Autocorrelated Correlated errors Consequences of the Errors Being Autocorrelated

SERIAL CORRELATION error corresponding to different observations are correlated.

Non Independence of Error Variables A time series is constituted if data were collected over time. Examining the residuals over time, no pattern should be observed if the errors are independent. When a pattern is detected, the errors are said to be autocorrelated. Autocorrelation can be detected by graphing the residuals against time.

Non Independence of Error Variables Patterns in the appearance of the residuals over time indicates that autocorrelation exists. Residual Residual + + + + + + + + + + + + + + + Time Time + + + + + + + + + + + + + Note the runs of positive residuals, replaced by runs of negative residuals Note the oscillating behavior of the residuals around zero.

自我相關的問題

The impacts- (1) unbiased, consistency, but not efficient 即 為不偏的, 即 為不偏的, 但 為有偏的, biased, 且not minimum variance (2) MSE underestimate the variance of the error terms (3) underestimate (4) CI and tests using the t and F distributions are no longer strictly applicable

Correlated errors (AR(1) noise) Suppose that, instead of being independent, the errors in our model were If rho is close to 1, then errors are very correlated, rho= 0 is independence. This is “Auto-Regressive Order (1)” noise (AR(1)). 第一階自迴歸誤差模型 Many other models of correlation exist: ARMA, ARIMA, ARCH,GARCH, etc.

線性迴歸 單預測變數當隨機誤差項服從第一階自我迴歸過程AR(1)時的廣義簡單線性迴歸模型為： 隨機誤差項服從第一階自我迴歸過程時的廣義複迴歸模型為：

4.Corrections for serial correlation (1)subscript….t total observations ….T (2)假設

自我相關的Durbin-Watson檢定 由於商業及經濟應用上相關的誤差項傾向於有正的序列相關，常見的檢定假說為 要得到Durbin-Watson檢定統計量，先以普通最小平方法配適迴歸函數，計算普通殘差 然後計算統計量 其中n為個案數。

正確臨界值難求，但Durbin和Watson有得到上下界限 和 ，當D值落於界限之外時可得一確定結論。對(12.12)之假說檢定的決策規則為

自我相關的矯正策略 考慮轉換的應變數： 其中 其中 為獨立的干擾項。

因此，使用轉換變數 及 則得具獨立誤差項的標準簡單線性迴歸模型。這表示普通最小平方法用在此模型具有其最適性。

estimating rho Cochrane-Orcutt程序 Cochrane-Orcutt包含三步驟的迭代。 假設的自我迴歸誤差過程視為一通過原點的迴歸

Hildreth-Lu程序 估計自我相關參數 以Hildreth-Lu程序選擇的 值要將轉換後迴歸模型的誤差平方和極小化： 有電腦程式可找出使SSE最小的 值。

第一階差程序 由於自我相關參數 常很大，且將SSE視為 的函數，且其在 大時相當平坦，如Blaisdell公司之例，因此有些經濟學家及統計學家建議在轉換的模型 = 1.0代入。 其中 因此，迴歸係數 再一次可直接用普通最小平方法估計，但這次是依經過原點迴歸計算。

轉換後變數的配適迴歸函數為 可轉回原始變數如下： 其中