Robust Regression as a Sensible Alternative to the Weighted Ordinary Least Squares Regression in case of Heteroskedasticity. A Tutorial

Background: The robust regression is rarely used in the statistical analyses in comparison with the Ordinary Least Squares regression and the Weighted Regression. In addition, in the frequent case of the heteroskedasticity of the residuals, a weighted regression carried out once is the main suggestion of the statistical books and the resulting reduced heteroscedasticity is usually considered sufficiently satisfactory. Methods: We showed the OLS regression analysis on data simulated with a well evident heteroskedasticity and an ad hoc outlier, followed by a weighted regression iteratively carried out by using iteratively reweighted least squares, an estimation method used also in several procedures of the robust regression analysis. Therefore, the link between the iteratively performed weighted regression and the robust regression becomes immediate. Furthermore, the same data have been analysed using some robust regression procedures. Results: It has been shown that in a simulated sample of heteroscedastic data with and without an obvious artificially created outlier the weighted regression performs worse with more biased parameter estimates than robust regression procedures (such as the robust MO procedure) as the presence of the outlier is not adequately neutralized. Discussion: In presence of a heteroskedastic pattern of the residuals, the suggestion to use robust regression procedures which can also deal with the almost sure presence of outliers seems more sensible. Among the robust regression procedures carried out, the performance of the robust MO procedure appears particularly appealing since it allows biostatisticians a more reasoned management of the outliers shown in a very illustrative “ad hoc” plot. Robust regression procedures represent a sensible alternative to OLS regression taking into account that its assumptions are practically not always fulfilled and that outliers, which are almost certainly present, are not only difficult to handle in classical OLS regression but can also provide highly biased estimates.