A New LASSO Procedure Based on Alarm-M Estimator for Feature Selection
Abstract
Variable selection with the least squares method, commonly used would be ineffective in dealing problem containing outliers or extreme observations. So, it requires a robust parameter estimation approach in which the estimation result is not significantly impacted by slight changes in the data. Least Absolute Shrinkage and Selection Operator (LASSO) is a commonly used method for shrinkage estimation and variable selection. But LASSO uses the conventional least squares technique for feature selection which is very sensitive to outliers. As a result, when the data is contaminated the LASSO technique gives untrustworthy conclusions. To solve this, the notion of redescending Alarm M-estimator is used to form a new feature selection method with the help of MM-estimator and by adding the LASSO penalty, and it is named as Redescending MM-LASSO (RMM- LASSO). The efficiency of the proposed method has been studied in the real and simulation environment and compared with other existing procedures with measures like Median Absolute Error (MDAE), False Positive Rate (FPR), False Negative Rate (FNR) and Mean Absolute Percentage Error (MAPE). The performance of the RMM-LASSO feature selection method shows a significant improvement compare d to LASSO method which pays a way to the importance of robust penalized regression methods.
References
1. A. Alfons; C. Croux and S. Gelper; Sparse Least Trimmed Squares Regression for Analyzing High-Dimensional Large Data Sets. The Annals of Applied Statistics, 7(1), 226-248 (2013).
2. Alamgir, A. Ali; S.A. Khan; D.M. Khan and U. Khalil; A New Efficient Redescending M-Estimator:
Alamgir Redescendig M-Estimator, Research Journal of Recent Sciences, 2(8), 79-91 (2013).
3. H. Wang; G. Li and G. Jiang; Robust Regression Shrinkage and Consistent Variable Selection through the Lad-Lasso. Journal of Business and Economic Statistics, 25(3), 347-355 (2007).
4. H. Zou; The Adaptive Lasso and Its Oracle Properties. Journal of the American Statistical Association,
101(476), 1418-1429 (2005).
5. J. Fox; An R and S-Plus Companion to Applied Regression. 1* Edition, SAGE Publications, 2002.
6. P. Rousseeuw and V. Yohai; (1984) Robust Regression by Means of s-Estimators, In Robust and Nonlinear Time Series Analysis, 26(1), 256-272.
7. P.J. Huber; Robust Estimation of a Location Parameter. The Annals of Mathematical Statistics, 35(1), 73-
101 (1964).
8. R. Muthukrishnan and C.K James; Feature Selection through Robust LASSO Procedures in Predective Modelling, 21(10), 6103-6115 (2022).
9. R. Tibshirani; Regression Shrinkage and Selection via the Lasso. Journal of the Royal Statistical Society, Series B (Methodological), 58, 267-288 (1996).
10. R.A Maronna; Robust Ridge Regression for High-Dimensional Data. Technometrics, 53(1), 44-53 (2011).
11. R.D. Cook; Detection of Influential Observation in Linear Regression. Technometrics, 42(1), 65-68
(2000).
12. S. Lambert-Lacroix and L. Zwald; Robust Regression through the Huber's Criterion and Adaptive Lasso Penalty. Electronic Journal of Statistics, 5(1), 1015-1053 (2011).
13. S.E. Anekwe and S.I. Onyeagu; A Redescending M-Estimator for Detection and Deletion of Outliers in Regression Analysis. Pakistan Journal of Statistics and Operation Research, 17(3), 997-1014 (2021)
14. V.J. Yohai; High Breakdown-Point and High Efficiency Robust Estimates for Regression. The Annals of Statistics, 15(2), 642-656 (1987).
15. Y. Qin; S. Li; Y. Li and Y. Yu; Penalized Maximum Tangent Likelihood Estimation and Robust Variable Selection, ArXiv Preprint, ArXiv: 1708.05439 (2017).
Trion Studio
