International Journal of Statistics and Reliability Engineering

No Publication Cost

Vol 10, No 2:

subscription

A Precise Performance of Robust Weighted Support Vector Regression Approach under Various Kernels

Abstract

Support Vector Machine (SVM) which is used to solve regression problems is referred to as Support Vector Regression (SVR). This work mainly focuses on improving the robustness of SVR along with the study of various kernels such as linear, polynomial, Radial Basis Function (RBF) and bessel. A novel idea, the Hampel’s weight function has been used to increase the accuracy of the SVR, and the name of the proposed procedure coined as Robust Weighted Support Vector Regression (RWSVR). The performance of the RWSVR has been studied with the help of various kernels by checking their efficiencies under the various error measures such as Mean Absolute Error (MAE), Median Absolute Error (MDAE), Mean Absolute Percent Error (MAPE) and Root Mean Square Error (RMSE). Further, the real and simulation study concluded that RWSVR along with the RBF kernel performs well over the other kernels. The RWSVR may be applied in statistical learning in order to get more accuracy than the existing procedures, even the data with contamination.

Full Text

PDF

References

1. A.J. Smola and B. Scholkopf; A Tutorial on Support Vector Regression. Statistics and Computing, 14(2), 199-122 (2004).
2. A. Kazem; E. Sharifia; F.K. Hussainb; M. Saberi and O.K. Hussain; Support Vector Regression with Chaos-Based Firefly Algorithm for Stock Market Price Forecasting. Applied Soft Computing, 13(2), 947-958 (2013).
3. A. Karatzoglou; D. Meyer and K. Hornik; Support Vector Machines in R. Journal of Statistical Software, 15(9), 1-28 (2006).
4. A.K. Gupta; S.C. Guntuku; R.K. Desu and A. Balu; Optimisation of Turning Parameters by Integrating Genetic Algorithm with Support Vector Regression and Artificial Neural Networks. International Journal of Advanced Manufacturing Technology, 77(1-4), 331–339 (2015).
5. A.M. Deris; A.M. Zain and R. Sallehuddin; (2011) Overview of Support Vector Machine in Modeling Machining Performances. International Conference on Advances in Engineering, Procedia Engineering, 24, 308 – 312.
6. B.E. Erdogan; Prediction of Bankruptcy using Support Vector Machines: An Application to Bank Bankruptcy. Journal of Statistical Computations and Simulation, 12(8), 1543-1555 (2013).
7. B.M. Gupta; S.M. Dhawan and G.M. Mamdapur; Support Vector Machine (SVM) Research in India: A Scientometric Evaluation of India’s Publications Output During 2002-19. Journal of Indian Library Association, 57(3), 12-25 (2021).
8. C. Cortes and V.N. Vapnik; Support Vector Networks. Machine Learning, 20(3), 273–297 (1995).
9. C.C. Chang and C.J. Lin; Training ν-Support Vector Classiﬁers: Theory and Algorithms. Neural Computation, 13(9), 2119–2147 (2001).
10. C.H. Wu; J.M. Ho and D.T. Lee; Travel-Time Prediction with Support Vector Regression. IEEE Transactions on Intelligent Transportation Systems, 5(4), 276-281 (2004).
11. C. Shoko and C. Sigauke; Short-Term Forecasting of COVID-19 using Support Vector Regression: An Application using Zimbabwean Data. American Journal of Infection Control (2023). DOI: https://doi.org/10.1016/j.ajic.2023.03.010.
12. D.K. Basha and T. Venkateswarlu; Linear Regression Supporting Vector Machine and Hybrid LOG Filter-Based Image Restoration. Journal of Intelligent Systems, 29(1), 1480–1495 (2020).
13. D. Lee; G. Kim and K.E. Lee; Soil Moisture Prediction using a Support Vector Regression. Journal of the Korean Data & Information Science Society, 24(2), 401–408 (2013).
14. F.E.H. Tay and L.J. Cao; A Comparative Study of Saliency Analysis and Genetic Algorithm for Feature Selection in Support Vector Machines. Intelligent Data Analysis, 5(3), 191−209 (2001).
15. F. Kaytez; M.C. Taplamacioglu; E. Cam and F. Hardalac; Forecasting Electricity Consumption: A Comparison of Regression Analysis, Neural Networks and Least Squares Support Vector Machines. International Journal of Electrical Power and Energy Systems, 67(C), 431– 438 (2015).
16. F.R. Hampel; E.M. Ronchetti; P.J. Rousseeuw and W.A. Stahel; Robust Statistics: The Approach Based on Influence Functions. John Wiley & Sons, New York (1986).
17. G. Wang; (2012) Demand Forecasting of Supply Chain Based on Support Vector Regression Method. In International Workshop on Information and Electronics Engineering, Procedia Engineering, 29, 280 – 284.
18. G.W. Flake and S. Lawrence; Efﬁcient SVM Regression Training with SMO. Machine Learning, 46(1-3), 271–290 (2002).
19. G. Camps-Valls; L. Bruzzone; J.L. Rojo-Alvarez and F. Melgani; Robust Support Vector Regression for Biophysical Variable Estimation from Remotely Sensed Images. IEEE Geoscience and Remote Sensing Letters, 3(3), 339-343 (2006).
20. H. Yang; K. Huang; I. King and M.R. Lyu; Localized Support Vector Regression for Time Series Prediction, Neurocomputing, 72(10-12), 2659-2669 (2009).
21. H. Drucker; C.J.C. Burges; L. Kaufman; A. Smola and V. Vapnik; (1996) Support Vector Regression Machines. Proceedings of the 9th International Conference on Neural Information Processing Systems, 155-161.
22. H. Takeda; S. Farsiu and P. Milanfar; Kernel Regression for Image Processing and Reconstruction. Transactions on Image Processing, 16(2), 349-366 (2007).
23. H. Huang; X. Wei and Y. Zhou; An Overview on Twin Support Vector Regression. Neurocomputing, 490, 80-92 (2022).
24. J. Prada; General Noise Support Vector Regression with Non-Constant Uncertainty Intervals for Solar Radiation Prediction. Journal of Modern Power Systems and Clean Energy; 6(2), 268–280 (2018).
25. J. Nayak; B. Naik and H.S. Behera; A Comprehensive Survey on Support Vector Machine in Data Mining Tasks: Applications & Challenges. International Journal of Database Theory and Application, 8(1), 169-186 (2015).
26. J. Ma; J. Theiler and S Perkins; Accurate On-line Support Vector Regression. Neural Computation, 15(11), 2683–2703 (2003).
27. K. Mohammadi; S. Shamshirband; H. Anisi; K.A. Alam and D. Petkovic; Support Vector Regression based Prediction of Global Solar Radiation on a Horizontal Surface. Energy Conversion and Management, 91, 433-441 (2015).
28. M. Bozic and M. Stojanovic; Application of SVM Methods for Mid-Term Load Forecasting. Serbian Journal of Electrical Engineering, 8(1), 73-83 (2011).
29. O.L. Mangasarian and D.R. Musicant; Robust Linear and Support Vector Regression. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(9), 1-6 (2000).
30. P.J. Rousseeuw and V. Yohai; Robust Regression by Means of S-Estimators. Robust and Non-Linear Time Series Analysis, 26, 256-272 (1984).
31. P. Borah and D. Gupta; Review: Support Vector Machines in Pattern Recognition. International Journal of Engineering and Technology, 9(3), 43-48 (2017).
32. R.R. Perez and J. Bajorath; Evolution of Support Vector Machine and Regression Modeling in Chemoinformatics and Drug Discovery. Journal of Computer-Aided Molecular Design, 36, 355-362 (2022).
33. R. Collobert and S. Bengio; SVM Torch: Support Vector Machines for Large-Scale Regression Problems. Journal of Machine Learning Research, 1, 143–160 (2001).
34. S. Balasundaram and Y. Meena; Robust Support Vector Regression in Primal with Asymmetric Huber Loss. Neural Processing Letters, 49(3), 1399-1431 (2019).
35. S.S. Keerthi; S. Shevade; C. Bhattacharyya and K. Murthy; Improvements to SMO Algorithm for SVM Regression. IEEE Transactions on Neural Networks, 11(5), 1188–1193 (2000).
36. S.V. Looy; T. Verplancke; D. Benoit; E. Hoste; G.V. Maele; F.D. Turck and J. Decruyenaere, A Novel Approach for Prediction of Tacrolimus Blood Concentration in Liver Transplantation Patients in the Intensive Care Unit through Support Vector Regression. Critical Care, 11(4), 1-7 (2007).
37. S. Jore and P. R. Badadapure; Remote Sensing Image Segmentation using Linear Regression. International Journal of Engineering Research & Technology, 3(4), 2279-2282 (2014).
38. T.B. Trafalis and H. Ince; (2000) Support Vector Machine for Regression and Applications to Financial Forecasting, Neural Network-IJCNN 2000. Proceedings of the IEEE-INNS-ENNS International Joint Conference, 6, 348−353.
39. T.A. Altameem; V. Nikolic; S. Shamshirband; D. Petkovic; H. Javidnia; M.L.M. Kiah and A. Gani; Potential of Support Vector Regression for Optimization of Lens System. Computer-Aided Design, 62, 57–63 (2015).
40. V. Anandhi and R.M. Chezian; Support Vector Regression in Forecasting. International Journal of Advanced Research in Computer and Communication Engineering, 2(10), 4148-4151 (2013).
41. V.N. Vapnik; S. Golowich and A. Smola; (1997) Support Vector Method for Function Approximation, Regression Estimation, and Signal Processing. Proceedings of Advances in Neural Information Processing Systems, 9, MIT Press, Cambridge, 281-287.
42. V. Vapnik; The Nature of Statistical Learning Theory. Springer, New York (1995).
43. V. Yohai; High Breakdown Point and High Efficiency Robust Estimates for Regression. The Annals of Statistics, 15(2), 642-656 (1987).
44. X. Pan; X. Pang; H. Wang and Y. Xu; A Safe Screening Based Framework for Support Vector Regression. Neurocomputing, 287, 163–172 (2018).
45. X. Liu and Y. Zuo; Computing Half-Space Depth and Regression Depth. Simulation and Computation, 43(5), 969-985 (2014).
46. Y. Radhika and M. Shashi; Atmospheric Temperature Prediction using Support Vector Machines. International Journal of Computer Theory and Engineering, 1(1), 1793-8201 (2009).
47. Y. Wang; B. Wang and X. Zhang; (2012) A New Application of the Support Vector Regression on the Construction of Financial Conditions Index to CPI Prediction. Procedia Computer Science, International Conference on Computational Science, ICCS 2012, 9, 1263 – 1272.

ISSN(P) 2350-0174

ISSN(O) 2456-2378

Journal Content