Prediction model framework for imbalanced datasets - De Leon, Maria Rossana C. (College of Engineering, University of the Philippines Diliman, 2014)
Generally, prediction requires significant and good quality input data that will give accurate prediction. However, real-data are often noisy, inconsistent and imbalanced. If the classes are imbalance, the class accuracy is unlikable because the prediction tends to favor those in majority class since it has relatively significant class size. This paper presents a framework for resolving class imbalance in prediction and classification. The framework is composed of pre-processing, prediction and output components. Pre-processing component consist of data cleaning, data transformation, discretization and data resampling. To resolve the imbalance problem, a resampling algorithm was tested in 4 different datasets each using different prediction and classification methodologies such as Regression Analysis, Decision Tree, Rule Induction and Artificial Neural Networks. Results show that the framework works in either methodologies and prediction accuracy generally improves after resampling. The framework was also compared to the existing sampling methodologies and results show that it is comparable with the hybrid ROS/RUS, but the resampling rate is minimized with the proposed framework. Keywords: Prediction model framework, class imbalance problem, data mining