A Comparison between Random Forest and Mixed Effects Random Forest to Predict Students' Math Performance in Indonesia

Abstract

The Center for Assessment and Learning in Indonesia developed a national assessment system called the Indonesian Student Competency Assessment (AKSI/Asesesmen Kompetensi Siswa Indonesia) to measure the competence of school graduates. The results of AKSI 2019 showed that Indonesian students had poor math performance. However, the factors that affect student scores are not only caused by individual students but can also be influenced by the quality of the school. It indicates that the data has a hierarchical structure. Thus, this study aims to compare the Mixed effects random forest (MERF) method with random forest (RF) to predict students’ math performance on AKSI 2019.  The data set in this study was divided into two sets, train (85% of the original data set) and test set (15% of the original data set). Furthermore, the MERF model was built using training data by estimating the fixed component (fixed part) and the random part effects on the model, while the RF ignores the random effects on the model. Then, estimation of the value of the response variable on the test data was carried out using the MERF model that has been built. These steps are repeated 10 times to give accurate results. The Model then evaluated by calculating the mean square error prediction (MSEP), root mean square error prediction (RMSEP), and mean absolute error prediction (MAEP) from 10 models. The results showed that the MERF model produced higher prediction accuracy than the RF model. This means that school variables have an effect in predicting student performance, which can increase the accuracy of predictions if the effects are included in the model.