Ranking Feature Sets for Emotion Models used in Classroom Based Intelligent Tutoring Systems, submitted to the International Conference on User Modeling and Adaptive Presentation
Recent progress has been made in using sensors with Intelligent Tutoring Systems in classrooms in order to predict the affective state of students users. If tutors were able to interpret sensor data with new students based on past experience, rather than having to be individually trained, then tutor developers could evaluate various methods of adapting to each student's affective state using consistent predictions. Our classifiers for emotion have predicted student emotions with an accuracy between 78% and 87%. However, it is still unclear which sensors are best, and the educational technology community needs to know this to develop better than baseline classifiers, e.g. ones that use only frequency of emotional occurrence to predict affective state. This paper suggests a method for comparing classifiers using different sensors as well as a method for validating the classifiers on a novel population. This involves training our classifiers on data collected in the Fall of 2008 and testing them on data collected in the Spring of 2009. Results of the comparison show that the classifiers for some affective states are significantly better than the baseline, and a validation study found that not all classifiers rankings generalize to new settings. The analysis suggests that though there is some benefit gained from simple linear classifiers, more advanced methods are needed for better results.