Investigation of Transformers and Other Machine Learning Techniques for Health Data Classification

Abstract

This thesis explores the application of transformer model and other machine learning techniques in predicting outcomes based on physiological data, with a primary focus on a unique dataset capturing physiological responses during real-world cognitive stressors. Initially, the transformer model was applied to an ECG dataset to understand its capabilities and evaluate the outcomes. The primary investigation, however, centers on an open-access dataset that includes electrodermal activity, heart rate, blood volume pulse, skin surface temperature, and accelerometer data recorded during three exam sessions—midterm 1, midterm 2, and finals—along with the corresponding grades. This dataset is intended to investigate physiological stress responses and assess the effectiveness of Transformer models in analyzing such data. By applying advanced machine learning techniques, especially Transformers, the aim is to uncover deeper insights into how physiological indicators of stress correlate with performance, particularly in exam scenarios. This approach demonstrates the potential of Transformer models in managing complex, real-world physiological data while offering meaningful insights through detailed analysis. The data was preprocessed and initially modeled using Logistic Regression and Random Forest to establish baseline predictive capabilities. Following this, the transformer model was applied to evaluate its performance in predicting exam grades. Comparative analysis between the models was conducted to assess the predictive power of the transformer model relative to traditional methods. The findings provide important insights into the effectiveness of Transformer models in managing complex physiological data. While their versatility and strong performance make them a powerful tool, the results of this study suggest that their application should be selective, particularly in tasks like stress and performance prediction. Nonetheless, the ability of Transformers to handle intricate, multivariate datasets demonstrates their potential for broader applications in health-related fields, paving the way for future research in predictive modeling.