EXPERIMENTAL INVESTIGATION OF THE PERFORMANCE OF TRANSFORMER-BASED PREDICTION MODELS FOR REMAINING USEFUL LIFE

Abstract

Remaining Useful Life (RUL) prediction is an essential task in predictive maintenance. This study aims to improve the performance of deep learning models for predicting the RUL of turbojet engines using the C-MAPSS dataset. The study proposes a sequence length-based dataset labeling method and evaluates pure transformer-based models on regression, with and without pertaining, and classification tasks. Different transformerbased ensemble models are also evaluated. The results show that a shorter sequence length provides better performance in general. Pretraining the model does not always improve performance. The study also shows a potential for transformer-based ensemble models with careful choice of combined models. The conversion of the RUL prediction task from regression to classification resulted in low accuracy. While the transformerbased architecture did not outperformstate-of-the-art hybrid models for RUL prediction, it outperformed simpler single models, such as MLP and SVR. The study concludes that, while the transformer model is state-of-the-art in natural language processing, hybrid deep learning models can outperform it in other applications such as RUL prediction.