Physics-informed Multi-task Tabular Transformer for Joint Fault Prediction and Remaining Useful Life Estimation of Industrial Equipment
DOI:
https://doi.org/10.65455/s4kq0p29Keywords:
Predictive Maintenance, Fault Diagnosis, Remaining Useful Life, Physics-Informed Learning, Multi-Task LearningAbstract
Accurate fault prediction and remaining useful life (RUL) estimation are essential for predictive maintenance of industrial equipment. However, existing methods mainly focus on time-series signals and often ignore the heterogeneous characteristics and degradation knowledge in tabular equipment data. This study proposes a Physics-informed Multi-task Tabular Transformer (PIMTT) for joint fault prediction and RUL estimation. The proposed model integrates numerical, categorical, and physics-informed tokens, where a type-aware physics gate is introduced to adaptively enhance degradation-related representations. A multi-task learning strategy is further developed to jointly optimize fault classification and RUL regression. Experiments on a synthetic industrial-equipment condition dataset with approximately 260,000 samples demonstrate that PIMTT achieves an F1-score of 0.8461 for fault prediction and an R2 of 0.9899 for RUL estimation, outperforming several machine learning, Transformer-based, and multi-task baselines. The results verify the effectiveness of combining physical knowledge with tabular Transformer learning for intelligent equipment health assessment.
References
[1]MOBLEY R K. An introduction to predictive maintenance. Oxford, UK: Butterworth-Heinemann, 2002. DOI: https://doi.org/10.1016/B978-075067531-4/50006-3
[2]PENG Y, DONG M, ZUO M J. Current status of machine prognostics in condition-based maintenance: a review. The International Journal of Advanced Manufacturing Technology, 2010, 50(1): 297-313. DOI: https://doi.org/10.1007/s00170-009-2482-0
[3]LEI Y, LI N, GUO L, et al. Machinery health prognostics: A systematic review from data acquisition to RUL prediction. Mechanical Systems and Signal Processing, 2018, 104: 799-834. DOI: https://doi.org/10.1016/j.ymssp.2017.11.016
[4]JARDINE A K S, LIN D, BANJEVIC D. A review on machinery diagnostics and prognostics implementing condition-based maintenance. Mechanical Systems and Signal Processing, 2006, 20(7): 1483-1510. DOI: https://doi.org/10.1016/j.ymssp.2005.09.012
[5]VAPNIK V N. The Nature of Statistical Learning Theory. New York: Springer, 1995. DOI: https://doi.org/10.1007/978-1-4757-2440-0
[6]CHEN T, GUESTRIN C. XGBoost: A scalable tree boosting system//Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York, NY: ACM, 2016: 785-794. DOI: https://doi.org/10.1145/2939672.2939785
[7]CARVALHO T P, SOARES F A, VITA R, et al. A systematic literature review of machine learning methods applied to predictive maintenance. Computers & Industrial Engineering, 2019, 137: 106024. DOI: https://doi.org/10.1016/j.cie.2019.106024
[8]LECUN Y, BENGIO Y, HINTON G. Deep learning. Nature, 2015, 521(7553): 436-444. DOI: https://doi.org/10.1038/nature14539
[9]ZHAO R, YAN R Q, CHEN Z H, et al. Deep learning and its applications to machine health monitoring. Mechanical Systems and Signal Processing, 2019, 115: 213-237. DOI: https://doi.org/10.1016/j.ymssp.2018.05.050
[10]HOCHREITER S, SCHMIDHUBER J. Long short-term memory. Neural Computation, 1997, 9(8): 1735-1780. DOI: https://doi.org/10.1162/neco.1997.9.8.1735
[11]ZHENG S, RISTOVSKI K, FARAHAT A, et al. Long short-term memory network for remaining useful life estimation//2017 IEEE International Conference on Prognostics and Health Management. Piscataway, NJ: IEEE, 2017: 88-94. DOI: https://doi.org/10.1109/ICPHM.2017.7998311
[12]PROKHORENKOVA L, GUSEV G, VOROBEV A, et al. CatBoost: unbiased boosting with categorical features. Advances in Neural Information Processing Systems, 2018, 31: 6638-6648.
[13]GORISHNIY Y, RUBACHEV I, KHRULKOV V, et al. Revisiting deep learning models for tabular data. Advances in Neural Information Processing Systems, 2021, 34: 18932-18943.
[14]KENDALL A, GAL Y, CIPOLLA R. Multi-task learning using uncertainty to weigh losses for scene geometry and semantics//IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2018: 7482-7491. DOI: https://doi.org/10.1109/CVPR.2018.00781
[15]MA J, ZHAO Z, YI X, et al. Modeling task relationships in multi-task learning with multi-gate mixture-of-experts//Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. New York, NY: ACM, 2018: 1930-1939. DOI: https://doi.org/10.1145/3219819.3220007
[16]ZHANG C, ZHANG X, WANG Y, et al. Multi-task learning for machinery fault diagnosis and remaining useful life prediction: A review. IEEE Transactions on Instrumentation and Measurement, 2023, 72: 1-15.
[17]ARIK S O, PFISTER T. TabNet: Attentive interpretable tabular learning//AAAI Conference on Artificial Intelligence. Menlo Park, CA: AAAI Press, 2021: 6679-6687. DOI: https://doi.org/10.1609/aaai.v35i8.16826
[18]RAISSI M, PERDIKARIS P, KARNIADAKIS G E. Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational Physics, 2019, 378: 686-707. DOI: https://doi.org/10.1016/j.jcp.2018.10.045
[19]LI X, ZHANG W, MA H, et al. Transformer-based fault diagnosis and remaining useful life prediction for rotating machinery: A review. Knowledge-Based Systems, 2024, 292: 111612.
[20]MO H, ZUO M J, WANG D. Remaining useful life estimation using a temporal convolutional network with attention mechanism. IEEE Transactions on Industrial Electronics, 2023, 70(10): 10458-10467.
[21]VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need. Advances in Neural Information Processing Systems, 2017, 30: 5998-6008.
[22]HUANG X, KHETAN A, CVITKOVIC M, et al. TabTransformer: Tabular data modeling using contextual embeddings[Z/OL]. arXiv:2012.06678, 2020.
[23]SOMEPALLI G, GOLDBLUM M, SCHWARZSCHILD A, et al. Saint: Improved neural networks for tabular data via row attention and contrastive pre-training[Z/OL]. arXiv:2106.01342, 2021.
[24]POPOV S, MOROZOV S, BABENKO A. Neural oblivious decision ensembles for deep learning on tabular data//International Conference on Learning Representations, 2019.
[25]FINK O, ZIO E, WEIDMANN U. A framework for prognostic uncertainty assessment based on multiple degradation process characteristics. IEEE Transactions on Reliability, 2014, 63(2): 439-448.
[26]KARNIADAKIS G E, KEVREKIDIS I G, LU L, et al. Physics-informed machine learning. Nature Reviews Physics, 2021, 3(6): 422-440. DOI: https://doi.org/10.1038/s42254-021-00314-5
[27]WILLARD J, JIA X, XU S, et al. Integrating physics-based modeling with machine learning: A survey[Z/OL]. arXiv:2003.04919, 2020.
[28]WANG H, XU C, LIU Y, et al. Physics-informed neural networks for machine fault diagnosis and remaining useful life prediction: A review. Reliability Engineering & System Safety, 2024, 245: 109965.
Downloads
Published
Data Availability Statement
Data are available from the corresponding author upon reasonable request.Issue
Section
License
Copyright (c) 2026 The Author(s). Applied Artificial Intelligence Research published by CSTDP
This work is licensed under a Creative Commons Attribution 4.0 International License.
