Physics-informed Multi-task Tabular Transformer for Joint Fault Prediction and Remaining Useful Life Estimation of Industrial Equipment
DOI:
https://doi.org/10.65455/s4kq0p29关键词:
Predictive Maintenance, Fault Diagnosis, Remaining Useful Life, Physics-Informed Learning, Multi-Task Learning摘要
Accurate fault prediction and remaining useful life (RUL) estimation are essential for predictive maintenance of industrial equipment. However, existing methods mainly focus on time-series signals and often ignore the heterogeneous characteristics and degradation knowledge in tabular equipment data. This study proposes a Physics-informed Multi-task Tabular Transformer (PIMTT) for joint fault prediction and RUL estimation. The proposed model integrates numerical, categorical, and physics-informed tokens, where a type-aware physics gate is introduced to adaptively enhance degradation-related representations. A multi-task learning strategy is further developed to jointly optimize fault classification and RUL regression. Experiments on a synthetic industrial-equipment condition dataset with approximately 260,000 samples demonstrate that PIMTT achieves an F1-score of 0.8461 for fault prediction and an R2 of 0.9899 for RUL estimation, outperforming several machine learning, Transformer-based, and multi-task baselines. The results verify the effectiveness of combining physical knowledge with tabular Transformer learning for intelligent equipment health assessment.
参考
[1]MOBLEY R K. An introduction to predictive maintenance. Oxford, UK: Butterworth-Heinemann, 2002. DOI: https://doi.org/10.1016/B978-075067531-4/50006-3
[2]PENG Y, DONG M, ZUO M J. Current status of machine prognostics in condition-based maintenance: a review. The International Journal of Advanced Manufacturing Technology, 2010, 50(1): 297-313. DOI: https://doi.org/10.1007/s00170-009-2482-0
[3]LEI Y, LI N, GUO L, et al. Machinery health prognostics: A systematic review from data acquisition to RUL prediction. Mechanical Systems and Signal Processing, 2018, 104: 799-834. DOI: https://doi.org/10.1016/j.ymssp.2017.11.016
[4]JARDINE A K S, LIN D, BANJEVIC D. A review on machinery diagnostics and prognostics implementing condition-based maintenance. Mechanical Systems and Signal Processing, 2006, 20(7): 1483-1510. DOI: https://doi.org/10.1016/j.ymssp.2005.09.012
[5]VAPNIK V N. The Nature of Statistical Learning Theory. New York: Springer, 1995. DOI: https://doi.org/10.1007/978-1-4757-2440-0
[6]CHEN T, GUESTRIN C. XGBoost: A scalable tree boosting system//Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York, NY: ACM, 2016: 785-794. DOI: https://doi.org/10.1145/2939672.2939785
[7]CARVALHO T P, SOARES F A, VITA R, et al. A systematic literature review of machine learning methods applied to predictive maintenance. Computers & Industrial Engineering, 2019, 137: 106024. DOI: https://doi.org/10.1016/j.cie.2019.106024
[8]LECUN Y, BENGIO Y, HINTON G. Deep learning. Nature, 2015, 521(7553): 436-444. DOI: https://doi.org/10.1038/nature14539
[9]ZHAO R, YAN R Q, CHEN Z H, et al. Deep learning and its applications to machine health monitoring. Mechanical Systems and Signal Processing, 2019, 115: 213-237. DOI: https://doi.org/10.1016/j.ymssp.2018.05.050
[10]HOCHREITER S, SCHMIDHUBER J. Long short-term memory. Neural Computation, 1997, 9(8): 1735-1780. DOI: https://doi.org/10.1162/neco.1997.9.8.1735
[11]ZHENG S, RISTOVSKI K, FARAHAT A, et al. Long short-term memory network for remaining useful life estimation//2017 IEEE International Conference on Prognostics and Health Management. Piscataway, NJ: IEEE, 2017: 88-94. DOI: https://doi.org/10.1109/ICPHM.2017.7998311
[12]PROKHORENKOVA L, GUSEV G, VOROBEV A, et al. CatBoost: unbiased boosting with categorical features. Advances in Neural Information Processing Systems, 2018, 31: 6638-6648.
[13]GORISHNIY Y, RUBACHEV I, KHRULKOV V, et al. Revisiting deep learning models for tabular data. Advances in Neural Information Processing Systems, 2021, 34: 18932-18943.
[14]KENDALL A, GAL Y, CIPOLLA R. Multi-task learning using uncertainty to weigh losses for scene geometry and semantics//IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2018: 7482-7491. DOI: https://doi.org/10.1109/CVPR.2018.00781
[15]MA J, ZHAO Z, YI X, et al. Modeling task relationships in multi-task learning with multi-gate mixture-of-experts//Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. New York, NY: ACM, 2018: 1930-1939. DOI: https://doi.org/10.1145/3219819.3220007
[16]ZHANG C, ZHANG X, WANG Y, et al. Multi-task learning for machinery fault diagnosis and remaining useful life prediction: A review. IEEE Transactions on Instrumentation and Measurement, 2023, 72: 1-15.
[17]ARIK S O, PFISTER T. TabNet: Attentive interpretable tabular learning//AAAI Conference on Artificial Intelligence. Menlo Park, CA: AAAI Press, 2021: 6679-6687. DOI: https://doi.org/10.1609/aaai.v35i8.16826
[18]RAISSI M, PERDIKARIS P, KARNIADAKIS G E. Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational Physics, 2019, 378: 686-707. DOI: https://doi.org/10.1016/j.jcp.2018.10.045
[19]LI X, ZHANG W, MA H, et al. Transformer-based fault diagnosis and remaining useful life prediction for rotating machinery: A review. Knowledge-Based Systems, 2024, 292: 111612.
[20]MO H, ZUO M J, WANG D. Remaining useful life estimation using a temporal convolutional network with attention mechanism. IEEE Transactions on Industrial Electronics, 2023, 70(10): 10458-10467.
[21]VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need. Advances in Neural Information Processing Systems, 2017, 30: 5998-6008.
[22]HUANG X, KHETAN A, CVITKOVIC M, et al. TabTransformer: Tabular data modeling using contextual embeddings[Z/OL]. arXiv:2012.06678, 2020.
[23]SOMEPALLI G, GOLDBLUM M, SCHWARZSCHILD A, et al. Saint: Improved neural networks for tabular data via row attention and contrastive pre-training[Z/OL]. arXiv:2106.01342, 2021.
[24]POPOV S, MOROZOV S, BABENKO A. Neural oblivious decision ensembles for deep learning on tabular data//International Conference on Learning Representations, 2019.
[25]FINK O, ZIO E, WEIDMANN U. A framework for prognostic uncertainty assessment based on multiple degradation process characteristics. IEEE Transactions on Reliability, 2014, 63(2): 439-448.
[26]KARNIADAKIS G E, KEVREKIDIS I G, LU L, et al. Physics-informed machine learning. Nature Reviews Physics, 2021, 3(6): 422-440. DOI: https://doi.org/10.1038/s42254-021-00314-5
[27]WILLARD J, JIA X, XU S, et al. Integrating physics-based modeling with machine learning: A survey[Z/OL]. arXiv:2003.04919, 2020.
[28]WANG H, XU C, LIU Y, et al. Physics-informed neural networks for machine fault diagnosis and remaining useful life prediction: A review. Reliability Engineering & System Safety, 2024, 245: 109965.
##submission.downloads##
已出版
##submission.dataAvailability##
Data are available from the corresponding author upon reasonable request.##submission.license##
##submission.copyrightStatement##
##submission.license.cc.by4.footer##