A multi-backbone framework with architecture-adaptive pruning and gated feature fusion for driver behavior recognition

Abstract Driver behavior recognition (DBR) is an important component of intelligent driving systems because it enables continuous monitoring of unsafe driver actions in complex environments. However, multi-backbone DBR models often provide improved representation capability at the cost of high computational complexity, while compression may introduce feature imbalance and unstable optimization. To address these issues, this study proposes a Multi-Backbone Adaptive Pruning and Fusion (MBAPF) framework. A Backbone-Aware Adaptive Pruning (BAAP) method compresses backbones using architecture-specific pruning granularities, where “adaptive” refers to adaptation to backbone structures rather than automatically learned pruning ratios. A Gated Multi-Backbone Feature Fusion (G-MBFF) mechanism then performs sample-dependent calibration of the compressed backbone features, and a Progressive Fine-tuning Strategy (PFS) stabilizes joint optimization by gradually expanding the trainable parameter scope. On the driver-independent SAA13 test set, MBAPF achieves 92.61% accuracy, 93.18% precision, 92.72% recall, and a 92.95% F1-score. Compared with the original multi-backbone fusion model, MBAPF reduces the parameter count from 74.334 to 54.312 M and GFLOPs from 12.873 to 8.279 G, while achieving a measured throughput of 27.5 FPS. Leave-one-dataset-out experiments further evaluate generalization to unseen acquisition domains. These results demonstrate that MBAPF provides a favorable trade-off among recognition performance, model complexity, and inference efficiency under the evaluated desktop GPU environment. Data availability No datasets were generated or analyzed during the current study. References Distracted driving in 2021, document Note DOT HS 813 443, Research, National Highway Traffic Safety Administration, National Center for Statics and Analysis, 2023. Global status report on road safety 2023, document Licence: CC BY-NC-SA 3.0 IGO, World Health Organization, Geneva, Switzerland, Safety and Mobility (SAM), Social Determinants of Health (SDH) 81, 2023. Li G, Wang G, Guo Z et al (2024) Domain adaptive driver distraction detection based on partial feature alignment and confusion-minimized classification. IEEE Trans Intell Transp Syst 25(9):11227–11240. https://doi.org/10.1109/TITS.2024.3367665 Ma Y, Du R, Abdelraouf A, Han K, Gupta R, Wang Z (2024) Driver digital twin for online recognition of distracted driving behaviors. IEEE T Intell Veh 9(2):3168–3180. https://doi.org/10.1109/TIV.2024.3353253 Mohsin AR, Khalid M, Ali J, Roh BH (2026) Enhancing driver safety with ResNet-RG-A deep learning-based distraction detection approach. IEEE Sens J 26(3):4664–4683. https://doi.org/10.1109/JSEN.2025.3640661 Chai W, Wang J, Chen J, Velipasalar S, Sharma A (2024) Rethinking the evaluation of driver behavior analysis approaches. IEEE Trans Intell Transp Syst 25(8):9958–9966. https://doi.org/10.1109/TITS.2024.3354506 Rampavan M, Ijjina EP (2025) An improved genetic algorithm based deep learning model with you only look once framework for driver distraction detection. Eng Appl Artif Intell. https://doi.org/10.1016/j.engappai.2025.111932 Hasan MZ, Chen J, Wang J et al (2024) Vision-language models can identify distracted driver behavior from naturalistic videos. IEEE Trans Intell Transp Syst 25(9):11602–11616. https://doi.org/10.1109/TITS.2024.3381175 Guo ZZ, Liu Q, Li GF, Luo XY, Wang GL, Olaverri-Monreal C (2025) A dynamic multi-level feature alignment method for domain adaptive driver distraction detection. IEEE Trans Intell Transp Syst. https://doi.org/10.1109/TITS.2025.3624514 Gao H, Hu MT, Yu KY, Xing H (2026) Attention distillation for low-cost driver behavior recognition. Appl Soft Comput. https://doi.org/10.1016/j.asoc.2025.114398 Li JF, Sun JJ, Li ZQ, Zhang J, Zhuo L (2026) Multimodal driver behavior recognition based on frame-adaptive convolution and feature fusion. Comput Vis Image Underst. https://doi.org/10.1016/j.cviu.2025.104587 Chen J, Zhang Q, Chen J et al (2024) A driving risk assessment framework considering driver’s fatigue state and distraction behavior. IEEE Trans Intell Transp Syst 25(12):20120–20136. https://doi.org/10.1109/TITS.2024.3446832 Huo R, Chen J, Zhang Y, Gao Q (2025) 3D skeleton aware driver behavior recognition framework for autonomous driving system. Neurocomputing. https://doi.org/10.1016/j.neucom.2024.128743 Mohsin AR, Khalid M, Ali J, Haidery SA, Roh B (2025) An efficient temporal attention framework for real-time detection of unsafe driver behaviors. Alex Eng J 133:318–328. https://doi.org/10.1016/j.aej.2025.11.035 Chen J, Zhang Z, Yu J et al (2025) DSDFormer: an innovative transformer-mamba framework for robust high-precision driver distraction identification. IEEE Trans Intell Transp Syst. https://doi.org/10.1109/TITS.2025.3625645 Uddin MA, Hossain N, Ahamed A et al (2025) Abnormal driving behavior detection: a machine and deep learning based hybrid model. Int J Intell Transp Syst Res 23(1):568–591. https://doi.org/10.1007/s13177-025-00471-2 Sun Y, Zhou M, Cai F, Yang X, Li C, Yi Y (2025) Noncontact multimodal sensing for driver distraction detection via behavioral-physiological fusion. IEEE Sens J 25(20):38576–38584. https://doi.org/10.1109/JSEN.2025.3608125 Lv M, Liu Y, Zha Z et al (2025) Si-CA MobileNet: a lightweight and efficient convolutional neural network for distracted driver detection. Neurocomputing 654:131281. https://doi.org/10.1016/j.neucom.2025.131281 Gu S, Wen B, Chen S et al (2025) Driver distraction detection based on adaptive tiny targets and lightweight networks. Signal Process Image Commun 138:10. https://doi.org/10.1016/j.image.2025.117342 Zhao X, Li Z, Ma Y, Cai Y, Sun X, Chen L (2025) Beyond classification and regression: a novel multi-task deep learning framework for driver state understanding in human-machine co-driving system. Knowledge-Based Syst 322:17. https://doi.org/10.1016/j.knosys.2025.113777 Deng W, Zhao C, Shen P, Zhang Z (2025) Driver behavior recognition in complex driving scenarios via multicollinear fusion network with optimizations. IEEE Sens J 25(8):13300–13315. https://doi.org/10.1109/JSEN.2025.3541769 Deng W, Zhao C, Zhang Z (2025) Driver behavior recognition in complex driving scenarios via deep attention network with geometric-spatial fusion features. Transp Res Rec 2680(5):395–417. https://doi.org/10.1177/03611981251378493 Deng W, Zhao C, Zhang Z (2026) Generalizing driver behavior recognition in complex driving scenarios via hierarchical attention network with hybrid-split strategy. J Transp Eng Part A: Syst. https://doi.org/10.1061/JTEPBS.TEENG-9339 Deng W, Zhao C, Ma X, Zhang Z, Wang J (2026) Pose-guided multilevel fusion network with conditional attention for driver-behavior recognition in complex driving scenarios. Transp Res Rec. https://doi.org/10.1177/03611981261425979 Gao H, Liu Y (2023) Improving real-time driver distraction detection via constrained attention mechanism. Eng Appl Artif Intell. https://doi.org/10.1016/j.engappai.2023.107408 Li T, Li X, Ren B, Guo G (2024) An effective multi-scale framework for driver behavior recognition with incomplete skeletons. IEEE Trans Veh Technol 73(1):295–309. https://doi.org/10.1109/TVT.2023.3308566 Kuang J, Li W, Li F, Zhang J, Wu Z (2024) MIFI: multi-camera feature integration for robust 3D distracted driver activity recognition. IEEE Trans Intell Transp Syst 25(1):338–348. https://doi.org/10.1109/TITS.2023.3304317 Zhang K, Wang S, Jia N, Zhao L, Han C, Li L (2024) Integrating visual large language model and reasoning chain for driver behavior analysis and risk assessment. Accid Anal Prev. https://doi.org/10.1016/j.aap.2024.107497 Yang X, Qiao Y, Han S, Feng Z, Chen Y (2024) Appearance-posture fusion network for distracted driving behavior recognition. Expert Syst Appl. https://doi.org/10.1016/j.eswa.2024.124883 Liao C, Lin K (2025) DDC-Chat: achieving accurate distracted driver classification through instruction tuning of visual language model. J Safety Sci Resilience 6(2):250–264. https://doi.org/10.1016/j.jnlssr.2024.10.001 Li R, Yu C, Qin X et al (2024) YOLO-SGC: a dangerous driving behavior detection method with multiscale spatial-channel feature aggregation. IEEE Sens J 24(21):36044–36056. https://doi.org/10.1109/JSEN.2024.3457686 Duan C, Liu Z, Xia J, Zhang M, Liao J, Cao L (2025) Enhancing cross-dataset performance of distracted driving detection with score softmax classifier and dynamic gaussian smoothing supervision. IEEE Trans Intell Veh 10(1):282–295. https://doi.org/10.1109/TIV.2024.3412198 Li P, Yang Y, Grosu R et al (2022) Driver distraction detection using octave-like convolutional neural network. IEEE Trans Intell Transp Syst 23(7):8823–8833. https://doi.org/10.1109/TITS.2021.3086411 Wang H, Chen J, Huang Z et al (2023) FPT: fine-grained detection of driver distraction based on the feature pyramid vision transformer. IEEE Trans Intell Transp Syst 24(2):1594–1608. https://doi.org/10.1109/TITS.2022.3219676 Yang H, Liu H, Hu Z, Nguyen A, Guerra T, Lv C (2024) Quantitative identification of driver distraction: a weakly supervised contrastive learning approach. IEEE Trans Intell Transp Syst 25(2):2034–2045. https://doi.org/10.1109/TITS.2023.3316203 Li Z, Zhao X, Wu F, Chen D, Wang C (2024) A lightweight and efficient distracted driver detection model fusing convolutional neural network and vision transformer. IEEE Trans Intell Transp Syst 25(12):19962–19978. https://doi.org/10.1109/TITS.2024.3447041 Liu Y, Du S, Han H, Chen X, Zeng W, Tian Z (2024) Adaptive distraction recognition via soft prototype learning and probabilistic label alignment. IEEE Trans Intell Transp Syst 25(11):18701–18713. https://doi.org/10.1109/TITS.2024.3444006 Wang C, Lin M, Shao L, Xiang J (2024) MF-YOLO: a lightweight method for real-time dangerous driving behavior detection. IEEE Trans Instrum Meas. https://doi.org/10.1109/TIM.2024.3472868 Gong S, Fang Y, Deng X, Paul A, Wu Z, Long W (2025) Radar-based fatigue driving action recognition using the combination of micro-Doppler time-frequency and energy features. IEEE Sens J 25(13):26207–26219. https://doi.org/10.1109/JSEN.2025.3572711 Zhang Y, Li T, Li C, Zhou X (2024) A novel driver distraction behavior detection method based on self-supervised learning with masked image modeling. IEEE Internet Things J 11(4):6056–6071. https://doi.org/10.1109/JIOT.2023.3308921 Guo Z, Liu Q, Zhang L, Li Z, Li G (2024) L-TLA: a lightweight driver distraction detection method based on three-level attention mechanisms. IEEE Trans Reliab 73(4):1731–1742. https://doi.org/10.1109/TR.2023.3348951 Mohammed AAQ, Geng X, Wang J, Ali Z (2024) Driver distraction detection using semi-supervised lightweight vision transformer. Eng Appl Artif Intell. https://doi.org/10.1016/j.engappai.2023.107618 Al-Mahbashi M, Li G, Peng Y, Al-Soswa M, Debsi A (2025) Real-time distracted driving detection based on GM-YOLOv8 on embedded systems. J Transp Eng Pt A-Syst 151(3):18. https://doi.org/10.1061/JTEPBS.TEENG-8681 Yang L, Wei H, Hu Z, Lv C (2025) A domain generalization method for deploying driver distraction detection models to practical application scenarios. Eng Appl Artif Intell 152:13. https://doi.org/10.1016/j.engappai.2025.110844 Sun H, Feng N (2026) "MVim : A high-accuracy, lightweight neural network for real-time driver behavior recognition. Expert Syst Appl. https://doi.org/10.1016/j.eswa.2025.129091 Shen Q, Zhang L, Zhang Y, Zhang Y, Liu S (2025) LEDDR - YOLO : a lightweight and efficient distracted driving recognition algorithm with a particular pruning method. J Supercomput 81(12):33. https://doi.org/10.1007/s11227-025-07709-6 Tang X, Chen Y, Ma Y, Yang W, Zhou H, Huang J (2024) A lightweight model combining convolutional neural network and transformer for driver distraction recognition. Eng Appl Artif Intell. https://doi.org/10.1016/j.engappai.2024.107910 Gao H, Hu M, Liu Y (2024) Learning driver-irrelevant features for generalizable driver behavior recognition. IEEE Trans Intell Transp Syst 25(10):14115–14127. https://doi.org/10.1109/TITS.2024.3396640 Hao J, Sun X, Liu X, Hua D, Hu J (2025) A lightweight and explainable model for driver abnormal behavior recognition. Eng Appl Artif Intel. https://doi.org/10.1016/j.engappai.2024.109559 Liu S, You L, Zhu R et al (2024) AFM3D: an asynchronous federated meta-learning framework for driver distraction detection. IEEE Trans Intell Transp Syst 25(8):9659–9674. https://doi.org/10.1109/TITS.2024.3357138 Zhou W, Jia Z, Feng C, Lu H, Lyu F, Li L (2024) Towards driver distraction detection: a privacy-preserving federated learning approach. Peer Peer Netw Appl 17(2):896–910. https://doi.org/10.1007/s12083-024-01639-5 Chillakuru P, Ananthajothi K, Divya D (2024) Three stage classification framework with ranking scheme for distracted driver detection using heuristic-assisted strategy. Knowl-Based Syst. https://doi.org/10.1016/j.knosys.2024.111589 Liu QC, Chen SQ, Liu GQ et al (2025) Dual-perspective safety driver secondary task detection method based on swin-transformer and cross-attention. Adv Eng Inform. https://doi.org/10.1016/j.aei.2025.103320 Huang C, Wang X, Cao J, Wang S, Zhang Y (2020) HCF: a hybrid CNN framework for behavior detection of distracted drivers. IEEE Access 8:109335–109349. https://doi.org/10.1109/ACCESS.2020.3001159 Yang W, Tan C, Chen Y et al (2023) BiRSwinT: bilinear full-scale residual swin-transformer for fine-grained driver behavior recognition. J Frankl Inst 360(2):1166–1183. https://doi.org/10.1016/j.jfranklin.2022.12.016 Liu Z, Mao H, Wu C et al (2022) A ConvNet for the 2020s, In: 2022 IEEE/CVF conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, pp. 11966–11976. Xie S, Girshick R, Dollar P, Tu Z, He K (2017), Aggregated Residual Transformations for Deep Neural Networks, In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, pp. 5987–5995. He K, Zhang X, Ren S, Sun J (2016), Deep Residual Learning for Image Recognition, In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, pp. 770–778. Kaggle (2024) State Farm Distracted Driver Detection. Available [Online]: https://www.kaggle.com/competitions/statefarmdistracted-driver-detection/data. Accessed 15 Aug 2024 Saad MH, Khalil MI, Abbas HM (2020), End-to-end Driver Distraction Recognition Using Novel Low Lighting Support Dataset, in 2020 15th International Conference on Computer Engineering and Systems (ICCES), Cairo, Egypt, pp. 1–6. Eraqi HM, Abouelnaga Y, Saad MH, Moustafa MN (2019) Driver distraction identification with an ensemble of convolutional neural networks. J Adv Transp. https://doi.org/10.1155/2019/4125865 Wang J, Li W, Li F et al (2023) 100-Driver: a large-scale, diverse dataset for distracted driver classification. IEEE Trans Intell Transp Syst 24(7):7061–7072. https://doi.org/10.1109/TITS.2023.3255923 Jegham I, Ben Khalifa A, Alouani I, Mahjoub MA (2020) A novel public dataset for multimodal multiview and multispectral driver distraction analysis: 3MDAD. Signal Proces Image Commun. https://doi.org/10.1016/j.image.2020.115960 Yu W, Zhou P, Yan S, Wang X (2024), InceptionNeXt: When Inception Meets ConvNeXt, In: 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, pp. 5672–5683. Szegedy C, Vanhoucke V, Ioffe S, Shlens J, Wojna Z (2016), Rethinking the Inception Architecture for Computer Vision, In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, pp. 2818–2826. Chollet F (2017), Xception: Deep Learning with Depthwise Separable Convolutions, In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, pp. 1800–1807. Sandler M, Howard A, Zhu M, Zhmoginov A, Chen L, (2018), MobileNetV2: Inverted Residuals and Linear Bottlenecks, In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, pp. 4510–4520. Landola FN, Han S, Moskewicz MW, Ashraf K (2016), SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size, http://arxiv.org/abs/1602.07360. Han K, Wang Y, Tian Q, Guo J, Xu C, Xu C (2020), GhostNet: More Features From Cheap Operations, In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, pp. 1577–1586. Xiao W, Liu H, Ma Z, Chen W (2022) Attention-based deep neural network for driver behavior recognition. Futur Gener Comp Syst 132:152–161. https://doi.org/10.1016/j.future.2022.02.007 Baheti B, Talbar S, Gajre S (2020) Towards computationally efficient and realtime distracted driver detection with MobileVGG network. IEEE T Intell Veh 5(4):565–574. https://doi.org/10.1109/TIV.2020.2995555 Xiao W, Xie G, Liu H, Chen W, Li R (2024) FDAN: Fuzzy deep attention networks for driver behavior recognition. J Syst Architect. https://doi.org/10.1016/j.sysarc.2023.103063 Wang M, Deng W, Zhao C, Du Q, Zhao M (2026) A keypoint-assisted deep attention network with adaptive weighting for distracted driving behaviour detection. IET Intell Transp Syst. https://doi.org/10.1049/itr2.70212 Liu Z, Li JG, Shen ZQ et al. (2017), Learning Efficient Convolutional Networks Through Network Slimming, In: 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy, pp. 2755–2763. He Y, Liu P, Wang ZW, Hu ZL, Yang Y (2019), Filter Pruning via Geometric Median for Deep Convolutional Neural Networks Acceleration, In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, pp. 4335–4344. Lin MB, Ji RR, Wang Y et al. (2020), HRank: Filter pruning using high-rank feature map, In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1526–1535. Acknowledgements This work was supported by the National Key R&D Program of China (Grant No. 2022YFB4300300) and the Joint Research Project of Tai’an Dongxin Zhilian Information Technology Co., Ltd. and Southeast University (Grant No. DX202506130302). The authors gratefully acknowledge this support. Author information Authors and Affiliations Contributions Wenhao Deng helped in data curation, methodology, writing—original draft; Chihang Zhao helped in conceptualization, funding acquisition, supervision, and project administration; Xinyi Ma helped in formal analysis and validation; Jinzhao Liu helped in investigation and visualization; and Junjun Wang helped in resources and writing—review and editing. All authors reviewed the results and approved the final version of the manuscript. Corresponding author Ethics declarations Conflict of interest The authors declare that they have no competing interests. All authors have read and approved the submitted manuscript. The manuscript has not been published previously and is not under consideration for publication elsewhere. Additional information Publisher's Note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. Rights and permissions Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law. About this article Cite this article Deng, W., Zhao, C., Ma, X. et al. A multi-backbone framework with architecture-adaptive pruning and gated feature fusion for driver behavior recognition. J Supercomput 82, 719 (2026). https://doi.org/10.1007/s11227-026-08873-z Received: Accepted: Published: Version of record: DOI: https://doi.org/10.1007/s11227-026-08873-z