An environment-adaptive multi-modal perception edge-AI for real

Abstract Deploying high-fidelity deep learning models for robotic harvesting on edge platforms creates a fundamental conflict: the inference latency often exceeds stringent real-time control deadlines, resulting in the critical "stale data" problem. We present a cohesive edge computing system that resolves this conflict through two core contributions. First, we introduce M2PE, a high-fidelity perception engine that achieves an accuracy of 97.4% mAP and 0.8 cm 3D MAE via novel cross-modal attention fusion and direct point-cloud localization. Second, we propose LCOS-F2, a non-invasive latency-constrained output selection framework that guarantees data freshness. LCOS-F2 employs a closed-loop architecture that (i) uses controller feedback to dynamically adjust time budgets based on task urgency (e.g., searching vs. grasping), and (ii) leverages a graduated fallback portfolio to provide deterministic Quality-of-Service (QoS) when deadlines are tight. A post-hoc temporal arbiter ensures that the controller always receives the freshest valid estimate, effectively decoupling perception latency from control frequency. Deployed on an NVIDIA Jetson Orin Nano, our system achieves a 0% Deadline-Miss Rate (DMR) across all dynamic budget scenarios while maximizing utilized accuracy, demonstrating a robust paradigm for reliable real-time edge perception. Data Availability The data supporting the findings of this study were obtained from field experiments conducted by our laboratory in cooperation with local farms. These datasets are available from the corresponding author upon reasonable request. References Kootstra G, Wang X et al (2021) Selective harvesting robotics: Current research, trends, and future directions. Curr Robot Rep 2(2):95–104 Kang H, Zhou C et al (2020) Fruit detection, segmentation and 3d visualisation of apples in orchard environment. Comput Electron Agric 171:105302. https://doi.org/10.1016/j.compag.2020.105302 Hu T, Zhang X et al (2023) Research on apple object detection and localization based on deep learning and rgb-d. Agronomy 13(7):1816. https://doi.org/10.3390/agronomy13071816 Fang T, Li Y et al (2025) Location research and picking experiment of an apple-picking robot based on improved mask r-cnn and binocular vision. Horticulturae 11(7):801. https://doi.org/10.3390/horticulturae11070801 Hou C, Han H et al (2022) Detection and localization of citrus fruit based on improved yolov5 and binocular vision. Front Plant Sci 13:1111760 Kang S, Kim J (2024) Design, integration, and field evaluation of a selective broccoli harvesting robot. Comput Electron Agric Surantha N, Soeprapto A et al (2025) Key considerations for real-time object recognition on edge ai. Appl Sci 15(2):868 Kim J, Choi Y (2025) Real-time object detection for edge computing-based agricultural automation: A case study. Sensors Anonymous: deep learning and edge computing in agriculture: a comprehensive review of recent trends and innovations Agarwal S, Vora A (2023) A Scalable Multi-robot framework for decentralized and asynchronous perception-action-communication.arxiv: 2303.08056 Deng S, Guan X et al (2022) Coorp: Satisfying low-latency and high-throughput requirements of wireless network for coordinated robotic learning. IEEE Internet Things J 9(18):17156–17168 Soyyigit A, Yao S, Yun H (2022) Anytime-LiDAR: deadline-aware 3D object detection Soyyigit A, Yao S, Yun H (2024) Valo: A versatile anytime framework for lidar-based object detection dnns. In: Proceedings of EMSOFT 2024 https://doi.org/10.23919/EMSOFT60938.2024.10665490 . Also available as arXiv:2407.04253 Gehrig D, Hidalgo-Carri’o, J, Gehrig M, Scaramuzza D (2024) Low-latency automotive vision with event cameras. Nature 628. https://doi.org/10.1038/s41586-024-07409-w Hu J, Fan C, Wang Z, Ruan J, Wu S (2023) Fruit detection and counting in apple orchards based on improved yolov7 and multi-object tracking methods. Sensors 23(13):5903. https://doi.org/10.3390/s23135903 Sun Q, Chai X, Zeng Z, Zhou G, Sun T (2022) Noise-tolerant rgb-d feature fusion network for outdoor fruit detection. Comput Electron Agric 203:107457 Scoca V, Rodrigues A, Santos MY, Bernardino J (2018) Scheduling latency-sensitive applications in edge computing. In: Proceedings of the 8th international conference on cloud computing and services science (CLOSER 2018), pp. 229–236. SciTePress Hosseinzadeh M, Rahmani AM, Sahhaf S, Davy A, Jennings B (2021) Optimal accuracy-time trade-off for deep learning services in edge computing systems. IEEE International Conference on Communications (ICC). pp 1–7. https://doi.org/10.1109/ICC42927.2021.9500573 Zhang W, Zhao C, Pan Z, Wang X, Xu C, Zhang L (2020) Accelerating on-device dnn inference during service interaction. Pervasive Mob Comput 66:101204 Gao L, Peng L, Zeng X, Chen X, He Z (2023) A novel predictive model for edge computing resource scheduling. J Supercomput 79(8):8351–8375 Wang Z, Wu Y, Xu C, Zhang X, Xu C, Wang YC, Lin FX (2019) Scheduling early exit for mobile dnn inference, pp. 607–609. ACM Pereira J, Dias T, Carvalho T, Calisto F, Gonçalves D (2023) Inferencing on edge devices: A time- and space-aware scheduling infrastructure for embedded platforms with gpu accelerators. ACM Trans Embed Comput Syst (TECS) 22(6):91 Zhang R, Yu H, Shen L, Mi N, Ren K (2023) BCEdge: SLO-Aware DNN inference services with adaptive batching and concurrent execution on edge platforms. https://openreview.net/forum?id=2dE4DqQwMi. Preprint; NeurIPS 2023 Workshop on ML for Systems Xu X, et al (2021) A Co-scheduling framework for DNN models on mobile platforms. Project/technical report; add formal publication if available Hudson J, Ward J, Radecka K, Mendes P, Marshall A, Portilla J, Armendáriz-Iñigo JE (2021) QoS-aware placement of deep learning services on the edge with multiple service implementations Kaur H, Khanna R, Sharma V, Panesar GS (2025) A survey of advancements in scheduling techniques for efficient deep learning computations on gpus. Electronics 14(4):823 Wang X, Tang Z, Guo J, Meng T, Wang C, Wang T, Jia W (2025) Empowering edge intelligence: a comprehensive survey on on-device AI models. Comprehensive survey on on-device/edge AI models. https://arxiv.org/abs/2503.06027 Kapach K, Barnea E, Mairon R, Edan Y, Shahar O (2012) Computer vision for fruit harvesting and yield estimation: A review. Comput Electron Agric 81:15–34. https://doi.org/10.1016/j.compag.2011.09.004 Gongal A, Amatya S, Karkee M, Zhang Q, Lewis K (2015) Sensors and systems for fruit detection and localization: A review. Comput Electron Agric 116:8–19 Bac CW, Hemming J, Henten EJ (2014) Harvesting robots in agriculture and horticulture. J Field Robot 31(6):888–911. https://doi.org/10.1002/rob.21525 Bulanon DM, Burks TF, Alchanatis V (2009) Image fusion of visible and thermal images for fruit detection. Biosys Eng 103(1):12–22 Linker R, Cohen O, Naor A (2012) Determination of the number of green apples in rgb images recorded in orchards. Comput Electron Agric 81:98–107 Silwal A, Gongal A, Karkee M, Zhang Q (2014) Identification of red apples in field environment with over-the-row machine vision system. Agric Eng Int CIGR J 16(4):66–75 Stajnko D, Lakota M, Hočevar M (2004) Estimation of apple fruit number and diameter using thermal imaging. Comput Electron Agric 42(1):31–42 Henten EJ, Hemming J, Tuijl BAJ, Kornet JG, Meuleman J, Bontsema J, Os EA (2002) An autonomous robot for harvesting sweet-pepper in greenhouses. In: Proceedings of the International Conference of Agricultural Engineering (AgEng), Budapest, Hungary Early seminal work; add publisher/pages if needed Si Y, He D, Zhou K (2015) Two-stage apple recognition approach using stereovision. Comput Electron Agric 114:46–53 Author information Authors and Affiliations Contributions All authors contributed to the study conception and design. Supervision was performed by J.F.W. and D.Y.C. Material preparation, data collection and analysis were performed by C.J.C. All authors read and approved the final manuscript. Corresponding author Ethics declarations Competing interests The authors declare no competing interests. Additional information Publisher's Note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. Rights and permissions Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law. About this article Cite this article Wang, JF., Chan, DY. & Chang, CJ. An environment-adaptive multi-modal perception edge-AI for real-time robotic fruit harvesting. J Supercomput 82, 693 (2026). https://doi.org/10.1007/s11227-026-08829-3 Received: Accepted: Published: Version of record: DOI: https://doi.org/10.1007/s11227-026-08829-3