The network fabric as a storage orchestrator: a switch

Abstract Telemetry data, essential for operational intelligence and security, is produced in vast quantities by modern cloud data centers. In large-scale enterprise data centers operating hundreds of 100/400 Gbps links, this telemetry stream reaches 50 to 100 Terabytes per rack daily (exceeding 2 to 5 Petabytes per day facility-wide across millions of concurrent flows). This deluge of packets is processed by endpoint server CPUs in traditional storage architectures, resulting in a significant performance bottleneck that restricts visibility. Current methods ignore the network fabric’s computational potential for data orchestration, treating it as a passive collection of pipes. We introduce Synapse, a new switch-centric architecture that turns the network into an orchestrator for active storage. Synapse incorporates intelligent storage logic directly into a programmable switch’s data plane. Bypassing host CPUs entirely, the switch uses Remote Direct Memory Access (RDMA) over RDMA over Converged Ethernet (RoCE) to directly control data placement on a pool of distant Non-Volatile Memory Express (NVMe) SSDs. This design uses policy-driven Quality of Service to safeguard important data streams, autonomous data-plane resilience for microsecond-scale failover and rerouting, and zero-CPU-overhead in-fabric live indexing to make telemetry data query-ready upon ingress. Our analysis demonstrates that in-fabric live indexing accelerates diagnostic queries by up to 300x compared to unindexed scans, while eliminating the severe host CPU overhead (consuming over 90% of multiple dedicated host cores) incurred by software-based indexing baselines, minimizes host CPU overhead to almost zero, and achieves a linearly scalable storage throughput of over 730 Gbps. Microsecond-level failure rerouting is made possible by its data-plane-native resilience, which is five orders of magnitude quicker than conventional controller-based techniques. A new class of highly scalable, effective, and resilient network-native services is made possible by Synapse, which removes the main storage bottleneck in contemporary data centers by offloading the entire telemetry capture process to the network fabric. Data availability No datasets were generated or analyzed during the current study. References Makonyi K, Abrahamsson H, Henriksson D, Hock D, Kremling S, Lipp F, Salisbury J, Sandell J (2024) On the use of streaming telemetry data for network health monitoring and anomaly detection. Swedish National Computer Networking and Cloud Computing Workshop. SNCNW. Putina A, Rossi D, Bifet A, Barth S, Pletcher D, Precup C, Nivaggioli P (2018) Telemetry-based stream-learning of bgp anomalies. In: Proceedings of the 2018 Workshop on Big Data Analytics and Machine Learning for Data Communication Networks. pp 15–20 Ahmadpanah SH, Mirabi M, Sahafi A, Erfani SH (2025) Fleximigrate: enhancing live container migration in heterogeneous computing environments. Clust Comput 28(13):847. https://doi.org/10.1007/s10586-025-05548-x Zhu S, Lu J, Lyu B, Pan T, Zhang S, Sun X, Jia C, Cheng X, Kang D, Lv Y, Yang F, Xue X, Yang X, Wang Z, Yang J (2024) Proactive telemetry in large-scale multi-tenant cloud overlay networks. IEEE/ACM Trans Netw 32:3002–3017. https://doi.org/10.1109/TNET.2024.3381786 Tovarňák D, Racek M, Velan P (2021) Cloud native data platform for network telemetry and analytics. In: 17th International Conference on Network and Service Management (CNSM). pp 394–396. https://doi.org/10.23919/CNSM52442.2021.9615568 Pentaparthi SKR (2025) Observability for ai-enabled cloud-native networks: A unified framework integrating opentelemetry, cortexdb, loki, genai, and rag. Int J Sci Res Comput Sci Eng Inf Technol. https://doi.org/10.32628/cseit25112811 Bhosale P (2025) A comprehensive study of opentelemetry collector: Architecture, use cases, and performance. Int Sci J Eng Manag. https://doi.org/10.55041/isjem00020 Zhang J, Deng X (2021) Ldsqp: Scalable and efficient log data storage and query processing scheme for cloud data centers. In: Proceedings of the 5th International Conference on High Performance Compilation, Computing and Communications. https://doi.org/10.1145/3471274.3471276 Zawoad S, Dutta A, Hasan R (2016) Towards building forensics enabled cloud through secure logging-as-a-service. IEEE Trans Dependable Secure Comput 13:148–162. https://doi.org/10.1109/TDSC.2015.2482484 Simsek, G., Ergenç, D., Onur, E.: Efficient network monitoring via in-band telemetry. In: 2021 17th International Conference on the Design of Reliable Communication Networks (DRCN), 1–6 (2021) https://doi.org/10.1109/DRCN51631.2021.9477344 Kumar, H., Chakraborty, B., Sharma, S., Chawla, N., Mukhopadhyay, S.: Xmd: An expansive hardware-telemetry based malware detector to enhance endpoint detection. ArXiv abs/2206.12447 (2022) https://doi.org/10.48550/arXiv.2206.12447 Cerović D, Piccolo VD, Amamou A, Haddadou K, Pujolle G (2018) Fast packet processing: a survey. IEEE Commun Surv Tutor 20:3645–3676. https://doi.org/10.1109/COMST.2018.2851072 Hauser F, Häberle M, Merling D, Lindner S, Gurevich V, Zeiger F, Frank R, Menth M (2021) A survey on data plane programming with p4: fundamentals, advances, and applied research. J Netw Comput Appl 212:103561. https://doi.org/10.1016/j.jnca.2022.103561 Yazdinejad A, Parizi R, Bohlooli A, Dehghantanha A, Choo K (2020) A high-performance framework for a network programmable packet processor using p4 and fpga. J Netw Comput Appl 156:102564. https://doi.org/10.1016/j.jnca.2020.102564 Hussain M, Cho B (2024) Hptcollector: high-performance telemetry collector. Clust Comput 27:14729–14744. https://doi.org/10.1007/s10586-024-04650-w Pradeep S, Sharma Y, Verma C, Sreeram G, Rao PH (2023) Express data processing on fpga: network interface cards for streamlined software inspection for packet processing. Appl Syst Innov. https://doi.org/10.3390/asi6010009 Zhang Q, Ansari N, Zhu Z (2023) Adaptive smartnic offloading for unleashing the performance of protocol-oblivious forwarding. IEEE Int Things J 10:7303–7314. https://doi.org/10.1109/JIOT.2022.3228796 Kfoury EF, Crichigno J, Bou-Harb E (2021) An exhaustive survey on p4 programmable data plane switches: taxonomy, applications, challenges, and future trends. IEEE Access 9:87094–87155. https://doi.org/10.1109/ACCESS.2021.3086704 Zhang X, Cui L, Tso FP, Jia W (2021) Pheavy: predicting heavy flows in the programmable data plane. IEEE Trans Netw Serv Manag 18:4353–4364. https://doi.org/10.1109/tnsm.2021.3094514 Johnson DS, London JM, Lea M, Durban J (2008) Continuous-time correlated random walk model for animal telemetry data. Ecology 89(5):1208–15. https://doi.org/10.1890/07-1032.1 Liu Z, Niu Z, Shu R, Cheng W, Yuan L, Nelson J, Ports DRK, Cheng P, Xiong Y (2025) Hyperdrive: direct network telemetry storage via programmable switches. IEEE Trans Cloud Comput 13(2):498–511. https://doi.org/10.1109/TCC.2025.3543477 Deri L, Cardigliano A, Fusco F (2013) 10 gbit line rate packet-to-disk using n2disk. In: 2013 IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS). IEEE, pp 441–446 Emmerich P, Pudelko M, Gallenmüller S, Carle G (2017) Flowscope: Efficient packet capture and storage in 100 gbit/s networks. In: 2017 IFIP Networking Conference (IFIP Networking) and Workshops. IEEE, pp 1–9 Zhu H (2020) Data Plane Development Kit (DPDK): A Software Optimization Guide to the User Space-Based Network Applications. CRC Press. ntop: PF_RING: High-speed Packet Capture, Filtering and Analysis. http://www.ntop.org/products/pf_ring/. Accessed: 2025-07-18 (2025) Zhu Y, Kang N, Cao J, Greenberg A, Lu G, Mahajan R, Maltz D, Yuan L, Zhang M, Zhao BY (2015) Packet-level telemetry in large datacenter networks. In: Proceedings of the 2015 ACM Conference on Special Interest Group on Data Communication. pp 479–491 Kim C, Sivaraman A, Katta N, Bas A, Dixit A, Wobker LJ (2015) -band network telemetry via programmable dataplanes, vol 15. pp 1–2 (ACM SIGCOMM) Gupta A, Harrison R, Canini M, Feamster N, Rexford J, Willinger W (2018) Sonata: Query-driven streaming network telemetry. In: Proceedings of the 2018 Conference of the ACM Special Interest Group on Data Communication. pp 357–371 Liu Z, Manousis A, Vorsanger G, Sekar V, Braverman V (2016) One sketch to rule them all: Rethinking network flow monitoring with univmon. In: Proceedings of the 2016 ACM SIGCOMM Conference. pp 101–114 Langlet J, Ben Basat R, Oliaro G, Mitzenmacher M, Yu M, Antichi G (2023) Direct telemetry access. In: Proceedings of the ACM SIGCOMM 2023 Conference. pp 832–849 Mu, Y., Yao, K., Li, Y., Li, Z., Sun, T., Lu, L., He, J., Huang, M.: Sosp: A smartnic-based offloading framework for cloud storage pooling. In: Proceedings of the 2022 9th International Conference on Wireless Communication and Sensor Networks (2022) https://doi.org/10.1145/3514105.3514110 Kfoury EF, Choueiri S, Mazloum A, AlSabeh A, Gomez J, Crichigno J (2024) A comprehensive survey on smartnics: architectures, development models, applications, and research directions. IEEE Access 12:107297–107336. https://doi.org/10.1109/ACCESS.2024.3437203 Basu S, Nadig D (2024) Offloading nvme over fabrics (nvme-of) to smartnics on an at-scale distributed testbed. In: IEEE 10th International Conference on Network Softwarization (NetSoft). pp 316–318. https://doi.org/10.1109/NetSoft60951.2024.10588915 Kim, D., Liu, Z., Zhu, Y., Kim, C., Lee, J., Sekar, V., Seshan, S.: Tea: Enabling state-intensive network functions on programmable switches. In: Proceedings of the Annual Conference of the ACM Special Interest Group on Data Communication on the Applications, Technologies, Architectures, and Protocols for Computer Communication, pp. 90–106 (2020) Zhu, H., Wang, T., Hong, Y., Ports, D.R., Sivaraman, A., Jin, X.: \(\{\)NetVRM\(\}\): Virtual register memory for programmable networks. In: 19th USENIX Symposium on Networked Systems Design and Implementation (NSDI 22), pp. 155–170 (2022) NVM Express: NVMe-over-Fabrics Specification. https://nvmexpress.org/developers/nvme-of-specification/ . Accessed: 2025-04-05 (2019) Funding There is no funding for this research. Author information Authors and Affiliations Contributions Amir Sahafi was involved in conceptualization, supervision, and project administration. Seyed Hossein Erfani contributed to methodology, resources, and consulting. Seyed Hossein Ahmadpanah was involved in writing—original draft, writing—review and editing, data curation, formal analysis, investigation, validation, visualization, experimentation, and evaluation. Corresponding author Ethics declarations Conflict of interest The authors have no relevant financial or non-financial interests to disclose. The authors declare no conflict of interest. Ethics approval Not applicable. Consent to participate Not applicable. Consent for publication Not applicable. Additional information Publisher's Note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. Rights and permissions Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law. About this article Cite this article Ahmadpanah, S.H., Sahafi, A. & Erfani, S.H. The network fabric as a storage orchestrator: a switch-centric architecture. J Supercomput 82, 699 (2026). https://doi.org/10.1007/s11227-026-08850-6 Received: Accepted: Published: Version of record: DOI: https://doi.org/10.1007/s11227-026-08850-6