Fault injection attack against deep neural network-parameter exposure in random Bits

Abstract Recent studies have revealed that fault injection attacks can expose internal parameters of deep neural networks (DNNs). However, prior efforts to reverse-engineer DNNs through fault injection have been largely theoretical, as flipping even a single bit, particularly the sign bit, is both challenging to execute and verify. To overcome these limitations, we propose an algorithm capable of recovering the bias and weight parameters of DNN output layers, even in the presence of multiple unknown bit flips, including non-sign bits such as exponent bits, caused by fault injection attacks. Specifically, for output layers employing the Softmax activation function, our approach infers secret parameters by analyzing changes in parameter values before and after fault injection. The algorithm was validated on image classification neural networks implemented in Keras using 64-bit floating point representations, with fault injection attacks performed during output layer computations. Results demonstrate that the algorithm can recover output layer parameters with an error rate as low as , even when the sign bit and up to the 8th exponent bit near the least significant bit are randomly flipped. Moreover, this methodology extends to output layers with other activation functions and more complex Convolutional neural networks architectures, revealing a critical security vulnerability in DNNs. 1. Introduction Research on Artificial Intelligence (AI) systems tools to manage vast amounts of information beyond human capability is advancing rapidly. Deep neural networks (DNNs), a core component of AI systems, are designed in diverse architectures based on hidden layer structures and are employed in tasks such as object recognition (He, Zhang et al., Citation2015) and machine translation (Brown et al., Citation2020). Additionally, DNNs play a crucial role in security-sensitive applications, including medical image analysis (Kaissis et al., Citation2020), malware detection (Dahl and Stokes, Citation2013), and autonomous vehicles (Grigorescu et al., Citation2020). To efficiently process massive datasets and achieve high task accuracy, DNN architectures have grown increasingly complex. This evolution has led to the adoption of GPUs (Graphics Processing Units) and AI accelerators to expedite the computation of large networks by leveraging parallel processing capabilities (Chen et al., Citation2014; Tajik & Ganji, Citation2022). Recent advancements in deep learning have also facilitated the compression of DNNs through weight quantisation techniques (Zhou et al., Citation2016), enabling AI technology on resource-constrained mobile devices. Such advancements allow integration into smart IoT (Internet of Things) devices for use in environments closely connected to users. To efficiently safeguard end devices that utilise AI for processing sensitive information, such as personal data, it is crucial to address security concerns associated with neural network structure, a fundamental component of these systems. However, such devices are often deployed in environments where they are easily accessible to users or potential attackers, rendering them vulnerable to physical attacks on hardware, including fault injection attacks. Fault injection attacks involve introducing physical disruptions to a device to impair its operation or modify values in its memory. These attacks can be broadly categorised into hardware-based methods, such as inducing glitches via power fluctuations (Endo et al., Citation2011), injecting electromagnetic waves (Schmidt & Hutter, Citation2007), or using laser beams (Roscian and Sarafianos, Citation2013; Vasselle et al., Citation2018), and software-based techniques, such as the rowhammer attack (Kim et al., Citation2014), which repeatedly accesses memory. Historically, these attacks have predominantly targeted devices running cryptographic algorithms, aiming to expose secret keys (Boneh and DeMillo, Citation1997). However, as AI technology becomes more prevalent, several studies have revealed fault injection vulnerabilities in DNN systems. For instance, adversarial input attacks—where subtle, imperceptible triggers are embedded into inputs to disrupt neural network functionality—have been demonstrated (Goodfellow et al., Citation2014; Ren and Huang, Citation2021). Additionally, the DRAM (Dynamic Random Access Memory), widely employed for storing large DNN weights and enhancing computational throughput (Chen et al., Citation2014; Rakin and He, Citation2019), has emerged as a critical attack. Research has shown that memory bits in DRAM can be flipped using the row hammer attack (Kim et al., Citation2014), leading to the rise of adversarial weight attacks. These attacks manipulate DNN weights to evaluate the network’s resilience, gathering extensive attention. Most adversarial weight attacks, included by fault injection attacks focus on evaluating the robustness of neural networks based on the occurrence of image misclassification occurs (Hou et al., Citation2021; Liu and Wei, Citation2017; Rakin and He, Citation2019; Rakin et al., Citation2020; Rakin et al., Citation2021). However, assessing whether the internal structure or parameters of a trained neural network are leaked is equally critical for safeguarding the neural network. Considerable training costs are invested to achieve high classification accuracy, and if the internal structure of a neural network is compromised, it enables replication of the DNN model, potentially infringing on the Intellectual Property (IP) of AI systems. Previous research has explored reverse engineering of internal structures through techniques such as power analysis and electromagnetic analysis (Batina and Bhasin, Citation2019; Maji et al., Citation2021; Takatoi and Sugawara, Citation2020). In 2021, Breier et al. introduced SNIFF, a method for reverse engineering parameters by exploiting the changes in DNN outputs caused by fault injection attacks. SNIFF conducts fault injection attacks on the output layer of a transfer-learned neural network, proposing a methodology to infer target parameters by analysing the normal and abnormal output values when the sign bit of a weight or bias is flipped. However, this approach relies on a strong attacker assumption: the fault injection attack must precisely flip only the sign bit, and the attacker must accurately determine whether the sign bit of the target parameter has been flipped based on the changes in the DNN’s output. Consequently, if the required sign bit flip does not occur, it becomes challenging to verify whether the neural network parameters have been leaked. Therefore, our paper proposes a methodology for recovering target parameters in a constrained attack scenario, where a fault injection attack is performed on the output layer of a deep neural network utilising a Softmax activation function and subjected to transfer learning. Specifically, we introduce a new algorithm that can extend the applicability of the existing SNIFF technique, regardless of the type of bit error encountered. 1.1. Contribution The main contributions of our paper are as follows: | ▪ | Experimental analysis of bit flips: We investigate the types of bit flips in memory induced by fault injection attacks and establish attacker assumptions critical for reverse engineering neural network parameters. | |||| | ▪ | Introduction of a fixed variable: We propose a fixed variable to compute the ratio of parameters before and after fault injection, enabling the recovery of target parameters in the output layer regardless of the number of flipped bits. | |||| | ▪ | Improved algorithm Parameter Recovery: We present an enhanced algorithm that integrates the previous SNIFF technique with λ, to recover parameters effectively. In addition, the proposed algorithm is experimentally validated through bit flip simulations conducted on a simple toy neural network. | 1.2. Organisation This paper is organised as follows. Section 2 provides an overview of DNNs and Fault Injection Attack. Section 3 discusses fault injection attacks on DRAM and reviews previous research on fault injection attacks targeting DNNs. Sections 4 and 5 outline the objectives of this study and present the proposed methodology for recovering the parameters of DNNs. Section 6 validates the proposed fault injection attack methodology using a toy neural network. Finally, Section 7 concludes the paper and outlines directions for future work. 2. Preliminaries 2.1. DNN 2.1.1. Basic structure and softmax DNN is a mathematical model used for classification or regression in deep learning, designed to emulate a biological neural network. Its architecture consists of an input layer, an output layer, and two or more hidden layers in between, as illustrated in . Each layer consists of multiple neurons interconnected as depicted in . The output of each neuron is computed using the formula given in Equation (Equation1).(1) (1) At this stage, the inputs represent the outputs of all neurons in the preceding layer, are the weights associated with these connections, is the bias of the neuron, and is the activation function. In this paper, the weights and biases are collectively referred to as the parameters of the DNN. DNN comes in various forms depending on the structure of their hidden layers. A fully connected neural network, where all neurons are interconnected, is referred as a MLP (Multi-Layer Perceptron), as illustrated in . In addition, there are specialised architectures such as CNN (Convolutional Neural Network) and RNN (Recurrent Neural Network), among others. As AI technology finds applications across diverse fields, the structure of DNNs has evolved, becoming increasingly complex and larger in scale. To manage the substantial computational demands of DNNs and enable faster inference, these models are often implemented on GPUs or specialised accelerators (Chen et al., Citation2014; Tajik & Ganji, Citation2022). In particular, DRAM has been employed to store DNN parameters efficiently, supporting high-throughput data processing (Jouppi et al., Citation2017; Rakin and He, Citation2019). DNNs designed for classification typically utilise the Softmax function as the activation function in the output layer, which is defined by Equation (Equation2).(2) (2) Here, yi is the number of classification classes. Consequently, each neuron in the output layer has a value between 0 and 1, with the class corresponding to the node with the highest value being the predicted output of the DNN. Since the output value represents the ratio of a specific node’s value to the total value of the last layer nodes, the relative order of the output values mirrors the order of the input values. The outputs and parameters of all neurons in a DNN are represented as real numbers. Deep-learning frameworks, such as Keras, typically use the IEEE-754 standard for floating point arithmetic (Zuras et al., Citation2008). This paper utilises the 64-bit double-precision floating point representation, which consists of a 1-bit sign , an 11-bit exponent , and a 52-bit mantissa , as defined in Equation (Equation3).(3) (3) 2.1.2. Transfer learning Transfer learning is a deep learning approach that leverages the architecture and parameters of a pretrained teacher neural network, which has been trained on extensive datasets, to develop a new student neural network. As shown in , the first layers of the layers in the teacher neural network, along with their parameters, are fixed, while the remaining layers are discarded. The fixed layers are then replicated, and a new trainable layer is appended to the network. The resulting student neural network comprises the newly added layers, which are retrained using a new dataset. Throughout this process, the parameters in the upper layers of the student neural network, inherited from the teacher neural network, remain unchanged. Since the upper layers of the copied teacher neural network are fixed and reused, they can potentially be exposed to an attacker. This exposure enables the attacker to derive fixed outputs for specific inputs. In other words, the input to the newly trained layer in the student neural network can be inferred. This characteristic is leveraged for parameter recovery, as demonstrated in SNIFF (Breier et al., Citation2021) and in this paper. 2.2. Fault injection attack A fault injection attack is a well-known active attack method that extracts secret information by intentionally inducing faults into a device to disrupt its normal operation (Boneh and DeMillo, Citation1997). While traditional used as to retrieve cryptographic keys in algorithms, recent advancements in AI technology and its integration into security-critical domains, such as GPUs and accelerators, have prompted investigations into the vulnerability of neural networks to fault injection attacks. Common fault sources used in these attacks include voltage (Endo et al., Citation2011), electromagnetic waves (Schmidt & Hutter, Citation2007), and laser beams (Roscian and Sarafianos, Citation2013; Vasselle et al., Citation2018). Among these, laser fault injection attack stands out for their high precision, enabling the induction of single-bit error, which makes them particularly effective for targeting SRAM (Static Random Access Memory) (Roscian and Sarafianos, Citation2013; Selmke and Brummer, Citation2016). In contrast, the rowhammer attack (Kim et al., Citation2014; Razavi and Gras, Citation2016), a software-based attack, repeatedly accesses specific DRAM rows to cause bit flips in the targeted row or adjacent memory rows. The resulting memory variations caused by fault injection attacks are referred to as fault models. Depending on the precision of the attack, fault models can be 1) single-bit flip: flip a single bit in a specific register. 2) multiple-bit flip: change the value in a register to a random different value. 3) instruction skip model: Nullifies specific instructions in a compiled function. 3. Related works 3.1. Fault injection on the memory Injecting faults into a system’s memory and flipping stored data bits poses significant risks to its operation and reliability. For example, Roscian et al. demonstrated that laser fault injection can induce bit flips in SRAM (Roscian and Sarafianos, Citation2013; Selmke and Brummer, Citation2016). Particularly, several studies have highlighted that memory bit flips can be induced through rowhammer attacks. Kim et al. showed that bits in DRAM memory can be flipped to modify data via a rowhammer attack (Kim et al., Citation2014). Furthermore, Kaveh et al. proposed an advanced rowhammer attack that selectively flips a single bit by profiling the bit-flip patterns in DRAM beforehand (Razavi and Gras, Citation2016). Given that DNNs store weights and biases in DRAM to accelerate computation and data processing, fault injection into DRAM represents a direct threat to the injection of DNNs. Some studies emphasise the importance of memory protection in smart grids and broadly address false data injection. However, they do not delve into the specific challenges posed by memory fault injection in DNN systems. This underscores the need for further exploration in this area (Ruan, Liang, et al., Citation2023; Ruan, Fan, et al., Citation2023; Ruan, Yang, et al., Citation2023). 3.2. Fault injection in DNNs Fault injection attacks targeting DNNs can affect any component, including the input data, parameter, or activation functions, across various layers; input, hidden, or output (). In 2017, Liu et al. introduced a methodology for adversarial weight and bias attacks using fault injection. They demonstrated that modifying the values of biases or weights could lead to image misclassification when employing the single bias attack and gradient descent attack algorithms (Liu and Wei, Citation2017). Additionally, Breier et al. conducted laser fault injection attacks on commonly used activation functions, revealing that instruction skips could occur, thereby modifying the neural network’s output (Hou et al., Citation2021). Similarly, Jap et al. analysed a single bit-flip model targeting the input to the output layer and described its impact on the neural network’s misclassification rate (Jap et al., Citation2021). In addition, (Rakin and He, Citation2019; Rakin et al., Citation2020, Citation2021) studies have shown that parameter bit flips induced by fault injection attacks can disrupt the functionality of neural networks, leading to critical malfunctions. Some models propose robust frameworks for detecting and classifying cyberattacks using deep learning techniques (Alzubi et al., Citation2022a, Citation2022b; Alzubi et al., Citation2023; Gheisari et al., Citation2021; Movassagh and Alzubi, Citation2023). While these models are effective in identifying and mitigating attacks, they fall short in addressing the unique challenges posed by unpredictable fault injection attacks targeting DNN parameters. Previous research on fault injection attacks in DNNs has primarily focused on techniques to induce misclassification by flipping a minimal number of parameter bits. Although validating a network’s incorrect behaviour is critical for ensuring its safety, safeguarding the confidentiality of the network’s structure and parameters is equally important. If fault injection attacks expose the internal structure or parameters, the entire network can be replicated. While there have been studies on reverse engineering DNNs through power or electromagnetic waveform analysis (Batina and Bhasin, Citation2019; Maji et al., Citation2021; Takatoi and Sugawara, Citation2020), SNIFF, proposed by Breier et al. in 2021, remains the sole technique for reverse engineering DNNs via fault injection attacks. SNIFF is designed to recover parameters of output layer by exploiting sign bit flips caused by fault injections. Their approach successfully recovered output layer parameters with an error rate of using various well-known publicly available CNNs implemented in the Keras library withe 64-bit floating point numbers. 3.3. Limitations of previous works The SNIFF technique for recovering DNN output layer parameters assumes that the attacker only needs to flip the sign bits of the parameters. However, in memory fault injection attacks, the most common method is the rowhammer attack, which requires the attacker to directly read and write data by accessing the memory addresses where the data is stored. Furthermore, accurate single bit flipping often requires memory profiling, which incurs additional costs beyond the attack itself, and access to specific memory addresses may be limited. To address these limitations, we conducted a toy experiment to investigate the types of bits flips that occur in memory when faults are externally injected. Our findings demonstrate that it is possible to recover the parameters under a weaker attacker assumption that is independent of the type of bit flip occurring in memory. 4. Threat model To recover the parameters of a DNN, an attacker requires access to the main memory (i.e. DRAM) where the parameters are stored. However, unlike traditional approaches that rely on direct access memory addresses, such as performing rowhammer attack, our method does not require direct memory access to recover the parameters. Instead, external hardware-based fault injection techniques, such as electromagnetic or laser attacks, are sufficient for this purpose. The targeted neural network for parameter recovery is a student neural network that has undergone transfer learning, similar to the setup in SNIFF. Since all layers of the model, except for the target layer, are publicly accessible, the attacker can replicate all public layers of the network. In our approach, the attacker must assume, similar to SNIFF, that they need to target exactly one parameter. However, the assumption is more relaxed, as the attacker is not restricted to flipping only the sign bit of the parameter. Indeed, the attacker is allowed to flip any bit(s) of the parameter. Additionally, our method requires the attacker to perform two or more fault injection attacks on the same parameter to facilitate recovery. We acknowledge that performing multiple fault injections on the same parameter requires high precision in practical environments, which makes it challenging to implement. In this work, such repeated injections are assumed in order to verify the core principle of the proposed algorithm in a proof-of-concept setting. This assumption is not unique to our study. It has been adopted in prior multi-fault injection research, where the same target was subjected to repeated fault injections as part of the experimental methodology (Roscian and Sarafianos, Citation2013; Selmke and Brummer, Citation2016). Therefore, the repeated-injection assumption used in this study should not be regarded as an unrealistic special case, but rather as a standard approach that has been widely employed in the FIA research domain. If the attacker successfully recovers the parameters of the last layer of a deep neural network, irrespective of the specific bits being flipped, they can replicate the structure of the target network. This replication could enable unauthorised duplication of the unique technologies underlying systems utilising the DNN, using only the information about the network’s layer structure. 5. Methodology Building on the attacker model and assumptions detailed in the previous section, this section outlines the specific methods and techniques for recovering the DNN output layer parameters under fault injection attacks. We begin by presenting the mathematical foundations of parameter recovery, followed by a detailed description of the algorithm used to carry out the attack. 5.1. Parameter recovery formula The student neural network consists of nodes in the last hidden layer, and nodes in the output layer, with a bias vector and a weight matrix . As shown in Equation (Equation4), the output from the last hidden layer is used to compute the input to the output layer.(4) (4) The parameters to be recovered, and of the output layer, are initially unknown to the attacker. In this study, the Softmax function is used as the activation function for the output layer to facilitate the verification of parameter recovery. Consequently, the neural network’s output, can be derived by inputting into Equation (Equation2). In the proposed parameter recovery algorithm described in this paper, the most critical role is played by which represents the ratio between the preattack parameter value and the postattack parameter value . To analyse the changes caused by the fault injection, the algorithm computes λ using Equation (Equation5).(5) (5) As an example, consider a 64-bit floating point number represented as shown in Equation (Equation3). If a fault injection causes the least significant bit (LSB) of the exponent to flip from 1 to 0 the resulting becomes 2. This occurs because the 11-bit exponent, initially represented as changes to . Consequently, the number becomes , where Using the computed value of , we propose a theorem that facilitates the recovery of parameters even when one or more bits of the -th bias have been flipped due to a fault injection attack. Similar to the SNIFF approach, by leveraging the correct and incorrect outputs of a DNN, the target bias can be recovered, as outlined in Theorem 1 and Theorem 1-1. This process ultimately enables the recovery of the bias vector Additionally, by following a similar derivation, the weight matrix can be recovered using Theorem 2 and Theorem 2-1. Theorem 1. The target bias , can be recovered using the correct output and the incorrect output of a DNN for a given image input , along with the ratio . Specifically, it can be recovered as follows: Proof. The -th input and the fault-injected input of the output layer in the neural network can be defined as follows: Let’s represent and as follows. The output of the -th neuron in the neural network’s output layer is denoted as while the fault-injected output is represented as . By applying the inverse functions of these two outputs, the bias can be recovered as follows. Theorem 1-1. The error rate of the recovered parameters may increase due to the inverse operation of the DNN output and the natural logarithmic function operation. However, the target bias can be effectively recovered by leveraging the property that the order of the Softmax function’s output corresponds to the order of its input. Proof. The output of the neural network corresponds directly to the input of the Softmax function. Also,and by the above, Theorem 2. The target weight can be recovered for ∊ ∊ using the correct output and the incorrect output of a DNN for a given image input , along with the ratio . In this process, the non-zero output from the last hidden layer of the neural network is required. Specifically, the target weight can be recovered as follows: Proof. The -th input and the fault-injected input of the output layer in the neural network can be expressed as follows: Let’s represent and as follows: The output of the -th neuron in the neural network’s output layer is denoted as , while the fault-injected output is denoted as : By applying the inverse functions to these two outputs, the weight can be recovered as follows: Theorem 2-1. Under the same assumption as Theorem 1-1, the target weight can be effectively recovered as follows. In this process, the non-zero output from the last hidden layer of the neural network is required. Since SNIFF relies only on bit flips, providing a recovery formula was sufficient for output layer parameter recovery. However, an attacker who lacks knowledge of which bits were flipped and their specific indexes in the parameter after a fault injection attack cannot calculate the value of . Consequently, the theorems proposed in this paper cannot be directly applied, and the output layer parameters cannot be restored with a single fault injection. Nevertheless, because the parameters values after fault injection remain invariant regardless of the specific indexes of the flipped bits, this limitation can be addressed through multiple fault injection attacks. By performing the Theorem on two or more different faulted neural network outputs, and if the same parameter value is consistently obtained across all iterations, it can be concluded that the parameter has been correctly recovered. 5.2. Parameter recovery algorithm In this section, we propose a method for restoring restoration even when arbitrary bits are flipped, leveraging the Theorems presented in Section 5.1. Step 1. Precompute all possible . Before applying the algorithm, all possible parameter ratios are precalculated and stored. These precalculated ratios represent the possible relationships between an arbitrary parameter value and its corresponding faulted value that can be effectively calculated. Step 2. Fault Injections The target parameter to be restored is selected, which can either be a weight or a bias of the output layer. First, the student neural network is executed to obtain the correct neural output . Then, multiple independent fault injection attacks are performed on the target parameter, producing two or more distinct faulted neural network outputs . Step 3. Reduce the candidate parameters to the correct one. First, attempt to recover the parameter using the correct neural network output and two distinct faulted outputs, For all possible ratios , if the target parameter is a bias, apply Theorem 1 or Theorem 1-1 for each faulted output. If the target parameter is a weight, apply Theorem 2 or Theorem 2-1 for each faulted output. If identical results are obtained from the output pairs and those values are considered candidates for the parameter to be restored. If multiple candidates remain, use another faulted output to perform the Theorem calculation again for all . Compare the new results with the previously identified candidates to narrow down the list of possible correct parameters. This process is repeated until only one candidate remains, which is deemed the correctly recovered parameter. When recovering weights, if the output of the last hidden layer is 0, which prevents the recovery algorithm from proceeding, this issue can be resolved by using another input image. 6. Experimental results With the methodology established, we proceed to the practical application of our approach through a series of experiments aimed at validating the effectiveness of the proposed parameter recovery algorithm under fault injection attacks. This section details the experimental setup and provides an in-depth analysis of the results obtained. 6.1. Experimental setup To identify the types of bit flips that occur when a fault is injected externally into a memory chip, an electromagnetic fault injection (EM-FI) attack was conducted. The experimental setup, shown in , was designed to inject faults into an ATmega128P chip on an Arduino UNO board where floating-point numbers are stored. Ten different floating-point values were saved on the chip, and faults were injected using Riscure’s hardware (EM-FI Transient Probe, Spider) and Inspector software. Electromagnetic waves were emitted through a 1.5 mm polarised probe tip. Assuming each node of the neural network holds one value, 2,800 independent fault injections were performed for each floating-point value. The resulting bit flips were analysed by comparing the pre injection and postinjection floating point values. To validate the proposed algorithm, we implemented a neural network in Python using Keras (ver 2.2.4). All parameters of the implemented neural network, which operates on 64-bit floating point precision, are managed using the numpy library and set to the float64 data type. The toy neural network, designed for MNIST image classification, consists of: An input layer with 724 nodes; two hidden layers with 16 nodes each, using the ReLU activation function, and an output layer with 10 nodes utilising the Softmax activation function. In accordance with the assumptions described in the threat model, the experiments further assume that multiple independent fault injections can be applied to the same parameter to obtain distinct faulty outputs for recovery. While achieving such repeated injections with high precision can be challenging in practical environments, this assumption has also been adopted in prior multi-fault or double-fault injection research as part of the experimental methodology (Roscian and Sarafianos, Citation2013; Selmke and Brummer, Citation2016). This work therefore uses the repeated-injection assumption as part of a proof-of-concept design to verify the feasibility of the proposed approach. To align with the attacker assumptions in this paper, we treat the pretrained toy neural network as a student network derived through transfer learning from a teacher network. Parameter recovery verification focuses on the 170 output layer parameters, which are assumed to be unknown to the attacker, as all other parameters in the neural network are publicly available. In this paper, for experimental convenience, it is assumed that bit flips in parameters caused by fault injection occur only in the sign bit () and in the exponent bits up to the 8th bit from the LSB () of the floating point number. This selection of target bits is based on prior studies indicating that flips in the sign bit and exponent bits cause significantly larger numerical changes and error propagation compared to flips in the mantissa (Arora et al., Citation2024; Langroudi et al., Citation2023; Nair et al., Citation2021). Although bit flips can also occur in the mantissa, including this region would exponentially increase the number of possible values to be precomputed, making exhaustive evaluation infeasible within the scope of this paper. Additionally, to facilitate parameter restoration, the values of the modified output layer parameters were arbitrarily adjusted through simulation. However, these values can be sufficiently obtained in actual attack scenarios. This is achievable because the memory storing the parameters can be directly targeted, or faults can be injected at precise moments by monitoring the timing of output layer operations through EM measurements, as demonstrated in (Breier et al., Citation2021). 6.2. Results of the Bits-flip When 28,000 EM-FI tests were conducted, the stored data was found to change in 3,127 cases. Among these, illustrates the cumulative number of bit flips categorised by the number of flipped bits. For example, there were 362 cases where 7 bits were flipped, resulting in a change in the stored value. The results show that while single-bit flips were the most frequent, multi-bit flips (two or more bits) also occurred with notable frequency. As shown in , single bit flips were observed at various positions, not limited to the sign () but also occurring within the exponents (). Therefore, when injecting faults from external sources, multiple-bit flips can occur at various positions within the 64-bit double-precision floating-point representation, including bits beyond the sign bit, as observed in our experiment. While the exact pattern of bit flips may vary depending on the type of chip, the results indicate that it cannot be guaranteed that only the sign bit will flip, as assumed by SNIFF. Consequently, it is important to consider environments where bits other than the sign bit may also be subject to flipping. 6.3. Evaluation in toy DNN To validate the parameter recovery algorithm for the output layer of a toy MLP, all possible values of were precalculated as described in Step 1 of Section 5.2. As outlined in Section 6.1, since bit flips occur only in the sign bit () and exponent bits (), a total of 1,021 values were precalculated and stored. Subsequently, the fault injection process described in Step 2 of Section 5.2 was simulated, and more than two faulted neural network outputs were obtained by flipping random bits from the correct neural network output. Finally, parameter recovery for the target parameters was performed using Step 3. The results are summarised in . The top part of shows the recovery error rates for each of the 10 biases in the output layer, while the bottom part illustrates the recovery error rates for each of the 160 weights. The horizontal axis represents the -th target parameter, and the vertical axis denotes the error rate . The black dotted line in represents the average recovery error rate, indicating that both unknown biases and weights were successfully recovered with an average error rate of compared to the original correct output values. 7. Conclusion This paper introduces a novel algorithm for recovering neural network parameters compromised by fault injection attacks, with a particular focus on transfer-learned models. Our approach effectively recovers both bias and weight parameters, even in scenarios involving multiple bit flips, indicating a high level of robustness across diverse network structures and activation functions. While the experiments in this study were conducted on a simple MLP architecture, the proposed method targets the last hidden layer and operates independently of the complexity or structure of preceding layers. This characteristic makes the approach theoretically applicable to a wide range of architectures, including more complex convolutional neural networks such as ResNet and GoogleNet. In principle, the same recovery process can be applied as long as the attacker can access the outputs of the last hidden layer and induce bit flips—such as those in the sign and exponent bits—during the output layer computations. Although this work has not empirically evaluated such extensions, the underlying mechanism does not rely on architecture-specific features, suggesting strong potential for broader applicability. Consequently, the results presented here reveal an important security consideration for deep neural networks, highlighting the need for protective measures against parameter-recovery fault injection attacks in both simple and complex models. This work underscores the potential risks associated with parameter leakage, highlighting the critical need to secure neural networks against such vulnerabilities. While this study primarily focused on output layer recovery, future research aims to extend these techniques to include hidden layers, with the goal of preventing complete network replication. Moreover, the practical implications of this research are significant, particularly in real-world applications such as smart grids that utilise DNNs. In such contexts, the robustness of the proposed method could play a pivotal role in safeguarding against fault injection attacks, thereby enhancing the reliability and security of these critical infrastructures. This research emphasises the importance of ongoing efforts to enhance the security and integrity of deep learning models in increasingly adversarial environments. It also contributes to the broader fields of AI and cybersecurity by offering a practical method to mitigate risks in critical applications such as smart grids. Our findings expand the current understanding of fault injection attacks, demonstrating their potential impact on advanced neural network structures and underscoring the need to develop more resilient AI systems. As smart grids and similar infrastructures increasingly depend on deep learning models, this work provides vital insights into protecting such systems against emerging threats. Acknowledgement This research was supported by Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Education (No. RS-2025-25411243). Disclosure statement The authors declare that they have no conflicts of interest to report regarding the present study. Data availability statement The data that support the findings of this study are openly available at. The data that support the findings of this study are openly available in Figshare at https://doi.org/10.6084/m9.figshare.29987275.v1. Additional information Funding References - Alzubi, O. A., Qiqieh, I., & Alzubi, J. A. (2023). Fusion of deep learning based cyberattack detection and classification model for intelligent systems. Cluster Computing, 26(2), 1363–1374. https://doi.org/10.1007/s10586-022-03686-0 - Alzubi, O. A., Alzubi, J. A., Al-Zoubi, A. M., Hassonah, M. A., & Kose, U. (2022a). An efficient malware detection approach with feature weighting based on harris hawks optimization. Cluster Computing, 25(4), 2369–2387. - Alzubi, O. A., Alzubi, J. A., Alazab, M., Alrabea, A., Awajan, A., & Qiqieh, I. (2022b). Optimized machine learning-based intrusion detection system for fog and edge computing environment. Electronics, 11(19), 3007. https://doi.org/10.3390/electronics11193007 - Arora, R., Sinha, R., & Devitt, S. J. (2024). Investigating impact of bit-flip errors in control electronics on quantum computation. arXiv preprint arXiv:2405.05511 - Batina, L., Bhasin, S., Jap, D., & Picek, S. (2019). {CSI}{NN}: Reverse engineering of neural network architectures through electromagnetic side channel. In 28th USENIX Security Symposium (USENIX Security 19). - Boneh, D., DeMillo, R. A., & Lipton, R. J. (1997). On the importance of checking cryptographic protocols for faults. In International Conference on the Theory and Applications of Cryptographic Techniques. - Breier, J., Jap, D., Hou, X., Bhasin, S., & Liu, Y. (2021). SNIFF: Reverse engineering of neural networks with fault attacks. IEEE Transactions on Reliability, 71(4), 1527–1539. https://doi.org/10.1109/TR.2021.3105697 - Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J. D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., & Askell, A. (2020). Language models are few-shot learners. Advances in Neural Information Processing Systems, 33, 1877-1901. - Chen, T., Du, Z., Sun, N., Wang, J., Wu, C., Chen, Y., & Temam, O. (2014). Diannao: A small-footprint high-throughput accelerator for ubiquitous machine-learning. ACM SIGARCH Computer Architecture News, 42(1), 269–284. https://doi.org/10.1145/2654822.2541967 - Dahl, G. E., Stokes, J. W., Deng, L., & Yu, D. (2013). Large-scale malware classification using random projections and neural networks. In 2013 IEEE International Conference on Acoustics, Speech and Signal Processing. - Endo, S., Sugawara, T., Homma, N., Aoki, T., & Satoh, A. (2011). An on-chip glitchy-clock generator for testing fault injection attacks. Journal of Cryptographic Engineering, 1, 265–270. https://doi.org/10.1007/s13389-011-0022-y - Gheisari, M., Najafabadi, H. E., Alzubi, J. A., Gao, J., Wang, G., Abbasi, A. A., & Castiglione, A. (2021). OBPP: An ontology-based framework for privacy-preserving in IoT-based smart city. Future Generation Computer Systems, 123, 1–13. https://doi.org/10.1016/j.future.2021.01.028 - Goodfellow, I. J., Shlens, J., & Szegedy, C. (2014). Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6572 - Grigorescu, S., Trasnea, B., Cocias, T., & Macesanu, G. (2020). A survey of deep learning techniques for autonomous driving. Journal of Field Robotics, 37(3), 362–386. https://doi.org/10.1002/rob.21918 - He, K., Zhang, X., Ren, S., & Sun, J. (2015). Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. In Proceedings of the IEEE International Conference on Computer Vision. - Hou, X., Breier, J., Jap, D., Ma, L., Bhasin, S., & Liu, Y. (2021). Physical security of deep learning on edge devices: comprehensive evaluation of fault injection attack vectors. Microelectronics Reliability, 120, 114116. https://doi.org/10.1016/j.microrel.2021.114116 - Jap, D., Won, Y. -S., & Bhasin, S. (2021). Fault injection attacks on SoftMax function in deep neural networks, Proceedings of the 18th ACM International Conference on Computing Frontiers. - Jouppi, N. P., Young, C., Patil, N., Patterson, D., Agrawal, G., Bajwa, R., Bates, S., Bhatia, S., Boden, N., & Borchers, A. (2017). In-datacenter performance analysis of a tensor processing unit. Proceedings of the 44th Annual International Symposium on Computer Architecture, 2017, 1–12. https://doi.org/10.1145/3079856.3080246 - Kaissis, G. A., Makowski, M. R., Rückert, D., & Braren, R. F. (2020). Secure, privacy-preserving and federated machine learning in medical imaging. Nature Machine Intelligence, 2(6), 305–311. https://doi.org/10.1038/s42256-020-0186-1 - Kim, Y., Daly, R., Kim, J., Fallin, C., Lee, J. H., Lee, D., Wilkerson, C., Lai, K., & Mutlu, O. (2014). Flipping bits in memory without accessing them: an experimental study of DRAM disturbance errors. ACM SIGARCH Computer Architecture News, 42(3), 361–372. https://doi.org/10.1145/2678373.2665726 - Langroudi, H. F., Johnson, G., Lauter, K., & O’Neill, M. (2023). Evaluating the resiliency of posits for scientific computing. ACM Transactions on Architecture and Code Optimization (TACO), 20(4), Article 59. - Liu, Y., Wei, L., Luo, B., & Xu, Q. (2017). Fault injection attack on deep neural network. In 2017 IEEE/ACM International Conference on Computer-Aided Design (ICCAD). - Maji, S., Banerjee, U., & Chandrakasan, A. P. (2021). Leaky nets: Recovering embedded neural network models and inputs through simple power and timing side-channels—Attacks and defenses. IEEE Internet of Things Journal, 8(15), 12079–12092. https://doi.org/10.1109/JIOT.2021.3061314 - Movassagh, A. A., Alzubi, J. A., Gheisari, M., Rahimi, M., Mohan, S., Abbasi, A. A., & Nabipour, N. (2023). Artificial neural networks training algorithm integrating invasive weed optimization with differential evolutionary model. Journal of Ambient Intelligence and Humanized Computing, 14(5), 6017–6025. - Nair, A., Li, Q., Li, H., Zhang, M., & Chen, Z. (2021). The impact of faults on DNNs: A case study, 2021 IEEE 37th International Conference on Computer Design (ICCD) (pp. 49–56). IEEE. - Rakin, A. S., He, Z., & Fan, D. (2019). Bit-flip attack: Crushing neural network with progressive bit search. In Proceedings of the IEEE/CVF International Conference on Computer Vision. - Rakin, A. S., He, Z., & Fan, D. (2020). Tbt: targeted neural network attack with bit trojan. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition - Rakin, A. S., He, Z., Li, J., Yao, F., Chakrabarti, C., & Fan, D. (2021). T-bfa: Targeted bit-flip adversarial weight attack. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(11), 7928–7939. - Razavi, K., Gras, B., Bosman, E., Preneel, B., Giuffrida, C., & Bos, H. (2016). Flip feng shui: Hammering a needle in the software stack. In 25th USENIX Security Symposium (USENIX Security 16). - Ren, H., Huang, T., & Yan, H. (2021). Adversarial examples: Attacks and defenses in the physical world. International Journal of Machine Learning and Cybernetics, 12(11), 3325–3336. - Roscian, C., Sarafianos, A., Dutertre, J. -M., & Tria, A. (2013). Fault model analysis of laser-induced faults in sram memory cells. Workshop on Fault Diagnosis and Tolerance in Cryptography, 2013, 89–98. - Ruan, J., Liang, G., Zhao, J., Zhao, H., Qiu, J., Wen, F., & Dong, Z. Y. (2023). Deep learning for cybersecurity in smart grids: Review and perspectives. Energy Conversion and Economics, 4(4), 233–251. https://doi.org/10.1049/enc2.12091 - Ruan, J., Fan, G., Zhu, Y., Liang, G., Zhao, J., Wen, F., & Dong, Z. Y. (2023). Super-resolution perception assisted spatiotemporal graph deep learning against false data injection attacks in smart grid. IEEE Transactions on Smart Grid, 14(5), 4035–4046. https://doi.org/10.1109/TSG.2023.3241268 - Ruan, J., Yang, C., Wang, Q., Wang, S., Liang, G., Zhao, J., & Qiu, J. (2023). Assessment of spatiotemporally coordinated cyberattacks on renewable energy forecasting in smart energy system. Applied Energy, 347, 121470. https://doi.org/10.1016/j.apenergy.2023.121470 - Schmidt, J. -M., & Hutter, M. (2007). Optical and em fault-attackson crt-based rsa: Concrete results. na. - Selmke, B., Brummer, S., Heyszl, J., & Sigl, G. (2016). Precise laser fault injections into 90 nm and 45 nm sram-cells. In Smart Card Research and Advanced Applications: 14th International Conference, CARDIS 2015 (pp. 4–6). Bochum, Germany November 4−6, 2015. Revised Selected Papers 14. - Tajik, S., & Ganji, F. (2022). Artificial neural networks and fault injection attacks, Security and Artificial Intelligence: A Crossdisciplinary Approach (Vol. 13049, pp. 72–84). Springer. https://doi.org/10.1007/978-3-030-98795-4_4 - Takatoi, G., Sugawara, T., Sakiyama, K., & Li, Y. (2020). Simple electromagnetic analysis against activation functions of deep neural networks, Applied Cryptography and Network Security Workshops: ACNS 2020 Satellite Workshops, AIBlock, AIHWS, AIoTS, Cloud S&P, SCI (p. 18). SecMT, and SiMLA, Rome, Italy October 19–22, 2020, Proceedings 18. - Vasselle, A., Thiebeauld, H., Maouhoub, Q., Morisset, A., & Ermeneux, S. (2018). Laser-induced fault injection on smartphone bypassing the secure boot-extended version. IEEE Transactions on Computers, 69(10), 1449–1459. https://doi.org/10.1109/TC.2018.2860010 - Zhou, S., Wu, Y., Ni, Z., Zhou, X., Wen, H., & Zou, Y. (2016). Dorefa-net: Training low bitwidth convolutional neural networks with low bitwidth gradients. arXiv preprint arXiv:1606.06160. - Zuras, D., Cowlishaw, M., Aiken, A., Applegate, M., Bailey, D., Bass, S., Bhandarkar, D., Bhat, M., Bindel, D., & Boldo, S. (2008). IEEE standard for floating-point arithmetic. IEEE Standards Association, 754(2008), 1–70.