Enhancing generalization of cross-domain intrusion detection: a heterogeneous deep stacked ensemble approach

Abstract Intrusion Detection Systems (IDS) are critical for safeguarding network infrastructures, yet the challenges of heterogeneous environments and diverse data distributions often hinder their effectiveness. This study introduces the Heterogeneous Deep Stacked Ensemble for IDS (HDSE-IDS), a novel framework designed to improve cross-domain generalization by integrating Gated Recurrent Units (GRU), Long Short-Term Memory (LSTM), Deep Neural Networks (DNN), and Multi-Layer Perceptron (MLP) models in a stacked ensemble architecture. Each base model is trained on a distinct dataset and frozen to preserve domain-specific decision boundaries, while a meta-learner aggregates their predictions to form the final decision layer. The study conducted comprehensive cross-domain evaluations using four benchmark NetFlow-based datasets: NFv2-UNSW-NB15, NFv2-BoT-IoT, NFv2-ToN-IoT, and NFv2-CIC-2018. The proposed HDSE-IDS framework achieved an average F1-score of 88.77%, representing up to a 30% improvement over recent state-of-the-art methods. These findings demonstrate the robustness and effectiveness of HDSE-IDS in diverse and dynamic network scenarios, contributing to the advancement of scalable, generalizable IDS solutions for modern cybersecurity challenges. 1 Introduction The rapid growth of networked systems, encompassing enterprise networks, critical infrastructures, and cloud environments, has brought unparalleled advancements in connectivity and data exchange. However, this expansion has also led to a larger attack surface, exposing networks to sophisticated and evolving cyber threats (Frank et al., Citation2020). IDS have emerged as a critical defence mechanism to identify and mitigate such malicious activities as a first line of defence, ensuring the security and stability of network infrastructures (Aljabri et al., Citation2021). While the use of ML and DL has significantly enhanced the accuracy of intrusion detection (Yang et al., Citation2022), these models are often developed and evaluated on single benchmark datasets, leading to a critical gap in their ability to generalise across heterogeneous environments (Layeghy & Portmann, Citation2023). Specifically, IDS models that perform well on one dataset frequently experience substantial performance degradation when deployed on different network environments or datasets with varying traffic patterns and attack distributions. This phenomenon, known as the cross-domain generalisation problem, poses a serious challenge for real-world IDS deployment (Apruzzese et al., Citation2022). The root of this challenge lies in the distribution shift between training and deployment environments. Network traffic characteristics vary widely across organisations, devices, protocols, and periods. As a result, models trained on one dataset tend to overfit its specific characteristics and fail to recognise novel patterns in other domains. Prior studies such as DI-NIDS (Layeghy et al., Citation2023) and MLIDS22 (Abdulmajeed & Husien, Citation2022) have attempted to address this through domain adaptation techniques or dataset mixing strategies. However, these solutions require complex adversarial training or lack architectural mechanisms to retain the diversity of learned decision boundaries. Ensemble learning methods, particularly stacking-based approaches, have further enhanced detection accuracy by combining the strengths of multiple base models. Most existing models (Ahmad et al., Citation2022; Alotaibi & Alotaibi, Citation2020; Cao et al., Citation2023; Elijah et al., Citation2019; Lazzarini et al., Citation2023; Rajadurai & Gandhi, Citation2022; Shtayat et al., Citation2023; Wardana et al., Citation2024) primarily focus on in-domain evaluations, where models are trained and tested on the same dataset, often achieving high accuracy due to consistent feature distributions. However, their cross-domain generalisation abilities remain largely unexplored, as models trained on one dataset often experience significant performance degradation when evaluated on a different dataset with varying traffic patterns and attack distributions. Many studies apply ML techniques without fully considering the impact of cross-domain performance decay, leading to unrealistic expectations and potential failure when deployed in production environments (Ahmad et al., Citation2022; Elijah et al., Citation2019). Moreover, these ensembles often utilise homogeneous architectures or operate within a single domain, limiting their capacity to model diverse attack behaviours that emerge across different environments. To address these limitations, we propose HDSE-IDS, designed explicitly for cross-domain generalisation. HDSE-IDS integrates multiple deep learning models, each trained independently on distinct source domains. These base models are frozen and combined in a hierarchical stacking architecture through a meta-learner. This technique has demonstrated success in various domains, including face recognition (Muhammad et al., Citation2024), structural integrity and failure (Pernía-Espinoza et al., Citation2018), and medical diagnosis (Nguyen et al., Citation2021), where ensemble learning enhances generalisation and classification accuracy. This design enables the ensemble to retain diverse decision boundaries that capture unique patterns from each source domain while avoiding catastrophic forgetting. Unlike prior works that adapt to domain shifts through adversarial optimisation or data transformation, HDSE-IDS leverages architectural diversity and domain-specific training to promote natural generalisation to unseen domains. The key contributions of this paper are as follows: We propose HDSE-IDS, a deep stacked ensemble framework that integrates heterogeneous base learners, each trained on a distinct domain and frozen to retain domain-specific knowledge, enabling improved generalisation to unseen domains. Unlike prior ensemble methods, our framework is explicitly designed to address cross-domain generalisation without relying on domain adaptation or adversarial training, making it more scalable and architecture-agnostic. We conduct rigorous cross-domain evaluations across four NetFlow-based benchmark datasets, revealing that HDSE-IDS consistently outperforms individual models and previous ensemble approaches in detecting attacks under domain shift conditions. We present a detailed analysis comparing HDSE-IDS with state-of-the-art systems, highlighting its advantage in handling model heterogeneity and architectural diversity. The remainder of this paper is structured as follows. Section 2 reviews related work on ensemble learning techniques for IDS, highlighting the advancements and limitations of existing models. Section 3 details the methodology, describing the architecture of HDSE-IDS, the experimental design, and preprocessing steps. Section 4 presents the experimental results. Section 5 discusses the implications of the findings, focusing on cross-domain generalisation, the benefits of ensemble learning in IDS, and a comparative analysis of HDSE-IDS with related work. Finally, Section 6 concludes the study by summarising key contributions and outlining potential directions for future work. 2 Related work Intrusion detection has been an active research area, with various approaches leveraging ML and DL techniques to enhance detection accuracy. However, ensuring these models generalise well across diverse network environments remains a significant challenge. Traditional IDS solutions often exhibit performance degradation when evaluated on datasets different from their training data, highlighting the importance of cross-domain evaluation. To address these challenges, stacked ensemble learning has emerged as a promising strategy, combining multiple base models to improve robustness and generalisation. This section reviews prior work on cross-domain IDS evaluation, explores existing stacked ensemble techniques for intrusion detection, and identifies key research gaps that motivate the development of the proposed HDSE-IDS framework. The summary of related work is depicted in . 2.1 Cross-domain evaluation Cross-domain evaluation highlights the challenge of generalising machine learning-based IDS when tested across varying network environments. Most traditional IDS models show high accuracy on their training dataset but degrade significantly when evaluated on different datasets due to domain heterogeneity. Several studies have attempted to examine this gap. Sarhan et al. (Citation2022) proposed a standardised NetFlow-based feature set (43 features) to allow fair comparison across datasets like UNSW-NB15, BoT-IoT, ToN-IoT, and CSE-CIC-IDS2018. They found that inconsistent dataset features hinder practical cross-domain evaluation. While their work focused on feature set unification, they did not propose a model to solve the generalisation issue. Layeghy and Portmann (Citation2023) conducted a comparative study using four standardised NetFlow datasets and tested supervised and unsupervised models in cross-domain settings. None of the models achieved consistent performance across all datasets. The analysis revealed high asymmetry in performance when swapping the source and target datasets. Unsupervised models generally performed better, but the study did not propose a mitigation strategy. Al-Ambusaidi et al. (Citation2024) evaluated different ML classifiers on ToN-IoT and UNSW-NB15 but focused on improving intra-domain detection performance (up to 100% accuracy on ToN-IoT). Their work showed promising results but did not investigate cross-domain adaptability. In summary, existing cross-domain studies diagnose the problem or attempt partial solutions (e.g. dataset mixing), but most do not comprehensively propose dedicated models to address generalisation challenges. 2.2 Stacked ensemble methods Ensemble learning combines multiple base models to produce improved predictive performance compared to any single constituent model. Researchers widely recognise ensemble learning for its ability to reduce bias and variance, mitigate overfitting, and enhance the stability of decision-making processes (Polikar, Citation2006; Zhou, Citation2012). The primary categories of ensemble techniques include bagging, boosting, and stacking. Bagging (Bootstrap Aggregating), introduced by Breiman (Citation1996), reduces variance by training multiple homogeneous models on different bootstrap data samples and aggregating their outputs, typically through averaging or voting. Boosting focuses on sequentially training models, where each model attempts to correct the errors made by its predecessors, progressively improving overall accuracy. Techniques like AdaBoost and Gradient Boosting Machines exemplify this approach, mainly aiming to reduce bias by focusing on hard-to-predict samples. In contrast, stacking or stacked generalisation, first proposed by Wolpert (Citation1992), employs a meta-learner trained to combine the outputs of diverse base learners optimally. Unlike bagging and boosting, stacking offers a more effective approach for integrating heterogeneous models, as it enables the meta-learner to identify complex patterns in the base model outputs and assign dynamic importance to each model based on its strengths (Polikar, Citation2006; Zhou, Citation2012). Researchers find stacking especially advantageous in domain heterogeneity scenarios, where it effectively combines base models trained on distinct data distributions. Recent advancements have applied ensemble learning across several intrusion detection contexts. Stacking-based architectures have enhanced performance in vehicular networks and innovative infrastructure environments (Singh et al., Citation2024). In parallel, hybrid methods have explored the integration of decision trees with meta-heuristic algorithms, such as differential evolution and swarm intelligence, to optimise both feature selection and classifier performance (Bajpai et al., Citation2024). These approaches often combine base learners like XGBoost with advanced optimisation techniques to maximise detection accuracy under constrained conditions (Bajpai et al., Citation2024). Given the objectives of HDSE-IDS, stacking was selected as the ensemble strategy because it enables the integration of heterogeneous base models trained on different domains while learning optimal aggregation strategies through its meta-learner. This approach contrasts with bagging, which assumes homogeneous learners and focuses on variance reduction, and voting, which combines predictions without learning relationships among base model outputs. Stacking thus provides greater flexibility and potential for improved cross-domain generalisation, aligning with the design goals of this study. However, most existing studies focus solely on improving in-domain performance, with limited attention to cross-domain or multi-dataset adaptability. Lazzarini et al. (Citation2023) proposed DIS-IoT, a deep integrated stacking model combining MLP, DNN, CNN, and LSTM models into a standalone neural architecture. The authors evaluated the model on the ToN-IoT, CICIDS2017, and SWaT datasets, reporting excellent results in binary and multi-class classification tasks. DIS-IoT demonstrated high accuracy (up to 99.6%) and low false positive rates, outperforming several baseline deep learning models. However, all evaluations were conducted within each dataset, lacking cross-domain testing or explicit generalisation analysis. Rajadurai and Gandhi (Citation2022) introduced a stacked ensemble combining GBM and RF on the NSL-KDD dataset. The model achieved higher accuracy than individual classifiers and demonstrated robustness across various attack types. Nonetheless, it was limited to a single legacy dataset, and the ensemble did not include deep learning models. Amara et al. (Citation2025) developed a stacked deep ensemble incorporating CNN, TCN, and LSTM, using logistic regression as the meta-learner. The authors tested this model on the CICIDS2017 dataset using various data balancing techniques and achieved near-perfect performance, reporting up to 99.99% accuracy and a 100% F1-score with oversampling. Their work addressed class imbalance and model diversity but remained limited to in-domain evaluations. Elijah et al. (Citation2019) conducted an empirical study comparing ensemble (homogeneous and heterogeneous) and deep learning methods on the UNSW-NB15 dataset. Their homogeneous ensemble using Random Forest achieved a 97.9% detection rate, while the heterogeneous ensemble also showed competitive results. The authors included a separate LSTM model highlighting the performance gap between ensemble and single-model approaches. Again, evaluations were dataset-specific. Cao et al. (Citation2023) proposed a lightweight IDS for IoT using stacked ensemble learning with Naive Bayes and LightGBM as base learners and a TPE-optimised LightGBM as meta-learner. Evaluated on N-BaIoT and UNSW-NB15, the model showed high accuracy (up to 99.68%) with reduced computation cost due to feature selection. The focus was on efficiency and accuracy, without examining generalisation across datasets. Ahmad et al. (Citation2022) explored unknown attack detection using a deep learning ensemble comprising Autoencoder, CNN, and LSTM. They tested on four datasets (BoT-IoT, N-BaIoT, CICIDS2017, NSL-KDD), removing certain attack classes from training to simulate unknown threats. The ensemble performed well on some datasets, but suffered in detecting classes with distinct patterns (e.g. Reconnaissance, Heartbleed). While this study touches on generalisation and open-world challenges, it does not explicitly address cross-domain learning using stacked ensembles. These works validate the effectiveness of stacked ensembles in single-dataset IDS scenarios. However, they commonly lack cross-domain evaluation, multi-dataset training, or mechanisms to enhance generalisation. HDSE-IDS addresses this gap by designing its ensemble for heterogeneous sources and robustness across varying environments. 2.3 Generalisation The growing complexity and heterogeneity of network environments have underscored the importance of designing IDS models that generalise well across multiple domains. Recent works have addressed this generalisation gap by introducing novel training strategies, unsupervised techniques, and domain adaptation methods. Layeghy et al. (Citation2023) proposed DI-NIDS, a Domain-Invariant Network Intrusion Detection Systems that leverages DANN in combination with OSVM for anomaly detection, enhancing NIDS. The model projects traffic data into a domain-invariant space to mitigate the distributional shift between training and test datasets. The authors evaluated DI-NIDS using the NFv2-UNSW-NB15 and NFv2-CIC-IDS2018 datasets, where it demonstrated significantly less performance degradation than baselines such as LSTM, RF, and MLP in cross-domain settings. This method highlights the utility of adversarial feature alignment coupled with unsupervised detection in improving generalisation. Federated and privacy-aware ensembles have also gained traction, with implementations that adopt cross-silo federated learning structures to train intrusion detection systems collaboratively without sharing raw data (Bajpai et al., Citation2025). These solutions are particularly relevant in IoT environments where data locality and privacy are critical. de Carvalho Bertoli et al. (Citation2023) introduced a Stacked-Unsupervised Federated Learning approach to enhance cross-domain generalisation in heterogeneous network environments. Their architecture combines deep autoencoders and the EFC in a stacked ensemble and trains them under a federated setup across four datasets: TON-IoT, BoT-IoT, CIC-2018, and UNSW-NB15. Their results demonstrate that this hybrid setup outperforms conventional unsupervised baselines such as Isolation Forest and One-Class SVM in inter-dataset evaluations, achieving an average F1-score of 0.84 with benign and attack thresholds. These findings suggest that combining federated learning with effective stacking and unsupervised techniques can significantly enhance generalisation without compromising data privacy. Abdulmajeed and Husien (Citation2022) proposed MLIDS22, a hybrid CNN-LSTM architecture trained on a mixture of CIC-IDS2017 and CSE-CIC-IDS2018 datasets to overcome generalisation limitations observed in individual dataset training. Their inter-dataset evaluation reveals a sharp drop in performance when the model is trained on one dataset and tested on another, confirming the generalisation challenge. However, when trained on a combined dataset, the model demonstrated strong performance across all test domains, achieving AUC scores near 1.0. These results reinforce the importance of training on diverse datasets and provide empirical support for mixing datasets to develop robust IDS models. Oriola (Citation2020) proposed a stacked generalisation ensemble technique comprising two meta-learners based on Artificial Neural Networks ANNs. The first meta-learner was a classical stacked ensemble using predictions from four base models: SVM, Logistic Regression, Naïve Bayes, and MLP. The second tier applied a multi-feature-based stacked ensemble, optimised through grid search to select the best meta-feature combinations. This two-tiered stacking approach achieved 97% accuracy and 0.98 F1-score on the NSL-KDD dataset. Although the approach achieved impressive in-domain generalisation and outperformed previous ensemble methods, the authors tested it on only a single dataset and did not perform cross-domain validation. These studies indicate that models can achieve generalisation through various approaches, including domain-invariant feature learning, unsupervised stacking, federated architectures, and dataset aggregation. However, each has its trade-offs in terms of complexity, interpretability, and deployment requirements. These insights directly motivate the design of our HDSE-IDS framework, which aims to combine the strengths of ensemble learning with multi-dataset awareness to tackle the generalisation challenge in a scalable and model-agnostic manner. 2.4 Gap analysis The reviewed literature highlights significant advancements in developing IDS, particularly in improving detection accuracy through sophisticated deep learning architectures, ensemble methods, and recent efforts toward generalisation. However, several critical gaps hinder the development of IDS models that can generalise reliably across heterogeneous environments. Many studies on cross-domain evaluation acknowledge that model performance degrades substantially when tested on datasets that differ from their training data. While studies such as those by Sarhan et al. (Citation2022) and Layeghy and Portmann (Citation2023) have quantified this issue through extensive benchmarking, they stop short of proposing architectures that actively mitigate it. Others, like Abdulmajeed and Husien (Citation2022), attempt to address the issue using dataset mixing but do not explore model-level design strategies to enhance robustness across domains. Second, the work on stacked ensemble methods for IDS demonstrates that model fusion can significantly improve performance in in-domain settings. Studies such as DIS-IoT (Lazzarini et al., Citation2023) and Amara et al. (Citation2025) report high accuracy using stacked deep learning architectures. However, researchers often evaluate these methods strictly within the boundaries of their training datasets, leaving their ability to generalise to new domains untested. Furthermore, most ensembles lack architectural diversity, with base models often being variants of NN or traditional classifiers of the same family, limiting the diversity of learned decision boundaries. Third, while some recent efforts directly target generalisation in IDS, such as the works by de Carvalho Bertoli et al. (Citation2023) and Layeghy et al. (Citation2023), they often introduce architectural or operational complexity (e.g. adversarial training, federated threshold tuning) that limits their scalability. Others, like MLIDS22 (Abdulmajeed & Husien, Citation2022), rely solely on dataset-level diversity without exploiting the representational benefits of ensemble learning or heterogeneous client settings. Most generalisation-focused studies do not combine multiple model architectures from different data sources or systematically freeze or stack models to maximise decision diversity. In summary, existing research either identifies the generalisation problem without proposing robust solutions or improves local accuracy without validating performance across domains. None of the reviewed approaches systematically combine: Heterogeneous base models trained on distinct domains, Model stacking that preserves each domain's learned representation (via freezing), An ensemble-aware architecture explicitly designed to generalise across datasets. To address these gaps, we propose HDSE-IDS, a deep stacked ensemble architecture composed of heterogeneous base learners trained independently on different datasets. By freezing base models and integrating them using a meta-learner, HDSE-IDS captures diverse decision boundaries and enhances generalisation to unseen domains while remaining modular and scalable. This design directly builds on insights from all three areas-cross-domain evaluation, ensemble learning, and generalisation-focused design-while overcoming their limitations. 3 Methodology This study presents a structured methodology for developing and evaluating the HDSE-IDS. As illustrated in , the methodology is divided into three key stages: Data Operations, Model Performance Analysis, and Heterogeneous Deep Stacked Ensemble Mechanism. Stage One (Data Operations) begins by selecting and preparing data to ensure the inclusion of diverse and representative datasets for training and evaluation. This stage involves necessary preprocessing steps, such as handling imbalances and standardising features, to enhance model learning. In Stage Two (Model Performance Analysis), researchers select and train base models that will later be integrated into the ensemble framework. This phase also includes generalisation analysis, where the individual models are evaluated under in-domain and cross-domain settings to identify performance variations across datasets. Finally, Stage Three (Heterogeneous Deep Stacked Ensemble Mechanism) details the design and integration of the stacked ensemble architecture, which incorporates multiple deep learning models (Gated Recurrent Unit (GRU), LSTM, DNN, and MLP) to improve robustness and detection performance. The stacking mechanism concludes with a performance evaluation, where researchers assess the proposed approach using multiple benchmark datasets to validate its effectiveness. This methodology ensures a systematic approach to intrusion detection, leveraging diverse architectures in a stacked ensemble to enhance cross-domain adaptability and improve overall detection performance. 3.1 Data operations Data Operations is the first stage in the proposed methodology, ensuring that the datasets used for training and evaluation are preprocessed and prepared to enhance the performance of the HDSE-IDS. 3.1.1 Data selection A standardised feature set is critical to fairly comparing ML-NIDS across different datasets. This step is vital for cross-domain evaluation, where researchers train a model on one dataset and evaluate it on a different one. A consistent feature set across training and evaluation datasets is a prerequisite to eliminate biases introduced by differing dataset structures. However, earlier NIDS datasets often relied on different tools and methodologies, resulting in inconsistent feature sets that limit their applicability in cross-domain studies. To address this limitation, we employ four recently standardised NetFlow-based datasets: NFv2-UNSW-NB15, NFv2-CIC-2018, NFv2-ToN-IoT, and NFv2-BoT-IoT by Sarhan et al. (Citation2022). The authors derived these datasets from their original formats-UNSW-NB15 (Moustafa & Slay, Citation2015), CIC-2018 (Thakkar & Lohiya, Citation2020), ToN-IoT (Alsaedi et al., Citation2020), and BoT-IoT (Koroniotis et al., Citation2019)-and converted them into a unified format containing a standard set of 43 NetFlow features. This standardisation ensures that models can be consistently trained and evaluated across different datasets. The datasets are publicly accessible1. For a detailed discussion of the dataset standardisation process, including the extracted features and their relevance, readers may refer to (Sarhan et al., Citation2022). This standardised approach and dataset selection enable a robust evaluation of the domain-specific and cross-domain performance of ML-NIDS, laying a strong foundation for the experiments conducted in this study. 3.2 Data preprocessing Several preprocessing steps are applied to prepare the datasets for training and evaluation in this stage. The authors conduct feature selection to retain only the most relevant attributes, eliminating redundant or highly correlated features that could degrade model performance. Data normalisation is employed to standardise feature values, ensuring consistency across datasets and improving the learning stability of deep models. The dataset is split into training and testing subsets while maintaining a balanced distribution of normal and attack traffic to prevent bias during model learning. Synthetic data augmentation techniques, such as oversampling, are applied where necessary to address class imbalance. This step ensures that the models are trained on a more balanced representation of attack and benign traffic, improving their ability to generalise across different datasets. Furthermore, categorical attributes are encoded into numerical representations to ensure compatibility with deep learning models. These preprocessing steps collectively enhance the robustness of the model and contribute to the effective generalisation of the proposed approach. 3.3 Model performance analysis The Model Performance Analysis stage systematically evaluates individual deep learning models before their integration into the stacked ensemble. This phase consists of three key substages: Base Model Selection, Baseline Model Training, and Generalisation Analysis. Each step ensures that the most effective models are identified and assessed under in-domain and cross-domain scenarios before finalising the ensemble configuration. 3.3.1 Base model selection In IDS, selecting appropriate base models is critical for achieving robust detection performance and strong generalisation across heterogeneous network environments. This study focuses on deep learning architectures due to their proven ability to learn complex hierarchical representations and nonlinear relationships in high-dimensional network traffic data (Sowmya & Mary Anita, Citation2023). Deep architectures demonstrate significantly improved performance on structured security datasets, particularly in capturing intricate attack patterns and reducing false positives compared to shallow models such as decision trees or logistic regression. The choice of specific base models-GRU, LSTM, MLP, and DNN-was guided by both theoretical considerations and empirical evaluation. Recurrent models such as GRU and LSTM effectively model temporal dependencies and sequential patterns in NetFlow-based features, which are inherently ordered and time-sensitive (Subhan et al., Citation2022). On the other hand, feedforward models such as DNN and MLP are better suited for capturing nonlinear feature interactions in a non-sequential setting, making them complementary to their recurrent counterparts (Ashiku & Dagli, Citation2021). In the early experimental phase of this study, we evaluated a broader set of supervised deep learning models, including CNN-based architectures (Mohammadpour et al., Citation2022) and hybrid variants. However, based on comparative performance across multiple datasets (see Section 5.2), the four selected models consistently demonstrated superior trade-offs between accuracy, generalisation, and training stability. Due to space constraints, detailed results of these excluded models are not included in this paper, but can be made available upon request. Finally, the resulting ensemble ensures a diverse set of decision boundaries by selecting models from distinct architectural families and training them independently on separate domains. This diversity is a crucial enabler for the cross-domain generalisation ability of HDSE-IDS, as discussed in Section 3.4. 3.3.2 Baseline model training When the base models are selected, all of the base models are trained independently on each dataset to achieve a base performance metric. Following the training process, model architectures, learning rates, and other hyperparameters are optimised to identify configurations that yield better performance on the training set. Each model undergoes standard deep learning training procedures to ensure optimal convergence, including backpropagation and gradient descent optimisation. This substage aims to find each model's standalone performance before creating the stacked ensemble. 3.3.3 Generalisation analysis In the first part, we perform extensive generalisation analysis after baseline training, evaluating model performance in in- and cross-domain settings. In the in-domain, it means testing the model by training on the same dataset and testing it on the same dataset to see how effective it will be within a controlled environment. The cross-domain evaluation assesses the quality of a model trained on one dataset and how it performs in a different dataset, revealing the impact of data heterogeneity. Each model is evaluated for its performance in cross-domain scenarios, and the resulting performance degradation is analysed to assess generalisation capability. One of the insights that emerges from this analysis is how much the base models can cope with the distributional shifts, which is an essential factor for the robustness of the final stacked ensemble. 3.4 Heterogeneous deep stacked ensemble mechanism Finally, our methodology includes the Heterogeneous Deep Stacked Ensemble Mechanism, which aims to boost intrusion detection using various deep learning models in a stacked ensemble. The three phases of this phase are Stacking Architecture Design, Ensemble Integration, and Performance Evaluation. It allows different models to add complementary strengths to improve generalisation across various network conditions. 3.4.1 Problem formulation Let represent a collection of network intrusion detection datasets, where each dataset is drawn from a distinct domain and consists of samples , with denoting the input feature space and representing binary labels (benign or malicious). Due to variations in traffic patterns, attack types, and feature distributions, the marginal and conditional distributions differ across domains:(1) (1) Given this distribution shift, a classifier trained on source domain often performs poorly on a target domain due to limited generalisation. Thus, the objective is to learn a composite model that performs well on unseen target domains , despite not being trained directly on them. We define as a stacked ensemble:(2) (2) where: is a base learner trained solely on dataset , using a deep neural network architecture , where denotes a set of eligible architectures, is the meta-learner trained on the prediction outputs (logits or soft labels) of all base learners, All are frozen after individual training to retain domain-specific knowledge and prevent further updates during ensemble integration. The meta-learner is trained on a meta-dataset , where each is the concatenated prediction vector from the base models, and is the corresponding ground truth label. The optimised meta-learner is as follows:(3) (3) where is the binary cross-entropy loss. During inference, generalises to a new domain by leveraging the frozen knowledge of each , allowing the ensemble to respond to previously unseen traffic distributions. 3.4.2 Stacking architecture design This phase begins by defining the stacking ensemble architecture: the base models generate independent predictions, which serve as input features for the meta-learner. The meta-learner is a higher-level model that aggregates and refines these predictions, improving the overall classification performance. Insights from the previous evaluation phases guide the selection of base models and the design of the stacking strategy. 3.4.3 Ensemble integration The meta-learner processes the stacked feature vectors generated by the base model predictions to produce the final output. These predictions are collected from independently trained and frozen base models, preserving the knowledge each has learned from its respective training domain. Rather than retraining or modifying the base learners, the process combines their outputs to form a meta-representation that captures diverse decision boundaries across domains. The final decision-making process of the meta-learner arises from analysing this combined representation, enabling it to learn how to leverage each base model's strengths best. The training process applies the same data split used for the base models to train the meta-learner, ensuring consistency in evaluation. The training process applies data balancing techniques to improve the meta-learner's sensitivity to minority classes and mitigate the effects of class imbalance. By integrating the model predictions, the framework reduces individual model deficiencies while boosting decision-making effectiveness in in-domain and cross-domain testing scenarios. 3.5 Performance evaluation Performance evaluation of the ensemble model represents the last operation within this stage. The HDSE-IDS framework is tested with its trained configuration through the selected benchmark datasets to determine F1 score performance. The evaluation includes a comprehensive comparison between individual models and modern standards to assess performance improvement. The evaluation confirms that the ensemble efficiently improves cross-domain intrusion detection while preserving excellent generalisation potential. 3.5.1 Evaluation metrics To evaluate the performance of the proposed HDSE-IDS model, we employ four standard classification metrics: accuracy, precision, recall, and F1-score. F1-score is selected as the primary evaluation metric because it effectively handles imbalanced datasets-a common characteristic in intrusion detection scenarios (More et al., Citation2024). The F1-score, the harmonic mean of precision and recall, provides a balanced perspective on false positives and false negatives. This metric is significant for IDS, where missing an attack (false negative) or wrongly classifying benign traffic as malicious (false positive) can lead to significant security and operational issues.(4) (4) with: Precision = (True Positives/All predicted positives) Recall = (True Positives/All actual positives) While accuracy provides a general sense of correctness, precision and recall offer insights into specific aspects of the model. The F1-score is the most informative measure for evaluating IDS performance in imbalanced settings. 4 Implementation The implementation of the proposed methodology follows a structured process, as illustrated in . It begins with data preprocessing, then base model training and evaluation, and concludes with constructing the Heterogeneous Deep Stacked Ensemble (HDSE-IDS). Each phase incorporates key design choices, such as dataset splitting ratios, model selection, and evaluation strategies, ensuring robust and effective IDS performance. The following sections provide a detailed explanation of each stage. 4.1 Data preprocessing Data preprocessing ensures consistency and fairness when evaluating the proposed model across multiple datasets. This phase involves cleaning, normalising, encoding, and transforming the data to align with the study's research objectives while maintaining comparability across different datasets. The process begins with data cleaning, where missing values are handled using mean or median imputation, depending on the feature distribution. We use mean imputation for normally distributed features and prefer median imputation for skewed distributions. These techniques help maintain the statistical integrity of the data while minimising distortion of patterns relevant to intrusion behaviour. Next, categorical values (such as attack labels) are label-encoded to convert them into numeric form, which is essential for compatibility with deep learning architectures that require numerical inputs. For feature scaling, min-max normalisation is applied to all numerical features to rescale them into the range [0,1]. This technique was selected for its simplicity and effectiveness in accelerating model convergence during training, especially when used with gradient-based optimisers. The normalisation is computed using:(5) (5) where is the original value, and and are the minimum and maximum of that feature in the training set. The dataset-specific features, such as source and destination IP addresses, were excluded to ensure fairness in cross-domain evaluation, as these attributes introduce bias and hinder generalisation. All other original flow features were retained to preserve the intrinsic network behaviour and traffic characteristics necessary for intrusion detection. 4.2 Base model building The base model building phase is essential in constructing the HDSE-IDS framework, as it establishes the foundation for the heterogeneous deep stacked ensemble. This phase involves selecting, training, and evaluating multiple DL architectures to determine the best-performing models for each dataset. 4.2.1 Model design The base model building process starts with designing each model's architecture. The machine learning algorithms used in this study and their parameters are described in . The model design section outlines the architecture and configurations of the deep learning models used in this study: LSTM, GRU, MLP, CNN, and DNN, based on the selection criteria explained in the methodology section of this study. LSTM: The LSTM model captures temporal dependencies within network traffic data, making it well-suited for intrusion detection. A reshape layer follows the input layer to format the data into a 3D structure compatible with LSTM processing. The architecture consists of two LSTM layers with 24 and 16 neurons, respectively, utilising ReLU activation. A fully connected dense layer with eight neurons follows, along with a final sigmoid activation layer for binary classification. The model trains using the Adam optimiser and binary cross-entropy loss. MLP: The MLP model is a simple yet effective feed-forward neural network designed to capture feature interactions. The input layer is connected to two dense layers with 32 and 16 neurons, using ReLU activation to process extracted features. The final dense layer contains ten units with a sigmoid activation function for binary classification. The training process employs the Adam optimiser and binary cross-entropy as the loss function. DNN: The DNN model is a fully connected deep network designed to approximate complex feature dependencies. The architecture consists of an input layer followed by five dense layers with 32, 16, 8, 4, and 2 neurons, respectively, all using ReLU activation. The final output layer applies a sigmoid activation function for binary classification. This deep architecture enables the model to learn multi-level features, making it highly suitable for network intrusion detection. The training process utilises the Adam optimiser and binary cross-entropy loss. GRU: The GRU model, similar to LSTM, is designed for sequential data processing but with reduced computational complexity. A reshape layer formats the input for GRU processing, followed by two GRU layers with 24 and 16 units, both using ReLU activation. The architecture includes a fully connected dense layer with eight neurons and a final sigmoid output layer for binary classification. The training process optimises the model for network traffic classification using the Adam optimiser and binary cross-entropy loss. CNN: The CNN model leverages convolutional layers to detect spatial patterns in network traffic data. The architecture consists of four hidden layers, trained using the Adam optimiser with a learning rate of 0.001. The model processes input batches of 64 samples over 10 epochs, utilising the ReLU activation function in hidden layers and a sigmoid activation function in the output layer for binary classification. 4.2.2 Training and in-domain evaluation Each of the five deep learning models (LSTM, GRU, MLP, DNN, and CNN) was trained independently on each dataset using standard supervised learning. Training was conducted using a batch size of 64, over 10 epochs, with the Adam optimiser and binary cross-entropy loss. All five models followed a consistent architecture across datasets and were trained and validated using an 80:20 split to ensure comparability-a fixed random seed of 42 controlled data splitting, model initialisation, and training to ensure reproducibility. Training was conducted on a machine containing a single NVIDIA Tensor A10 GPU system, 8 CPU cores, and 16 GB of RAM. The in-domain evaluation results, presented in the results section, provided insights into model performance within their respective domains. The selection process identified the best-performing model for each dataset based on the F1-score: LSTM for NFv2-ToN-IoT, DNN for NFv2-UNSW-NB15, MLP for NFv2-BoT-IoT, and GRU for NFv2-CIC-2018. These four models served as candidates for cross-domain evaluation, allowing the assessment of their generalisation capabilities across different datasets. 4.2.3 Cross-domain evaluation The cross-domain evaluation stage examines the ability of the models to generalise across datasets. Using the pre-split data, models are trained on the training set of one dataset (e.g. NFv2-ToN-IoT) and tested on the test (evaluation) sets of other datasets (e.g. NFv2-CIC-2018, NFv2-BoT-IoT, and NFv2-UNSW-NB15). This stage highlights the challenges introduced by distribution shifts across domains, simulating real-world scenarios where models must operate in unseen environments. The evaluation quantifies performance degradation using accuracy, precision, recall, and F1-score metrics. The cross-domain evaluation process is depicted in , showcasing the flow from training on a single dataset to testing across multiple others. 4.3 Heterogeneous deep stacked ensemble integration The Heterogeneous Deep Stacked Ensemble Integration stage finalises the HDSE-IDS framework by combining the best-performing models from each dataset into a unified stacked ensemble. As illustrated in the workflow, D1-LSTM, D2-DNN, D3-MLP, and D4-GRU were selected based on their superior in-domain performance. These independently trained models are first frozen to preserve their domain-specific representations. The system concatenates their output logits (i.e. raw outputs before activation) to form a unified meta-feature vector that captures diverse predictive signals from each domain. The framework feeds this combined representation into a shallow meta-learner, which learns to produce the final prediction. The meta-learner is a simple feedforward neural network trained using the same dataset splits as the base models. To address potential class imbalance-particularly in datasets such as NFv2-UNSW-NB15-SMOTE oversampling is applied before training the meta-learner. Final evaluations are conducted on held-out test sets using standard performance metrics, including accuracy, precision, recall, and F1-score. This integration strategy allows the ensemble to leverage the complementary strengths of each base model while enhancing generalisation across heterogeneous domains. 4.3.1 Meta-learner for HDSE-IDS Unlike traditional approaches that use a separate model as a meta-learner, HDSE-IDS employs an integrated neural network layer to combine the outputs from multiple heterogeneous base learners within a unified architecture. This design supports end-to-end architectural integration by seamlessly connecting the meta-learner with the base models (LSTM, GRU, MLP, and DNN). It also facilitates potential future extensions, including joint optimisation, federated learning adaptations, and continual learning setups. The stacked ensemble model concatenates a single probability output from each base model to form the feature vector, which is processed through a 10-neuron hidden dense layer with ReLU activation, followed by a sigmoid-activated output layer for binary classification. This neural network meta-learner performs an additional training step to dynamically optimise the combination of base learner outputs dynamically, improving classification performance and generalisation across heterogeneous datasets. The training process uses binary cross-entropy loss and optimises the model with the Adam optimiser. HDSE-IDS will also compare this integrated NN meta-learner with traditional meta-learners such as RF and LR to ensure a rigorous and transparent evaluation. While not incorporated into the base learner architecture, these models provide valuable baselines because they are widely used, interpretable, and known for strong performance on tabular meta-feature data. The comparison aims to assess whether the architectural advantages of the neural network meta-learner translate into practical performance gains, particularly in challenging cross-domain scenarios. illustrates the overall process of the HDSE-IDS model, where the output layer provides a single output representing the binary classification task across all datasets. 4.3.2 The HDSE-IDS algorithm HDSE-IDS integrates the ensemble structure with the base learners trained independently on each dataset. During the training of the meta-learner, it freezes the weights and biases of its base learners so that they cannot be updated anymore. The system concatenates the outputs of the base learners from each input sample into meta-features, which then serve as input to the fully connected meta-learner. It then trains the meta-learner to make the final prediction by minimising the combination of tbase learners' outputs to maximise the classification's performance. This completes the process of HDSE-IDS summarised in Algorithm 1. By this approach, HDSE-IDS ensures that it captures and combines various learning inferences from various datasets so that the network intrusion detection has robust performance. The code repository is publicly accessible2. 4.3.3 Performance evaluation The meta-model is tested on all four test datasets in the Performance Evaluation stage to assess its effectiveness. The results are compared with the cross-domain evaluation of the base models to determine performance improvements. Additionally, the meta-model is evaluated against state-of-the-art related works to highlight its effectiveness in generalising across different domains. Table 5 Results This section presents the proposed HDSE-IDS framework's empirical evaluation across in-domain and cross-domain scenarios. The assessment uses four benchmark NetFlow-based datasets: NFv2-ToN-IoT, NFv2-BoT-IoT, NFv2-CIC-2018, and NFv2-UNSW-NB15. We assess the performance of the individual base models and the meta-learner and compare HDSE-IDS against related ensemble and non-ensemble baselines. The objective is to investigate the generalisation capabilities of the proposed model and validate its effectiveness in handling distributional variations across network domains. The evaluation process applies the F1-score to assess all experiments. 5.1 Dataset description For classification, all attack classes were consolidated into a single “Attack” label, resulting in a binary classification setup of “Benign” vs “Attack.” This simplification aligns with the study's objectives and ensures a more consistent evaluation across datasets. Among attack types, DoS attacks were selected as the primary focus since they are the only attack category consistently present in all datasets. This selection provides a uniform evaluation framework and enables a reliable comparison of model performance across domains. Furthermore, as DoS attacks remain one of the most prevalent and disruptive cybersecurity threats (Zlomislic et al., Citation2017), detecting them is critical for maintaining network availability. By isolating DoS attacks, this study mitigates unnecessary variability from heterogeneous attack types, ensuring a well-defined and interpretable problem space for cross-domain evaluation. shows the summary of the datasets along with their class distribution. overviews the NetFlow feature set utilised across the four studied datasets. The original NetFlow records contain 43 features describing flow characteristics, including source and destination addresses and ports, byte and packet counts, protocol information, duration metrics, and TCP/IP-specific flags. This study excluded features directly tied to IP addresses or similar identifiers (e.g. IPv4 source/destination addresses and source/destination port numbers) to ensure generalisability and privacy preservation. Instead, the selected feature subset focuses on attributes related to flow behaviour, traffic volume, timing, and packet properties, as these are more robust for cross-domain evaluation and intrusion detection. 5.2 In-domain evaluation When trained and tested on the same dataset, each selected deep learning model demonstrates excellent in-domain performance across all evaluation metrics, as shown in . The metrics considered include accuracy, precision, recall, and F1-score, offering a more comprehensive view of model behaviour in balanced and imbalanced scenarios. The results highlight that LSTM, GRU, and DNN consistently achieve high performance across all datasets, whereas CNN demonstrates the lowest scores overall. On the NFv2-UNSW-NB15 dataset, the DNN model achieved the highest overall performance, with an accuracy of 0.9983, a precision of 0.9964, a recall of 1.0000, and an F1-score of 0.9983. These results indicate that DNN was highly effective at detecting attacks without producing false negatives, while maintaining minimal false positives. While MLP and LSTM also performed competitively with F1-scores of 0.9957, DNN maintained a slight edge in precision and accuracy. illustrates this model's training and validation trends. On the NFv2-BoT-IoT dataset, the MLP model outperformed all others, achieving 0.9990 across accuracy, precision, and F1-score, as well as 0.9991 recall. This nearly perfect balance across all metrics suggests that MLP was able to capture the feature distribution of this dataset exceptionally well. DNN and GRU followed with F1-scores of 0.9800 and 0.9763, respectively. shows the training and validation curves for MLP on this dataset. For the NFv2-ToN-IoT dataset, LSTM emerged as the top performer with an accuracy of 0.9969, precision of 0.9959, recall of 0.9979, and F1-score of 0.9969. These scores reflect LSTM's ability to capture temporal dependencies in the dataset more effectively than other models. DNN and GRU achieved lower F1-scores of 0.9819 and 0.9800, respectively. depicts the learning performance of LSTM on NFv2-ToN-IoT. On the NFv2-CIC-2018 dataset, GRU demonstrated the strongest results across all metrics, achieving perfect accuracy, recall, and F1-score (1.0000), and a near-perfect precision of 0.9999. This performance confirms the GRU model's excellent generalisation capability within the domain. Other models, including LSTM and DNN, recorded F1-scores between 0.9733 and 0.9749. presents the corresponding training curves for GRU. These results indicate that LSTM, GRU, and DNN consistently exhibit high performance across datasets, highlighting their strong ability to model network intrusion patterns. MLP showed promising results in specific cases, such as NFv2-BoT-IoT, but performed less effectively in others. CNN, however, struggled to achieve competitive results, suggesting that its architecture is less suited for these intrusion detection tasks. These findings reinforce the advantages of recurrent-based and deep architectures in capturing complex attack behaviours within domain-specific environments. 5.3 Cross-domain evaluation presents the cross-domain evaluation results, where models trained on one dataset are tested on other datasets to assess their generalisation ability. The results demonstrate significant performance degradation across most cross-domain scenarios, especially in recall and F1-score, indicating the difficulty adapting to unseen distributions. When trained on NFv2-UNSW-NB15, the model achieved its best generalisation on NFv2-BoT-IoT, with an accuracy of 0.8435, precision of 0.8007, recall of 0.9147, and an F1-score of 0.8539. However, the performance dropped sharply on NFv2-ToN-IoT and NFv2-CIC-2018, where recall fell below 0.12 and F1-scores were only 0.1330 and 0.0714, respectively. This result highlights how distribution shifts can severely affect recall and detection performance, even using a high-quality source model. Similarly, the model trained on NFv2-BoT-IoT displayed poor generalisation. While accuracy values hovered around 0.50 for all targets, precision was highly inconsistent, and recall dropped to near-zero on all target datasets. For instance, recall was only 0.0086 on NFv2-UNSW-NB15 and 0.0052 on NFv2-CIC-2018, leading to very low F1-scores between 0.0103 and 0.0190. The model trained on NFv2-ToN-IoT showed the weakest overall generalisation, particularly failing to detect attack instances when evaluated on NFv2-UNSW-NB15 and NFv2-BoT-IoT, with all metrics reduced to 0.0000. The only reasonable performance appeared on NFv2-CIC-2018, with a recall of 0.1494 and an F1-score of 0.2298, though still relatively low. On the other hand, the model trained on NFv2-CIC-2018 showed moderate generalisation to NFv2-BoT-IoT, with an accuracy of 0.8169, precision of 0.9987, and recall of 0.6346, resulting in a comparatively strong F1-score of 0.7760. However, performance on the other two target datasets remained poor, particularly NFv2-ToN-IoT, where recall dropped to 0.0002 and the F1-score to just 0.0003. 5.4 Stacked ensemble evaluation This section comprehensively evaluates the proposed HDSE-IDS. We assess the effectiveness of HDSE-IDS in enhancing generalisation across diverse network domains by combining the outputs of multiple deep base learners using an integrated neural network meta-learner. The evaluation uses various NetFlow-based benchmark datasets and provides a detailed analysis of F1-score, accuracy, precision, and recall. To ensure a robust and transparent assessment, we compare the neural network meta-learner with alternative traditional meta-learners, including RF and LR. While not integrated into the architecture, these models provide valuable baselines for understanding the empirical benefits and architectural trade-offs of using a neural network meta-learner in the HDSE-IDS framework. 5.4.1 Proposed HDSE-IDS evaluation The evaluation results of the proposed HDSE-IDS model across multiple NFv2 datasets are presented in , covering accuracy, precision, recall, and F1-score. These metrics collectively demonstrate the enhanced generalisation capabilities achieved through the stacked ensemble design, which integrates the outputs of diverse base models (LSTM, GRU, MLP, and DNN) using a meta-learner. The ensemble achieved its highest performance on the NFv2-BoT-IoT dataset, reaching an accuracy of 0.915, a precision of 0.955, a recall of 0.950, and an F1-score of 0.952. This strong balance across all metrics indicates the model's effectiveness in handling the skewed attack distribution and accurately distinguishing between benign and malicious traffic. Similarly, on NFv2-ToN-IoT and NFv2-CIC-2018, the HDSE-IDS maintained high performance, achieving F1-scores of 0.900 and 0.874, respectively. Accuracy values of 0.919 and 0.884, and recall values of 0.855 and 0.930 support these results and reflect the model's strong ability to detect attacks in diverse network environments without overfitting. However, on NFv2-UNSW-NB15, the model's performance declined significantly, with an F1-score of 0.195, a recall of only 0.131, and an accuracy of 0.467. The dataset's highly imbalanced nature leads the model to favour the majority (benign) class, causing it to miss attack instances and perform poorly overall. To address this issue, SMOTE (Synthetic Minority Oversampling Technique) was applied to rebalance the training data by oversampling the minority class. As a result, the model's performance improved substantially. Accuracy increased to 0.789, precision reached 0.900, recall improved to 0.738, and the F1-score rose to 0.810. These findings confirm that applying class balancing significantly enhances detection performance on imbalanced datasets. The convergence behaviour of the HDSE-IDS model, with and without SMOTE, is shown in and . Without SMOTE (), both training and validation accuracy curves show consistent upward trends, though performance plateaus are lower. The corresponding loss curve declines steadily, indicating stable learning but with limited recall capacity. In contrast, the SMOTE-enhanced version () displays higher final accuracy and lower validation loss. The training and validation curves converge smoothly, indicating stronger generalisation and reduced overfitting. These trends confirm that integrating class balancing into the ensemble training pipeline significantly boosts the model's overall performance across all critical evaluation metrics (). 5.4.2 Alternative meta-learner evaluation To validate the selection of NN as the meta-learner in HDSE-IDS, we compared its performance against two traditional classifiers: RF and LR. The training process involved independently fitting each model on the outputs of frozen base learners and evaluating them on identical test datasets. The evaluation included both SMOTE and non-SMOTE configurations across all meta-learners to assess the influence of class balancing. The complete performance metrics, including accuracy, precision, recall, and F1-score, are presented in . In the non-SMOTE configuration, the NN meta-learner achieved strong results on three datasets: NFv2-BoT-IoT (accuracy: 0.915, precision: 0.955, recall: 0.950, F1-score: 0.952), NFv2-ToN-IoT (0.919, 0.952, 0.855, 0.900), and NFv2-CIC-2018 (0.884, 0.825, 0.930, 0.874). However, its performance dropped significantly on the highly imbalanced NFv2-UNSW-NB15 dataset, where it only achieved an F1-score of 0.195 with a low recall of 0.131 and accuracy of 0.467. Similar trends were observed for LR, which consistently struggled across datasets, particularly on NFv2-UNSW-NB15 (F1-score: 0.053), and NFv2-BoT-IoT (0.097), despite showing relatively higher recall (0.910) on NFv2-ToN-IoT. In contrast, RF demonstrated consistently strong performance in the non-SMOTE setting, achieving near-perfect scores on multiple datasets, such as 0.997 accuracy and F1-score on both NFv2-BoT-IoT and NFv2-CIC-2018, and 0.994 on NFv2-ToN-IoT. It also achieved the best results on NFv2-UNSW-NB15 among the three models, with an F1-score of 0.917 and a recall of 0.850, indicating better robustness to class imbalance even without oversampling. When SMOTE was applied to balance the NFv2-UNSW-NB15 dataset, the performance of NN improved substantially. Its accuracy rose to 0.789, its recall to 0.738, and its F1-score to 0.810. LR also benefited, increasing its F1-score from 0.053 to 0.238, though it still lagged behind the other two models. RF, which already performed well, showed only marginal gains, increasing its F1-score from 0.917 to 0.920. These results highlight that class balancing is especially beneficial for meta-learners more sensitive to skewed class distributions, such as NN and LR. Despite the empirically superior performance of RF across datasets, NN offers key architectural advantages. Its compatibility with end-to-end training, potential for joint optimisation with base learners, and extensibility into federated and continual learning scenarios make it a forward-compatible choice for advanced intrusion detection frameworks. Thus, the design choice of using NN in HDSE-IDS is both strategically and practically justified. 6 Discussion This section provides a detailed interpretation of the experimental results, focusing on understanding the generalisation behaviour of HDSE-IDS across heterogeneous domains. The discussion explores the impact of domain shift on model performance, the architectural benefits of the proposed ensemble strategy, and key observations from in-domain and cross-domain evaluations. This analysis focuses on why conventional single-domain learners struggle in unseen environments and explains how HDSE-IDS overcomes these limitations by leveraging structural diversity and aggregating predictions at the meta level. 6.1 Performance degradation in cross-domain The results of the cross-domain evaluation reveal substantial performance degradation across all evaluation metrics compared to in-domain results, highlighting the persistent challenge of achieving generalisation in ML-based network intrusion detection systems (NIDS). present the average decay values across four key metrics: accuracy, precision, recall, and F1-score, for each model and source dataset, offering a comprehensive view of this phenomenon. The evaluation calculated performance decay for each source dataset by measuring the difference between in-domain and cross-domain metrics across all target datasets, then averaged these values to estimate the generalisation loss for each model and dataset. For instance, the evaluation compared the GRU model's F1-scores on cross-domain targets (NFv2-BoT-IoT, NFv2-ToN-IoT, and NFv2-CIC-2018) against its in-domain F1-score on NFv2-UNSW-NB15, then averaged the differences to compute the F1 decay. The exact process was repeated across all models and performance metrics. Among the models, the LSTM showed the highest recall decay at 96.99%, and the highest F1-score decay at 94.98%, indicating its strong in-domain performance does not translate well to new domains. The GRU model followed closely with recall and F1 decays of 92.99% and 89.59%, respectively. Although these recurrent architectures are designed to model sequential data, they respond sensitively to distribution shifts between datasets. The DNN model exhibited more stable behaviour in terms of accuracy and precision (with decays of 54.65% and 70.20%, respectively), but still suffered an F1-score decay of 88.75%, indicating only slightly better generalisation than GRU and LSTM. Interestingly, MLP demonstrated the most consistent behaviour across accuracy (54.30% decay), precision (71.28%), and F1-score (87.03%), suggesting that its simplicity might confer some robustness to cross-domain shifts. From a dataset perspective, NFv2-UNSW-NB15 consistently had the highest average decay across all metrics: 57.14% (accuracy), 65.50% (precision), 77.82% (recall), and 74.11% (F1-score), reaffirming its difficulty as a generalisation source. This was followed by NFv2-BoT-IoT, which also showed severe average decays, particularly in recall (96.71%) and F1-score (94.09%). In contrast, NFv2-ToN-IoT and NFv2-CIC-2018 exhibited relatively lower decays across all metrics, especially in recall and F1-score, where both datasets reported decay rates under 99%, indicating comparatively better generalisation potential. These findings emphasise the domain-specific nature of network traffic characteristics and the limitations of conventional deep learning models in generalising across diverse network environments. The severe decay rates across all metrics underscore the need for more robust learning paradigms. Approaches such as stacked ensemble learning, domain adaptation, and federated training may provide pathways to mitigate these issues. Future research should focus on developing models capable of adapting to heterogeneous conditions and minimising sensitivity to dataset shifts, ultimately enhancing the reliability of NIDS in real-world multi-domain deployments. 6.2 Performance analysis of HDSE-IDS The results in highlight the comprehensive performance improvements achieved by the HDSE-IDS stacked ensemble model compared to the average cross-domain deep learning (DL) models. Improvements are observed across all key metrics-accuracy, precision, recall, and F1-score-demonstrating the ensemble's robustness in handling heterogeneity and adapting to varied traffic patterns. Among the datasets, the most substantial gains are reported for NFv2-BoT-IoT, with improvements of 42.32% in accuracy, 51.05% in precision, 88.40% in recall, and 89.76% in F1-score. Similarly, NFv2-ToN-IoT shows substantial gains across all metrics, particularly in recall (83.22%) and F1-score (83.94%), emphasising the ensemble's ability to generalise even in complex IoT environments. NFv2-CIC-2018 also benefits significantly, with notable gains in precision (46.72%) and recall (75.91%), leading to a 63.00% improvement in F1-score. These findings indicate that HDSE-IDS successfully integrates the strengths of its base learners to model cross-domain variations effectively. For NFv2-UNSW-NB15, the performance gain is minimal across most metrics (e.g. only 0.70% in accuracy and 2.80% in precision), and even shows a decline in recall (-25.20%) and F1-score (-18.38%). These results suggest that the ensemble model struggles with the dataset's severe class imbalance and complex feature distribution. However, after applying SMOTE, the improvements become much more pronounced-32.90% in accuracy, 54.60% in precision, 29.00% in recall, and 40.22% in F1-score-demonstrating the positive impact of balancing techniques in mitigating learning bias and enhancing generalisation. presents the performance degradation of HDSE-IDS relative to in-domain DL models. As expected, HDSE-IDS, which is optimised for cross-domain generalisation, shows some performance decline compared to models evaluated within their native training environments. However, this degradation is relatively modest for most datasets. For instance, the F1-score drops only 10.70% for NFv2-CIC-2018, 10.99% for NFv2-ToN-IoT, and 9.60% for NFv2-BoT-IoT, indicating that HDSE-IDS preserves competitive performance across domains. Corresponding drops in accuracy and precision also remain low, such as 7.79% and 4.89% for NFv2-ToN-IoT. On the other hand, degradation is more pronounced for NFv2-UNSW-NB15, with reductions of 53.13% in accuracy, 61.44% in precision, 89.10% in recall, and 82.93% in F1-score. The SMOTE-adjusted version improves these figures, reducing the F1-score degradation to 24.33% and recall drop to 34.90%, further supporting the utility of oversampling for imbalance correction. These combined results reinforce the value of HDSE-IDS as a generalisable solution for cross-domain intrusion detection. While minor trade-offs are present compared to in-domain performance, the ensemble achieves significant gains over standard cross-domain DL models. It adapts well with the aid of balancing techniques. This positions HDSE-IDS as a practical approach for real-world environments with prevalent data variability and heterogeneity. Moreover, running HDSE-IDS in the real world raises two practical issues: speed and scale. First, the system must keep up with live traffic. We can cut delay by grouping records before prediction, avoiding repeated feature work, and running feature extraction and model inference in parallel. We can also make the ensemble lighter by removing base models that add little value and by using smaller, compressed versions of the deep models. Second, the system must be able to handle busy periods and accommodate future growth. A straightforward approach is to run multiple identical prediction workers and add more when queues grow, then scale back when the load is low. Finally, network traffic changes over time, so it is essential to monitor for drift and refresh models on a regular schedule. These steps do not alter our method; they outline how to operate it within common speed and cost constraints. 6.3 Comparison with related work Recent intrusion detection research has increasingly focused on improving generalisation through techniques such as domain adaptation, ensemble learning, and federated strategies. Wardana et al. (Citation2024) proposed WEA-DNN, a weighted ensemble of deep neural networks for IoT intrusion detection. Bertoli et al. (de Carvalho Bertoli et al., Citation2023) developed DAE-EFC, an unsupervised federated approach using stacked denoising autoencoders to achieve privacy-preserving generalisation. Similarly, Layeghy et al. (Citation2023) introduced DI-NIDS, which uses adversarial domain-invariant training to enhance cross-domain robustness. While each of these methods addresses key aspects of generalisation, they come with trade-offs such as architectural rigidity, increased training complexity, or reliance on specialised protocols (e.g. federated or adversarial training). In contrast, HDSE-IDS employs a deep stacked ensemble of frozen base models and a lightweight meta-learner, offering a modular and easily deployable solution that does not depend on inter-domain coordination or shared optimisation. provides a quantitative comparison of HDSE-IDS with existing state-of-the-art methods using average cross-domain metrics. HDSE-IDS achieves a strong balance across all metrics, with an accuracy of 89.00%, precision of 91.55%, recall of 86.92%, and an F1-score of 88.77%. These results demonstrate its ability to generalise effectively while preserving predictive reliability and stability across domains. Among the other methods, DI-NIDS achieves a slightly higher F1-score at 89.04%, but lacks reported values for accuracy, precision, or recall, limiting direct interpretability. DAE-EFC delivers an F1-score of 84.00%, again without providing complete metric transparency, and it introduces added deployment overhead due to its federated architecture. WEA-DNN presents a high accuracy of 93.87%, but its relatively low recall (60.43%) and F1-score (67.88%) suggest difficulty detecting minority or novel classes across domains. In addition to comparing the average cross-domain F1-scores, we evaluated the key attributes of the related methods to provide a more comprehensive understanding of their design and capabilities, as shown in . Specifically, we examined whether each approach supports cross-domain evaluation, demonstrates generalisation ability, employs an ensemble-based structure, and incorporates heterogeneous base models. These attributes are critical in determining a method's suitability for deployment in diverse and evolving network environments. 7 Conclusion In this study, we introduced HDSE-IDS, a novel intrusion detection framework designed to address the challenges of heterogeneity and generalisation in IDS. By employing a deep stacked ensemble architecture that integrates diverse models-GRU, LSTM, DNN, and MLP-HDSE-IDS captures domain-specific decision boundaries while promoting robust performance across multiple network contexts. The model was evaluated using four benchmark NetFlow-based datasets: NFv2-UNSW-NB15, NFv2-BoT-IoT, NFv2-ToN-IoT, and NFv2-CIC-2018. Empirical results demonstrate that HDSE-IDS consistently achieves high detection performance in in- and cross-domain evaluations. In particular, it attained an average F1-score of 88.77% across cross-domain scenarios, outperforming several recent state-of-the-art methods such as WEA-DNN and DAE-EFC, and performing competitively against DI-NIDS, which achieved a slightly higher score of 89.04%. Applying SMOTE for class balancing significantly improved detection rates on highly imbalanced datasets, particularly NFv2-UNSW-NB15, where the F1-score improved by over 75%. These results underscore the adaptability and robustness of HDSE-IDS in addressing both class imbalance and distributional shifts between domains. By outperforming or matching current methods across diverse attack landscapes and domain boundaries, HDSE-IDS establishes itself as a strong candidate for real-world IDS deployment in heterogeneous and evolving network environments. Future work will focus on enhancing the scalability of HDSE-IDS by incorporating federated learning for decentralised training across distributed nodes. These improvements will help address the computational cost and resource demands of stacking by distributing model training and aggregation processes. Additionally, we plan to extend HDSE-IDS to support multi-class intrusion detection, enabling detection of a broader range of attack types beyond DoS. Moreover, integrating explainable AI (XAI) techniques can improve the transparency and interpretability of the system, fostering greater trust in high-stakes applications such as industrial IoT and cybersecurity operations. Through these advancements, we aim to contribute to developing resilient, adaptive, and generalisable intrusion detection solutions. Acknowledgements The authors would like to express their gratitude to Ajman University for their financial support under Internal Research Grant No. 2024-IRG-ENIT-15. Disclosure statement All authors declare that they have no conflicts of interest. Data availability statement The datasets used in this study are publicly available and can be accessed from the official repository: https://staff.itee.uq.edu.au/marius/NIDS_datasetsNFv2 NIDS Datasets. The processed versions of the datasets and relevant implementation details can be provided upon reasonable request. A public code repository for the full implementation is available at: https://github.com/ItzDeadShot/hdse-ids. Additional information Funding Notes 1 https://staff.itee.uq.edu.au/marius/NIDS_datasets/#v2 2 https://github.com/ItzDeadShot/hdse-ids References - Abdulmajeed, I. A., & Husien, I. M. (2022). MLIDS22- IDS design by applying hybrid CNN-LSTM model on mixed-datasets. Informatica (Slovenia), 46(8), 121–134. https://doi.org/10.31449/inf.v46i8.4348 - Ahmad, R., Alsmadi, I., Alhamdani, W., & Tawalbeh, L. (2022). A deep learning ensemble approach to detecting unknown network attacks. Journal of Information Security and Applications, 67, 103196. https://doi.org/10.1016/j.jisa.2022.103196 - Al-Ambusaidi, M., Yinjun, Z., Muhammad, Y., & Yahya, A. (2024). ML-IDS: An efficient ML-enabled intrusion detection system for securing IoT networks and applications. Soft Computing, 28(2), 1765–1784. https://doi.org/10.1007/s00500-023-09452-7 - Aljabri, M., Aljameel, S. S., Mohammad, R. M. A., Almotiri, S. H., Mirza, S., Anis, F. M., Aboulnour, M., Alomari, D. M., Alhamed, D. H., & Altamimi, H. S. (2021). Intelligent techniques for detecting network attacks: Review and research directions. Sensors, 21, 7070. https://doi.org/10.3390/s21217070 - Alotaibi, B., & Alotaibi, M. (2020). A stacked deep learning approach for IoT cyberattack detection. Journal of Sensors, 2020, 1–10. https://doi.org/10.1155/2020/8828591 - Alsaedi, A., Moustafa, N., Tari, Z., Mahmood, A., & Anwar, A. (2020). TON-IoT telemetry dataset: A new generation dataset of IoT and IIoT for data-driven intrusion detection systems. IEEE Access, 8, 165130–165150. https://doi.org/10.1109/ACCESS.2020.3022862 - Amara, M., Smairi, N., & Jaballah, M. (2025). Stacked ensemble deep learning for robust intrusion detection in IoT networks, Proceedings of the 17th International Conference on Agents and Artificial Intelligence (pp. 1146–1153). SCITEPRESS - Science and Technology Publications. https://doi.org/10.5220/0013290700003890 - Apruzzese, G., Pajola, L., & Conti, M. (2022). The cross-evaluation of machine learning-based network intrusion detection systems. IEEE Transactions on Network and Service Management, 19(4), 5152–5169. https://doi.org/10.1109/TNSM.2022.3157344 - Ashiku, L., & Dagli, C. (2021). Network intrusion detection system using deep learning. Procedia Computer Science, 185, 239–247. https://doi.org/10.1016/j.procs.2021.05.025 - Bajpai, S., Sharma, K., & Chaurasia, B. K. (2024). A hybrid meta-heuristics algorithm: XGBoost-based approach for IDS in IoT. SN Computer Science, 5(5), 537. https://doi.org/10.1007/s42979-024-02913-2 - Bajpai, S., Sharma, K., & Chaurasia, B. K. (2024). Anomaly detection in IoT networks using differential evolution and XGBoost. In Z. Illés, C. Verma, P. J. S. Gonçalves, & P. K. Singh (Eds.), Proceedings of International Conference on Recent Innovations in Computing (pp. 907–921). Springer. https://doi.org/10.1007/978-981-97-3442-9_64 - Bajpai, S., Sharma, K., & Chaurasia, B. K. (2025). Cross-Silo federated learning for IDS in IoT networks. In D. Bhattacharyya, & R. Ghosh (Eds.), EAI International Conference on Computational Intelligence and Generative AI (pp. 59–73). Springer. https://doi.org/10.1007/978-3-031-76610-7_5 - Breiman, L. (1996). Bagging predictors. Machine Learning, 24(2), 123–140. https://doi.org/10.1023/A:1018054314350 - Cao, Y., Wang, Z., Ding, H., Zhang, J., & Li, B. (2023). An intrusion detection system based on stacked ensemble learning for IoT network. Computers and Electrical Engineering, 110, 108836. https://doi.org/10.1016/j.compeleceng.2023.108836 - de Carvalho Bertoli, G., Alves Pereira Junior, L., Saotome, O., & dos Santos, A. L. (2023). Generalizing intrusion detection for heterogeneous networks: A stacked-unsupervised federated learning approach. Computers and Security, 127, 103106. https://doi.org/10.1016/j.cose.2023.103106 - Elijah, A. V., Abdullah, A., JhanJhi, N., Supramaniam, M., & Abdullateef, B. (2019). Ensemble and deep-learning methods for two-class and multi-attack anomaly intrusion detection: An empirical study. International Journal of Advanced Computer Science and Applications, 10(9). https://doi.org/10.14569/IJACSA.2019.0100969 - Frank, M., Drikakis, D., & Charissis, V. (2020). Machine-learning methods for computational science and engineering. Computation, 8, 15. https://doi.org/10.3390/computation8010015 - Koroniotis, N., Moustafa, N., Sitnikova, E., & Turnbull, B. (2019). Towards the development of realistic botnet dataset in the internet of things for network forensic analytics: Bot-IoT dataset. Future Generation Computer Systems, 100, 779–796. https://doi.org/10.1016/j.future.2019.05.041 - Layeghy, S., & Portmann, M. (2023). Explainable cross-domain evaluation of ML-based network intrusion detection systems. Computers and Electrical Engineering, 108(October 2022), 108692. https://doi.org/10.1016/j.compeleceng.2023.108692 - Layeghy, S., Baktashmotlagh, M., & Portmann, M. (2023). DI-NIDS: Domain invariant network intrusion detection system. Knowledge-Based Systems, 273, 110626. https://doi.org/10.1016/j.knosys.2023.110626 - Lazzarini, R., Tianfield, H., & Charissis, V. (2023). A stacking ensemble of deep learning models for IoT intrusion detection. Knowledge‐Based Systems, 279, 110941. https://doi.org/10.1016/j.knosys.2023.110941 - Mohammadpour, L., Ling, T. C., Liew, C. S., & Aryanfar, A. (2022). A survey of CNN-based network intrusion detection. Applied Sciences, 12, 8162. https://doi.org/10.3390/app12168162 - More, S., Idrissi, M., Mahmoud, H., & Asyhari, A. T. (2024). Enhanced intrusion detection systems performance with UNSW-NB15 data analysis. Algorithms, 17, 64. https://doi.org/10.3390/a17020064 - Moustafa, N., & Slay, J. (2015). UNSW-NB15: A comprehensive data set for network intrusion detection systems (UNSW-NB15 network data set), In: 2015 Military Communications and Information Systems Conference, MilCIS 2015 - Proceedings (pp. 1–6). IEEE. https://doi.org/10.1109/MilCIS.2015.7348942 - Muhammad, U., Laaksonen, J., Romaissa Beddiar, D., & Oussalah, M. (2024). Domain generalization via ensemble stacking for face presentation attack detection. International Journal of Computer Vision, 132(12), 5759–5782. https://doi.org/10.1007/s11263-024-02152-1 - Nguyen, D.-K., Chan, C.-L., Adams Li, A.-H., & Phan, D.-V. (2021). Deep stacked generalization ensemble learning models in early diagnosis of depression illness from wearable devices data, Proceedings of the 5th International Conference on Medical and Health Informatics. Icmhi ’21 (pp. 7–12). Association for Computing Machinery. https://doi.org/10.1145/3472813.3472815 - Oriola, O. (2020). A Stacked Generalization Ensemble Approach for Improved Intrusion Detection. International Journal of Computer Science and Information Security (IJCSIS), 18(5), 62–67. - Pernía-Espinoza, A., Fernandez-Ceniceros, J., Antonanzas, J., Urraca, R., & Martinez-de-Pison, F. J. (2018). Stacking ensemble with parsimonious base models to improve generalization capability in the characterization of steel bolted components. Applied Soft Computing, 70, 737–750. https://doi.org/10.1016/j.asoc.2018.06.005 - Polikar, R. (2006). Ensemble based systems in decision making. IEEE Circuits and Systems Magazine, 6(3), 21–45. - Rajadurai, H., & Gandhi, U. D. (2022). A stacked ensemble learning model for intrusion detection in wireless network. Neural Computing and Applications, 34(18), 15387–15395. https://doi.org/10.1007/s00521-020-04986-5 - Sarhan, M., Layeghy, S., & Portmann, M. (2022). Towards a standard feature set for network intrusion detection system datasets. Mobile Networks and Applications, 27(1), 357–370. https://doi.org/10.1007/s11036-021-01843-0 - Shtayat, M. M., Hasan, M. K., Sulaiman, R., Islam, S., & Khan, A. U. R. (2023). An explainable ensemble deep learning approach for intrusion detection in industrial internet of things. IEEE Access, 11, 115047–115061. https://doi.org/10.1109/ACCESS.2023.3323573 - Singh, A. P., Chaurasia, B. K., & Tripathi, A. (2024). Stacking enabled ensemble learning based intrusion detection scheme (SELIDS) for IoV. SN Computer Science, 5(8), 1000. https://doi.org/10.1007/s42979-024-03376-1 - Sowmya, T., & Mary Anita, E. A. (2023). A comprehensive review of AI based intrusion detection system. Measurement: Sensors, 28, 100827. https://doi.org/10.1016/j.measen.2023.100827 - Subhan, F., Wu, X., Bo, L., Sun, X., & Rahman, M. (2022). A deep learning-based approach for software vulnerability detection using code metrics. IET Software, 16(5), 516–526. https://doi.org/10.1049/sfw2.12066 - Thakkar, A., & Lohiya, R. (2020). A review of the advancement in intrusion detection datasets. Procedia Computer Science, 167, 636–645. https://doi.org/10.1016/j.procs.2020.03.330 - Wardana, A. A., Kołaczek, G., Warzyński, A., & Sukarno, P. (2024). Collaborative intrusion detection using weighted ensemble averaging deep neural network for coordinated attack detection in heterogeneous network. International Journal of Information Security, 23(5), 3329–3349. https://doi.org/10.1007/s10207-024-00891-3 - Wolpert, D. H. (1992). Stacked generalization. Neural Networks, 5(2), 241–259. https://doi.org/10.1016/S0893-6080(05)80023-1 - Yang, Z., Liu, X., Li, T., Wu, D., Wang, J., Zhao, Y., & Han, H. (2022). A systematic literature review of methods and datasets for anomaly-based network intrusion detection. Computers & Security, 116, 102675. https://doi.org/10.1016/j.cose.2022.102675 - Zhou, Z.-H. (2012). Ensemble Methods: Foundations and Algorithms. CRC Press. - Zlomislic, V., Fertalj, K., & Sruk, V. (2017). Denial of service attacks. Defences and Research Challenges, 20, 661–671.