SentiMatrix: Parameter-Efficient Fine-Tuning of Encoder-Based Transformers for Multidimensional Sentiment Analysis

Abstract Transformer-based architectures have established state-of-the-art benchmarks across Natural Language Processing (NLP) tasks; however, the computational overhead of full fine-tuning remains a significant barrier to scalable deployment. This paper presents SentiMatrix, a systematic evaluation of Low-Rank Adaptation (LoRA)-based Parameter-Efficient Fine-Tuning (PEFT) across four sentiment analysis paradigms: Intent-based, Aspect-based, Fine-grained, and Emotion detection, spanning seven benchmark datasets under a consistent three-stage protocol: inference using task-specific pretrained baseline models (i.e., HuggingFace checkpoints previously fine-tuned on the corresponding benchmarks, serving as informed upper-bound references rather than cold-start zero-shot baselines), Full Fine-Tuning (FFT) as a performance upper bound, and LoRA-based adaptation to quantify the efficiency-performance trade-off. Adaptive Low-Rank Adaptation (AdaLoRA) is benchmarked as a direct comparator. LoRA achieves competitive performance while reducing trainable parameters by up to 99.8%, with lower GPU memory usage and training time in most evaluated settings. It achieves 93.28% accuracy on SST-2, 80.74% on combined Laptop+Restaurant ABSA, 67.63% on E-Commerce 5-class, and 93.22% on emotion detection, while closely approaching FFT on E-Commerce 5-class (68.47%). The results demonstrate an effective efficiency-performance trade-off across diverse sentiment analysis tasks. Ablation experiments examine the effects of adaptation strategy, model capacity, and label granularity. The source code and datasets are available at https://github.com/ELTE-DSED/senti-matrix. Similar content being viewed by others 1 Introduction Sentiment analysis is a foundational task in Natural Language Processing (NLP) that aims to identify and extract subjective information, including opinions, attitudes, and emotional states, from written text. Lexicon-based and traditional machine learning methods dominated early sentiment analysis research (Yadollahi et al., 2017; Hu et al., 2018), before the field moved toward deep learning and, most recently, transformer-based architectures (Wankhade et al., 2022; Naithani & Raiwani, 2023), a shift driven by the explosive growth of user-generated content on digital platforms. The proliferation of social media, e-commerce platforms, and online review portals has generated an unprecedented volume of opinion-rich text. Harnessing this data has become strategically critical for applications such as customer feedback analysis, recommendation systems, market intelligence, and public opinion monitoring (Wankhade et al., 2022; Pang & Lee, 2008; Zimbra et al., 2018; Jia et al., 2026). These demands have collectively elevated sentiment analysis to one of the most actively studied problems in NLP. Early work in the field focused primarily on coarse-grained polarity classification, assigning text to positive, negative, or neutral categories. Real-world applications, however, require considerably more nuance. This has motivated the development of several complementary task formulations. In this work, intent-based sentiment analysis refers to sentiment-oriented classification where the model predicts the underlying sentiment polarity or communicative stance expressed in text, rather than dialogue-act or task-oriented intent classification. Intent-based sentiment analysis identifies the speaker’s underlying attitude or communicative intent (Tur & De Mori, 2011; Li et al., 2018). Aspect-based sentiment analysis (ABSA) determines polarity with respect to specific aspects or entities mentioned in the text, and has been extensively benchmarked through SemEval campaigns (Pontiki et al., 2016; Zhang et al., 2018). Fine-grained sentiment analysis models varying degrees of sentiment intensity via ordinal rating scales (Socher et al., 2013; Kiritchenko et al., 2014). Emotion detection goes beyond polarity to identify discrete affective states rooted in psychological theory (Ekman, 1992; Plutchik, 1980), with dedicated evaluation datasets and shared tasks (Mohammad et al., 2018). Figure 1 illustrates the four paradigms studied in this work. The recent dominance of large-scale pretrained language models (PLMs), including BERT (Devlin et al., 2019), RoBERTa (Liu et al., 2019), DistilBERT (Sanh et al., 2019), DeBERTa (He et al., 2020), and XLNet (Yang et al., 2019), has set new performance ceilings across virtually all NLP benchmarks, including SuperGLUE (Wang et al., 2019). These models leverage self-supervised pretraining on massive text corpora and are adapted to downstream tasks through fine-tuning. The Transformer architecture at its core enables efficient modeling of long-range dependencies through self-attention, replacing prior recurrent approaches (Vaswani et al., 2017). Despite these advances, two critical gaps remain in the literature. First, existing studies predominantly evaluate transformer models on a single sentiment task or dataset, precluding systematic cross-paradigm comparison. Hybrid architectures such as RoBERTa-LSTM (Tan et al., 2022) improved performance on individual benchmarks but are computationally expensive and do not generalize across task types. Second, full fine-tuning of large PLMs updates hundreds of millions of parameters, imposing prohibitive computational and memory costs. Parameter-efficient methods, including adapters (Houlsby et al., 2019) and Low-Rank Adaptation (LoRA) (Hu et al., 2022), addressed the cost barrier, but their systematic evaluation across diverse sentiment analysis paradigms remains underexplored (Dordevic & Stojkovic, 2024). Adaptive Low-Rank Adaptation (AdaLoRA) (Zhang et al., 2023) further extended LoRA by dynamically allocating the parameter budget across weight matrices based on importance scores derived via singular value decomposition, offering more targeted and importance-aware adaptation with comparable computational efficiency. This paper directly addresses both gaps. We present a unified evaluation framework that compares pretrained Transformer baselines, Full Fine-Tuning (FFT), and LoRA-based adaptation across four sentiment analysis paradigms and seven benchmark datasets, assessing both predictive performance and computational efficiency within a single, reproducible experimental setup. Adaptive Low-Rank Adaptation (AdaLoRA) is additionally benchmarked as a direct comparator. Experimental results demonstrate that LoRA and AdaLoRA achieve comparable accuracy overall, each outperforming the other on roughly half of the evaluated conditions, while LoRA consistently requires fewer trainable parameters and, in most settings, lower training time and memory footprint. The main contributions of this study are as follows: - The first systematic evaluation across four sentiment analysis paradigms, intent-based (binary and ternary), aspect-based, fine-grained (5-class), and emotion detection (6-class), is conducted using seven benchmark datasets under a consistent experimental protocol. The proposed SentiMatrix framework evaluates parameter-efficient LoRA adaptation across diverse sentiment tasks without task-specific architectural modifications. - Trade-offs among Pretrained Transformer baselines, Full Fine-Tuning (FFT), AdaLoRA, and LoRA-based PEFT are quantified across all evaluated tasks. LoRA reduces trainable parameters by up to 99.8% relative to FFT and consistently requires substantially less GPU memory, while reducing training time in most evaluated settings. Despite this lower computational footprint, LoRA maintains competitive predictive performance across diverse sentiment classification tasks, including 93.28% accuracy on SST-2, 80.74% on combined Laptop+Restaurant ABSA, 67.63% on E-Commerce 5-class, and 93.22% on emotion detection. - An ablation study over model capacity, label granularity, adaptation strategy, and linguistic domain is conducted to examine the factors influencing PEFT performance. The results demonstrate that LoRA provides an effective performance–efficiency trade-off, achieving substantial reductions in trainable parameters, GPU memory usage, and training time while retaining performance close to FFT on several tasks. Furthermore, collapsing five sentiment classes into three produces substantial accuracy improvements, with LoRA gains of 26.98 percentage points on Yelp and 17.33 26.98 percentage points on E-Commerce. The remainder of the paper is organized as follows. Section 2 surveys related work on transformer-based sentiment analysis and parameter-efficient fine-tuning. Section 3 details the methodology, including the system architecture, datasets, and the three adaptation strategies, with AdaLoRA included as a direct comparator. Section 4 presents quantitative results across all tasks. Section 5 presents cross-task analysis, an ablation study, and SOTA comparisons. Section 6 concludes with a summary of findings and future directions. 2 Related Work This section reviews the literature directly relevant to the present study, organized into three thematic areas: transformer-based sentiment analysis across task formulations, efficient adaptation of pretrained language models, and the research gap that motivates our work. 2.1 Transformer-Based Sentiment Analysis Across Task Formulations Prior to the transformer era, sentiment analysis methods ranged from lexicon-based approaches to classical machine learning and early deep learning architectures (Yadollahi et al., 2017; Wankhade et al., 2022; Naithani & Raiwani, 2023; Pang & Lee, 2008; Wang et al., 2016). The Transformer architecture introduced by Vaswani et al. (2017) fundamentally reshaped NLP by establishing self-attention as the dominant paradigm for sequence modeling. Devlin et al. subsequently introduced BERT (Devlin et al., 2019), a deep bidirectional Transformer pre-trained via masked language modeling and next-sentence prediction, which achieved state-of-the-art results across a broad range of NLP benchmarks, including sentiment classification. Liu et al. proposed RoBERTa (Liu et al., 2019), a robustly optimized BERT variant trained with larger batches and without the next-sentence prediction objective, yielding consistent gains across classification tasks. Sanh et al. introduced DistilBERT (Sanh et al., 2019), a knowledge-distilled compression of BERT that retains 97% of its language-understanding capability with a 40% reduction in model size, making it well-suited to resource-constrained settings. He et al. further advanced pre-training with DeBERTa (He et al., 2020), which incorporates disentangled attention over content and position embeddings, along with an enhanced mask decoder, yielding improvements on tasks requiring fine-grained linguistic reasoning. At the task level, Tan et al. proposed RoBERTa-LSTM (Tan et al., 2022) for intent-based sentiment classification, combining RoBERTa embeddings with a recurrent layer to capture long-distance dependencies, reporting macro-F1 scores of 93%, 91%, and 90% on IMDb, Twitter US Airline Sentiment, and Sentiment140, respectively. Jain et al. designed BERT-DCNN (Jain et al., 2023), which stacks BERT with parallel dilated convolutional layers for enhanced feature extraction, achieving 87.1% accuracy on the Twitter US Airline Sentiment dataset. The unique challenges of Twitter sentiment analysis, including informal language, emojis, hashtags, and class imbalance, have been extensively studied by Zimbra et al. (2018), benchmarked 28 academic and commercial systems and found that classification accuracies frequently fell below 70%, underscoring the difficulty of the task before the transformer era. Naseem et al. presented DICET (Naseem et al., 2020), a transformer-based framework augmented with deep contextual embeddings and an intelligent pre-processing pipeline to handle noisy social media text. Similarly, Pota et al. (2021) introduced a BERT-based two-step pipeline for Twitter sentiment analysis that first normalises tweet jargon, including emojis and emoticons, into plain text before fine-tuning, demonstrating strong performance on Italian Twitter data and highlighting the importance of domain-specific preprocessing for informal social media text. The TweetEval benchmark of Barbieri et al. (2020) introduced a unified evaluation suite for multiple Twitter classification tasks, establishing standardized baselines for intent and emotion analysis. More recently, Luoyao introduced the Dual Joint Classifier (DJC) (He, 2024), which integrates RoBERTa and BERT encoders with BiGRU and BiLSTM layers alongside focal loss and hard sample mining, achieving 81.67% accuracy on TweetEval sentiment, illustrating the continued interest in hybrid architectures for social media text. Prior to transformer-based approaches, Wang et al. (2016) introduced an attention-based LSTM for aspect-level sentiment classification, demonstrating that attention mechanisms could effectively capture aspect-relevant sentiment expressions, achieving strong results on the SemEval 2014 benchmark. For aspect-based sentiment analysis, Hoang et al. (2019) formulated ABSA as sentence-pair classification using BERT, enabling joint modeling of review text and target aspect semantics, with strong results on SemEval 2015 and 2016. Phan and Ogunbona (Phan & Ogunbona, 2020) enriched transformer representations with part-of-speech and dependency parse features, achieving state-of-the-art performance on the SemEval 2014 benchmarks. Recetly, Yang and Li (2024) proposed a local sentiment aggregation (LSA) paradigm that models aspect sentiment coherence by constructing a differential-weighted sentiment aggregation window over adjacent aspects, achieving 90.33% accuracy on the SemEval-2014 Restaurant subset with DeBERTa-v3-Large and establishing the current state of the art on that benchmark. Zhang et al. (2022) provided a comprehensive survey of ABSA methods and challenges. Wang proposed SS-Transformer (Wang, 2026), a structural-aware framework that incorporated syntactic structure alongside semantic content for ABSA in academic linguistics texts, demonstrating the applicability of transformer-based ABSA beyond standard review and social media domains. More recently, Šmíd et al. (2024) demonstrated that open-source LLMs fine-tuned with QLoRA can surpass state-of-the-art encoder-based models on compound ABSA tasks, highlighting the growing potential of generative approaches in this space. In the fine-grained setting, Almufareh et al. proposed BertSent (Almufareh et al., 2024), a BERT model fine-tuned for five-class Twitter sentiment classification using ordered preprocessing and resampling to address class imbalance. For emotion detection, Delbrouck et al. introduced a joint multimodal encoding framework achieving competitive performance on CMU-MOSEI (Delbrouck et al., 2020). The GoEmotions benchmark (Demszky et al., 2020) demonstrated a large-scale fine-grained emotion classification resource covering 28 categories. 2.2 Efficient Adaptation of Pretrained Language Models As transformer-based models scale to hundreds of millions of parameters, adapting them to downstream tasks through full fine-tuning becomes increasingly costly in terms of computation, memory, and storage. This has motivated a growing body of research into parameter-efficient fine-tuning (PEFT) methods that update only a small subset of model parameters while preserving the pretrained representations (Ding et al., 2023; Wang et al., 2025). Sun et al. (2019) characterized foundational guidance on fine-tuning BERT for text classification, analyzing the effects of learning rate scheduling, layer selection, and task-specific objectives. However, full fine-tuning updates all model parameters and becomes increasingly impractical as model size grows, motivating the development of parameter-efficient alternatives. Houlsby et al. (2019) introduced adapter modules, compact bottleneck layers inserted between transformer blocks, that enable task-specific adaptation while keeping the base model weights frozen, substantially reducing the number of trainable parameters. He et al. (2022) unify adapter tuning, prefix tuning, and low-rank adaptation under a generalized framework. Ding et al. (2023) conducted a comprehensive survey of delta tuning techniques, offering theoretical grounding and practical guidance for efficient adaptation strategy selection. Recently, Wang et al. (2025) conducted another comprehensive survey of PEFT methodologies for large language models, covering over 100 PEFT approaches, including LoRA, AdaLoRA, adapter tuning, prefix tuning, and prompt tuning, and situating these methods within the broader context of LLM scaling and sustainability of AI research. Among PEFT methods, Low-Rank Adaptation (LoRA) (Hu et al., 2022) has emerged as the most widely adopted approach. LoRA freezes pretrained weights and injects trainable low-rank decomposition matrices into transformer attention layers, achieving effective adaptation with as few as 0.1% of the original parameter count. Dordevic and Stojkovic (2024) applied LoRA to XLM-RoBERTa for English and Serbian sentiment classification, demonstrating that LoRA matches full fine-tuning performance while substantially reducing training time and memory usage, particularly in low-resource language settings. In 2024, Hosseini et al. (2024) proposed a resource-efficient fine-tuning approach that combines LoRA with dataset pruning for transformer-based language models. Their method reduces both the number of trainable parameters and the amount of training data while maintaining performance comparable to full fine-tuning. Experimental results on GLUE (Wang et al., 2019) tasks showed significant reductions in memory usage and training time, demonstrating the effectiveness of integrating parameter-efficient and data-efficient learning techniques. Later on, Duru and Sunar (2025) evaluated sentiment analysis models across traditional machine learning, pre-transformer deep learning, and transformer-based architectures using Amazon Magazine Subscription Reviews (2023). Among all models, DistilBERT achieved the highest accuracy of 92% with the lowest cross-entropy loss of 0.25, while FastText consistently outperformed GloVe due to its subword-level semantic richness. Several advanced variants of LoRA have been proposed to address specific limitations of fixed-rank adaptation. QLoRA (Dettmers et al., 2023) combined LoRA with 4-bit quantization to enable fine-tuning of foundation-scale models under extreme memory constraints, but is primarily motivated by decoder-based generative models rather than encoder-based classifiers. DyLoRA (Valipour et al., 2023) trained adapters across a range of ranks simultaneously to avoid costly rank-search, while AdaLoRA (Zhang et al., 2023) allocated the parameter budget dynamically across weight matrices via SVD-based importance scoring. LoRA is selected as the proposed adaptation strategy in this work for three reasons: (i) it is architecture-agnostic and directly applicable to all encoder families evaluated, namely BERT, RoBERTa, DistilBERT, and DeBERTa; (ii) its fixed-rank constraint provides implicit regularization that is particularly effective under the domain shift and label noise characteristic of sentiment corpora; and (iii) it represents the most widely adopted PEFT baseline, ensuring reproducibility and direct comparability with prior work. AdaLoRA is nonetheless included as a direct empirical comparator to quantify whether importance-aware rank allocation provides additional gains in sentiment classification settings, with results reported across all seven benchmarks. 2.3 Research Gap Notwithstanding this body of work, and despite comprehensive surveys of the field (Yadollahi et al., 2017; Wankhade et al., 2022; Naithani & Raiwani, 2023; Zhang et al., 2022; Ding et al., 2023), a critical gap persists: no study has simultaneously benchmarked pretrained baseline inference, FFT, and LoRA-based adaptation across multiple sentiment analysis paradigms, intent-based, aspect-based, fine-grained, and emotion detection, within a single unified experimental framework. Furthermore, existing work rarely reports a joint analysis of both predictive performance and computational efficiency, and ablation experiments isolating individual design factors are largely absent. This gap leaves practitioners without a principled, reproducible basis for selecting adaptation strategies in practical deployments. The present work directly addresses this by providing a comprehensive cross-paradigm evaluation across seven datasets and four task types. 3 Methodology The methodology is designed to rigorously evaluate transformer-based architectures across the full spectrum of sentiment analysis complexity, from binary polarity detection to six-class emotion recognition. The experimental framework follows a four-condition design: leveraging task-specific pretrained baseline models, performing full fine-tuning as an upper-bound reference, applying Low-Rank Adaptation (LoRA) for parameter-efficient adaptation, with Adaptive Low-Rank Adaptation (AdaLoRA) included as a direct comparator to contextualise LoRA’s efficiency-performance advantages, to enable direct, controlled comparison of each strategy’s impact on both classification performance and computational efficiency. All reported efficiency metrics in this study correspond to training-time efficiency, including trainable parameter count, peak GPU memory usage during training, and training time. Inference-time efficiency is not directly measured and depends on deployment configuration, such as whether LoRA adapters are merged into the base model or served separately. 3.1 Problem Formulation Let \(\mathcal {D} = \{(\textbf{x}_i, y_i)\}_{i=1}^{N}\) denote a labelled dataset where \(\textbf{x}_i \in \mathcal {X}\) is a natural language input and \(y_i \in \mathcal {Y}\) is a discrete sentiment label. The label space varies by paradigm: \(|\mathcal {Y}| \in \{2, 3, 5, 6\}\) for binary intent, ternary intent/aspect, fine-grained, and emotion detection tasks, respectively. A pretrained encoder \(f_{\varvec{\theta }}: \mathcal {X} \rightarrow \mathbb {R}^{|\mathcal {Y}|}\) is adapted by minimising the cross-entropy loss: where \(\varvec{\phi } \subseteq \varvec{\theta }\) is the subset of parameters updated during training. FFT sets \(\varvec{\phi } = \varvec{\theta }\) (all P parameters); LoRA constrains updates to \(\varvec{\phi } = \{\textbf{B}, \textbf{A}\}\) with \(|\varvec{\phi }| = r(d+k) \ll dk\); and AdaLoRA further restricts \(\varvec{\phi } = \{\textbf{P}, \varvec{\Lambda }, \textbf{Q}\},\) pruning singular values by importance score \(s_k = |\lambda _k|\cdot \Vert \textbf{p}_k\Vert \cdot \Vert \textbf{q}_k\Vert \) to concentrate the budget on the most informative weight components. 3.2 Proposed Model The proposed framework evaluates transformer-based models under three adaptation settings within a single unified pipeline: (i) pretrained baseline inference with task-specific pretrained models, (ii) Full Fine-Tuning (FFT) of base transformer architectures, and (iii) parameter-efficient fine-tuning via LoRA, with AdaLoRA included as a direct comparator to quantify the relative advantages of fixed versus importance-aware rank allocation. This design isolates the effect of each adaptation strategy while holding all other experimental factors constant, enabling fair performance and efficiency comparison. Figure 2 depicts the complete experimental workflow, from heterogeneous dataset ingestion and standardized preprocessing through the three modeling paradigms to a dual-axis evaluation. 3.3 Dataset Descriptions Seven publicly available benchmark datasets are employed, spanning four sentiment analysis paradigms with different domains, text lengths, class granularities, and linguistic styles. This diversity ensures that findings generalize beyond any single task or domain. Table 1 summarizes the statistical properties of each dataset. Binary sentiment classification is evaluated on two complementary benchmarks: SST-2 (Socher et al., 2013), which provides sentence-level annotations from movie reviews, and the IMDb Movie Reviews dataset (Maas et al., 2011), which requires document-level understanding of long-form reviews. Ternary classification is evaluated on the Twitter US Airline Sentiment dataset (Rane & Kumar, 2018), capturing positive, neutral, and negative classes in informal social media text. Sentiment labels were encoded as 0 (negative), 1 (neutral), and 2 (positive) for model training. ABSA is evaluated using the SemEval-2014 Task 4 benchmark (Pontiki et al., 2016), covering the Laptop and Restaurant domains. Each sample carries aspect-level sentiment annotations (Positive, Neutral, Negative), requiring models to associate polarity with specific target entities. Domain-specific analyses are conducted on each subset independently to assess cross-domain generalization. For the IMDb dataset, we further analyze the effect of document length and observe that many reviews exceed the 512-token limit, leading to truncation during preprocessing. This results in partial loss of contextual information in longer reviews, which negatively impacts classification performance. We find that errors are more frequent in longer inputs, indicating a clear correlation between sequence length and prediction difficulty. This helps explain the relative performance drop of LoRA compared to full fine-tuning on IMDb. Fine-grained sentiment is evaluated on the Yelp Reviews dataset (Alamoudi & Alghamdi, 2021) and the Amazon Women’s E-Commerce Reviews dataset (Zhang et al., 2015), both using a 5-star rating scale (Huang et al., 2023). To quantify the effect of label granularity, each dataset is additionally evaluated under a collapsed 3-class schema (Negative: 1–2 stars; Neutral: 3 stars; Positive: 4–5 stars). Emotion detection is evaluated on the CARER dataset (Saravia et al., 2018), a large-scale Twitter corpus annotated with six discrete affective categories: sadness, joy, love, anger, fear, and surprise. The 75,000-instance subset used in this study was drawn from the larger unsplit configuration of the CARER dataset (416,809 instances), from which the standard 80%/10%/10% split was applied. This task requires models to capture subtle linguistic distinctions between affective states that share overlapping sentiment polarity. 3.4 Pretrained Transformer Models Model selection was governed by the principle of architectural alignment with task-specific characteristics, including text length, linguistic register, and label granularity. Table 2 provides the architectural specifications of each deployed model. For binary SST-2 classification, DistilBERT (Sanh et al., 2019) (66 M parameters) provides an efficient baseline, while RoBERTa (Liu et al., 2019) offers a higher-capacity alternative. BERT (Devlin et al., 2019) is selected for the document-level IMDb task to capture long-range discourse structure. A RoBERTa variant pre-trained on Twitter sentiment data (Barbieri et al., 2020) addresses the informal register of the Twitter dataset. DeBERTa-v3 (He & Gao, 2023) is chosen for ABSA due to its disentangled attention mechanism, which is well-suited to aspect-level entity association. Fine-grained 5-class tasks are handled by a multilingual BERT variant optimized for star-rating prediction, and emotion detection employs a DistilBERT model pre-trained on emotion recognition data. This careful model-task matching ensures that each pretrained baseline constitutes a meaningful reference rather than an arbitrary one. 3.5 Full Fine-Tuning of Base Models Full Fine-Tuning (FFT) updates all parameters of the base transformer architecture on the target dataset, allowing the model to fully specialize its pretrained representations to the downstream sentiment distribution. Following the fine-tuning protocol of Sun et al. (2019), all layers from the embedding module through the classification head are updated jointly via gradient descent on the cross-entropy objective defined in Sect. 3, with architecture-specific learning rates and regularization as detailed in Table 3. While this allows each encoder, BERT (Devlin et al., 2019), RoBERTa (Liu et al., 2019), DistilBERT (Sanh et al., 2019), and DeBERTa-v3 (He & Gao, 2023), to fully adapt its attention weights and feed-forward layers to the target sentiment distribution, it requires storing the full gradient and optimizer states for all P parameters throughout training (Houlsby et al., 2019; Ding et al., 2023), resulting in peak GPU memory of 2.0-−4.2 GB and training times of 210–1,853 s across the seven benchmarks. This setting incurs the maximum computational cost among the three strategies, and serves as the performance upper bound against which parameter-efficient methods are evaluated. 3.6 Parameter-Efficient Fine-Tuning with LoRA LoRA (Hu et al., 2022) addresses the cost of full fine-tuning by decomposing weight updates into low-rank matrices. Formally, for a pretrained weight matrix \(\textbf{W}_0 \in \mathbb {R}^{d \times k},\) LoRA constrains the update \(\Delta \textbf{W}\) as: where \(\textbf{B} \in \mathbb {R}^{d \times r},\) \(\textbf{A} \in \mathbb {R}^{r \times k},\) and \(r \ll \min (d, k)\) is the rank hyperparameter. During training, \(\textbf{W}_0\) is frozen and only \(\textbf{B}\) and \(\textbf{A}\) are updated, reducing the number of trainable parameters from \(d \times k\) to \(r(d + k)\). LoRA adapters were primarily applied to the query and value projection matrices of all multi-head attention layers in BERT, RoBERTa, DistilBERT, and DeBERTa, with key and/or output projections additionally included for select configurations, as detailed in Table 3b. Rank and scaling hyperparameters were tuned per task to ensure stable convergence. Figure 3 illustrates the LoRA adaptation mechanism. 3.7 Benchmarking AdaLoRA as a Comparative Baseline As motivated in Sect. 2, AdaLoRA (Zhang et al., 2023) is included as a direct empirical comparator to LoRA to assess whether importance-aware rank allocation yields additional gains over fixed-rank adaptation in sentiment classification settings. While other variants such as QLoRA (Dettmers et al., 2023) and DyLoRA (Valipour et al., 2023) were considered, they are primarily designed for decoder-based generative models and dynamic rank search, respectively, making AdaLoRA the most architecturally comparable alternative to LoRA for encoder-based sentiment classification. Unlike LoRA, which applies a uniform rank across all weight matrices, AdaLoRA dynamically allocates the parameter budget based on each matrix’s importance, parameterizing the weight update \(\Delta \textbf{W}\) via singular value decomposition (SVD) as: where \(\textbf{P} \in \mathbb {R}^{d \times r}\) and \(\textbf{Q} \in \mathbb {R}^{r \times k}\) are the left and right singular vectors, and \(\varvec{\Lambda } = \text {diag}(\lambda _1, \lambda _2, \ldots , \lambda _r)\) is a diagonal matrix of singular values representing the importance of each rank component. During training, singular values are pruned iteratively based on an importance score, allowing the model to concentrate its parameter budget on the most informative weight matrices while discarding redundant components. AdaLoRA adapters are applied to the same target modules as LoRA for each dataset configuration in BERT, RoBERTa, DistilBERT, and DeBERTa, as detailed in Table 3c, supporting a closely aligned comparison across both adaptation strategies, as illustrated in Fig. 4. 3.8 Experimental Setup This section details the training configuration, data partitioning strategy, and evaluation framework used across all experiments. All procedures were held constant across the three adaptation strategies to ensure fair and reproducible comparison. 3.8.1 Training Configuration All experiments were implemented using the HuggingFace Transformers library on PyTorch. Each dataset was partitioned into training, validation, and test sets following an 80%/10%/10% ratio, applied consistently across all tasks. Optimization was performed using AdamW with task-specific learning rates: \(1 \times 10^{-5}\) to \(2 \times 10^{-5}\) for FFT, and higher rates for LoRA-based training to compensate for the reduced parameter count. Batch sizes were adjusted according to available GPU memory. Models were trained for a fixed number of epochs with early stopping based on validation performance (macro-F1-score or loss, depending on the task) to prevent overfitting. Regularization was applied consistently through dropout and weight decay across all settings. Mixed-precision training (fp16) was enabled throughout to maximize computational throughput. For each setting, the best-performing checkpoint on the validation set was retained for final test evaluation. Full details of all hyperparameter configurations across FFT, LoRA, and AdaLoRA are reported in Table 3. The LoRA rank r was selected per dataset through validation-based tuning over a fixed candidate set \(r \in \{2,4,6,8,16\}\). For each task, we chose the configuration that achieved the best validation accuracy while maintaining stable convergence. As shown in Table 3b, lower ranks are sufficient for simpler or smaller datasets (e.g., IMDb, Emotion), while moderately higher ranks benefit more complex or noisy datasets (e.g., Twitter Reviews). This indicates that the performance gains of LoRA are consistent across different rank settings and are not solely driven by increased model capacity, but by the efficiency of low-rank adaptation. A systematic ablation study on the effect of rank r is left as future work to further quantify its impact on performance–efficiency trade-offs. 3.9 Evaluation Metrics Model evaluation covers two complementary dimensions: classification effectiveness and computational efficiency. Classification performance is quantified using four standard metrics derived from the confusion matrix elements, True Positives (TP), True Negatives (TN), False Positives (FP), and False Negatives (FN), as follows: For multi-class settings (3-class, 5-class, 6-class), metrics are computed per class using a one-versus-rest strategy and aggregated via macro-averaging, which gives equal weight to each class and mitigates the influence of class imbalance. To capture predictive calibration beyond hard-decision accuracy, we additionally report the Confidence Score (\(\mathcal {C}\)), defined as the mean softmax probability assigned to the ground-truth class over all test instances: where \(P(y_{\text {true}}^{(i)} \mid \textbf{x}^{(i)})\) is the softmax probability for the correct class on the i-th sample. A high \(\mathcal {C}\) indicates not only correct predictions but also well-calibrated decision boundaries. Notably, \(\mathcal {C}\) can reveal latent uncertainty even when accuracy remains high, a distinction that standard metrics do not expose. Prediction alignment is assessed through the Similarity Score, computed as the cosine similarity between the model’s predicted probability distribution \(\textbf{p} \in \mathbb {R}^{C}\) and the one-hot representation of the ground-truth class \(\textbf{y} \in \mathbb {R}^{C},\) where C denotes the number of classes: The Similarity Score quantifies the degree of alignment between the predicted probability distribution and the target class. Unlike confidence, which reflects only the maximum predicted probability, this metric accounts for the entire probability distribution, offering a complementary perspective on prediction quality. Higher values indicate that the model concentrates a greater proportion of probability mass on the correct class while preserving a distribution consistent with the target label. 3.10 Efficiency Metrics Three resource-oriented metrics are recorded to quantify computational cost: - Trainable Parameters (\(P_{\text {train}}\)): the number of model parameters updated during training, in millions (M) or thousands (K). FFT updates all weights; LoRA updates only the injected low-rank matrices. - Training Time (\(T_{\text {train}}\)): total wall-clock time in seconds from training initiation to convergence, measured on the same hardware across all experiments. - Peak GPU Memory (\(M_{\text {peak}}\)): maximum GPU memory consumption in gigabytes during training, including model parameters, optimizer states, gradients, and intermediate activations. 4 Results This section reports the quantitative outcomes of all experiments, organized by sentiment analysis paradigm. All tables follow a consistent four-row structure per model family: pretrained baseline (task-aligned pretrained model, no target-dataset training), Full Fine-Tuning (FFT); all parameters updated, LoRA (Ours); only low-rank adapter matrices updated, and AdaLoRA; importance-aware rank allocation, direct comparator. Results are visualized as paired radar charts (performance profile) and bar charts (efficiency profile), supplemented by detailed numerical tables. Per-class confusion matrices and training/validation loss curves for all models and datasets are provided in Supplementary Information. 4.1 Intent-Based Sentiment Analysis (2-Class) Binary intent-based sentiment analysis classifies text as either positive or negative. We evaluate models on two structurally distinct benchmarks, the sentence-level SST-2 and the document-level IMDb dataset, in Table 4, to probe performance across different input length regimes. 4.1.1 SST-2 Dataset SST-2 consists of short, grammatically complete sentences with clear binary sentiment annotations. Two pretrained baselines, DistilBERT and RoBERTa, are each compared against their respective FFT and parameter-efficient LoRA and AdaLoRA variants. Figures 5 and 6 display both performance and efficiency profiles for the respective models, while Table 4 reports the numerical results. 4.1.2 IMDb Dataset IMDb reviews are substantially longer than SST-2 sentences, requiring models to maintain coherent sentiment representations across multi-sentence text. The BERT model is evaluated using the pretrained baseline, FFT, LoRA, and AdaLoRA configurations. Results are shown in Fig. 7 and Table 4. 4.2 Intent-Based Sentiment Analysis (3-Class) Ternary classification requires models to discriminate between clearly polarized sentiment and ambiguous or neutral statements, a substantially harder task than binary classification. We evaluated pre-training baseline, FFT, LoRA, and AdaLoRA conditions in the Twitter US Airline Sentiment dataset, with performance and efficiency profiles shown in Fig. 8 and complete numerical results consolidated in Table 4. 4.2.1 Twitter Dataset The Twitter dataset consists of short, informal texts containing emojis, hashtags, and domain-specific vocabulary, presenting a challenging setting for pretrained models. Figure 8 presents the performance and efficiency profiles, with numerical results reported in Table 4. The pretrained baseline models achieve strong performance on SST-2 and IMDb, reflecting their prior task-specific adaptation to these datasets. Therefore, these results should be interpreted as reference points rather than unseen evaluations. In contrast, the Twitter results demonstrate the importance of task-specific adaptation under domain shift, where FFT substantially outperforms the pretrained baseline. Accordingly, the primary comparison remains between FFT and the parameter-efficient LoRA and AdaLoRA approaches, all initialized from the same general-purpose base architecture. 4.3 Aspect-Based Sentiment Analysis (3-Class) Aspect-Based Sentiment Analysis (ABSA) assigns sentiment to specific aspects of entities mentioned in the text, requiring models to resolve aspect-level context and polarity. We evaluate DeBERTa-v3 under pretrained baseline, FFT, LoRA, and AdaLoRA conditions on the combined SemEval-2014 benchmark and on each domain independently. Performance and efficiency profiles are shown in Figs. 9, 10, and 11, with numerical results reported in Table 5. 4.3.1 SemEval 2014 Task 4 Dataset We first evaluate the combined Laptop and Restaurant benchmark, with performance and efficiency profiles shown in Fig. 9 and numerical results reported in Table 5. Domain-specific analyses for the Laptop and Restaurant subsets are presented separately in Figs. 10 and 11. 4.3.2 Domain-Specific Analysis The Laptop and Restaurant domains differ substantially in linguistic style: Laptop reviews are technical and structured, whereas Restaurant reviews are more experiential and stylistically varied. Figures 10 and 11 present the performance and efficiency profiles for both domains. 4.4 Fine-Grained Sentiment Analysis (5-Class) Fine-grained sentiment classification models sentiment intensity on a five-point scale (Very Negative to Very Positive), making it challenging to distinguish between adjacent sentiment categories. To examine the effect of label granularity, a collapsed 3-class setting is also evaluated for the E-Commerce and Yelp datasets. 4.4.1 E-Commerce Reviews Dataset The E-Commerce dataset is evaluated using mBERT in the native 5-class setting and RoBERTa in the collapsed 3-class setting. Figures 12 and 13, together with Fig. 14, present the results, with numerical details reported in Table 6. Figure 14 provides a class-wise analysis of prediction behavior across the four model configurations. The results illustrate the classification difficulty associated with distinguishing fine-grained sentiment categories. Overall, the adapted models provide improved performance over the pretrained baseline, with FFT achieving the highest accuracy in the 5-class setting and LoRA achieving the highest accuracy in the collapsed 3-class setting. 4.4.2 Yelp Reviews Dataset The Yelp dataset provides a larger-scale evaluation of fine-grained sentiment classification. mBERT is evaluated in the native 5-class setting, while RoBERTa is evaluated in the collapsed 3-class setting. Figures 15 and 16, together with Fig. 17, present the results, with numerical details reported in Table 6. Figure 17 provides a class-wise analysis of model behavior on the Yelp dataset. The confusion matrices illustrate the prediction patterns across the five sentiment categories and the extent of class-level confusion among the model variants. The quantitative results show that AdaLoRA achieves the strongest performance in the 5-class setting, whereas FFT performs best in the collapsed 3-class setting. 4.5 Emotion Detection (6-Class) Emotion detection differentiates six discrete affective states that may overlap in sentiment polarity while remaining linguistically distinct. We evaluate DistilBERT variants on the CARER dataset. Results are presented in Figs. 18, 19, and Table 7. Figure 19 provides a class-wise analysis of model behavior across all four configurations. The confusion matrices demonstrate that predictions are largely concentrated along the principal diagonal, confirming effective discrimination among the six emotion categories. The remaining misclassifications occur primarily between emotions with overlapping semantic characteristics, particularly sadness, fear, and anger. Compared with the pretrained baseline, FFT, LoRA, and AdaLoRA exhibit fewer off-diagonal errors, indicating improved class separation after adaptation. Notably, the LoRA and AdaLoRA variants achieve confusion patterns comparable to full fine-tuning while updating only a small fraction of model parameters, further supporting the effectiveness of parameter-efficient adaptation for multi-class emotion recognition. 5 Discussion This section synthesizes the experimental findings from Sect. 4, examines cross-task behavioral trends, presents an ablation study over three key factors, and contextualizes the proposed models against contemporary state-of-the-art methods. It further provides an intuitive explanation of the effectiveness of LoRA-based adaptation compared to full fine-tuning and AdaLoRA, highlighting the role of low-rank constraints as an implicit regularizer that can improve generalization under domain shift, label noise, and limited data conditions. 5.1 Analysis of Adaptation Strategies Three distinct behavioral regimes emerge across all tasks. Pretrained baselines perform well when the target distribution closely mirrors the pretraining objective, as evidenced by strong performance on SST-2 (DistilBERT: 98.13%, RoBERTa: 97.51%) and IMDb (BERT: 95.62%). However, performance degrades substantially under domain shift: the Twitter 3-class pretrained baseline model achieves only 64.87%, and fine-grained 5-class baselines plateau between 54.87-−55.81%, confirming that pretrained baseline inference is inadequate for distribution-shifted or high-granularity tasks. Full Fine-Tuning (FFT) reliably closes this gap, serving as an effective but expensive upper bound. Training times range from 210 s (DistilBERT on SST-2) to 1,853 s (BERT on IMDb), with peak GPU memory between 2.0-−4.2 GB. LoRA emerges as a practical parameter-efficient strategy across the evaluated conditions, substantially reducing the number of trainable parameters by 98.87-−99.82% relative to full fine-tuning. LoRA also reduces training time in most conditions, with the largest reduction reaching 50.8%, although it is slightly slower than FFT in two of the twelve evaluated conditions. It should be noted that all efficiency comparisons reported here reflect training-time metrics only. Inference-time latency and memory depend on deployment configuration, such as whether LoRA adapters are merged into the base model prior to serving. Across all tasks, LoRA’s Confidence and Similarity Scores are competitive with or superior to FFT, indicating that the quality of probability distributions, not merely hard-decision accuracy, is maintained under parameter-efficient adaptation. A plausible intuitive explanation is that LoRA’s fixed low-rank constraint forces the model to capture only the most essential directional changes in the weight space, rather than memorising task-specific noise, which may explain its relative advantage over full fine-tuning in domain-shifted and high-granularity sentiment tasks. Full fine-tuning has the freedom to update every parameter, which is beneficial when training data is large and clean, but becomes a liability when the target domain is noisy, or the label boundaries are ambiguous, as in social media text or fine-grained star ratings. By restricting updates to a low-dimensional subspace, LoRA acts as an implicit regularizer that retains the broad linguistic knowledge encoded during pretraining while adapting only the most task-relevant directions. This explains why LoRA’s advantage is most pronounced on Twitter and Yelp 5-class, where noise and label ambiguity are highest, and smallest on IMDb, where long-form coherent text rewards the additional capacity that full fine-tuning provides. 5.2 Effect of Label Granularity As noted in the results for Figs. 13 and 16, collapsing the 5-star rating scale to three classes (Negative/Neutral/Positive) is associated with an accuracy improvement of 16.71-−26.98 percentage points across the E-Commerce and Yelp datasets. The 5-class settings use BERT/mBERT while the 3-class settings use RoBERTa, so this gap reflects a combination of reduced label ambiguity and the change in backbone architecture, and should not be attributed to label granularity alone. A plausible contributing factor is the inherent ambiguity of adjacent ordinal categories: distinguishing a 3-star from a 4-star review requires modeling subtle differences in enthusiasm and emphasis that lie largely outside the vocabulary-level signal available to contextualized encoders. Confirming this mechanism, and isolating it from the backbone confound, would require a fixed-backbone ablation across both granularity settings, which we identify as an important direction for future work. 5.3 Domain Robustness in Aspect-Based Analysis As shown in Tables 5 and Figs. 10, 11, the domain-specific ABSA results reveal a meaningful asymmetry between the Laptop and Restaurant subsets. The Laptop domain yields a higher pretrained baseline (81.86%), consistent with the structured technical vocabulary of product reviews, which overlaps substantially with DeBERTa’s pretraining corpus. In contrast, the Restaurant domain’s experiential, conversational language creates a greater distributional shift, leading to a lower pretrained baseline (73.09%) and a larger gain from FFT (+8.02 points). LoRA maintains competitive performance in both domains while reducing training time by 45–48% across combined and domain-specific evaluations, making it the practical default for multi-domain ABSA deployments that require frequent model updates. 5.4 Ablation Study To isolate the contribution of individual design factors, we conduct a systematic ablation study over three dimensions: adaptation strategy, label granularity, and model capacity. All ablations are drawn directly from the experimental results reported in Sect. 4 under the same training protocol, ensuring controlled comparison. 5.4.1 Ablation I: Adaptation Strategy and Parameter Budget Table 8 summarizes accuracy, trainable parameter count, and training time across all tasks and adaptation conditions. The parameter reduction column (\(\Delta P\)) expresses LoRA parameters as a percentage of FFT parameters, and the accuracy delta (\(\Delta \)Acc.) reports the signed accuracy difference of LoRA relative to FFT. LoRA outperforms FFT in accuracy on two of the twelve evaluated conditions: SST-2 with RoBERTa (\(+2.37\) percentage points) and E-Commerce in the 3-class setting (\(+0.34\) percentage points). In most other conditions, FFT achieves higher accuracy, with the largest performance gaps observed on Twitter (\(-20.80\) percentage points) and the combined ABSA dataset (\(-7.30\) percentage points). AdaLoRA achieves the highest accuracy on Yelp 5-class (\(+3.10\) percentage points over FFT), while remaining competitive with FFT on E-Commerce 5-class and 3-class settings. Despite these performance differences, both LoRA and AdaLoRA substantially reduce the number of trainable parameters relative to FFT, with LoRA requiring less than 1.2% of the FFT parameter budget across all evaluated conditions. 5.4.2 Ablation II: Effect of Label Granularity Table 9 isolates the effect of label granularity by comparing 5-class and 3-class performance on the same datasets and models under FFT and LoRA conditions. The granularity gain is substantial across both datasets and adaptation strategies. Yelp shows a larger improvement from 5-class to 3-class classification than E-Commerce, with LoRA gaining 26.98 percentage points and AdaLoRA gaining 18.85 percentage points. E-Commerce also shows consistent gains of 17.33 and 16.71 points for LoRA and AdaLoRA, respectively. These results indicate that reducing label granularity can substantially improve classification performance, although the observed gains also reflect differences in model architecture between the 5-class and 3-class settings. 5.4.3 Ablation III: Effect of Model Capacity on LoRA Efficiency Table 10 compares the LoRA parameter efficiency ratio and resulting accuracy across model architectures spanning 66 M to 124 M parameters. LoRA updates only 0.18%-−1.13% of the total model parameters across the evaluated architectures, substantially reducing the trainable parameter budget compared with FFT. LoRA matches or exceeds FFT accuracy in two of the twelve evaluated settings, while remaining competitive in several others despite its considerably smaller parameter footprint. The highest parameter efficiency is observed with DeBERTa-v3, where LoRA uses only 0.18% of the total parameters for ABSA. However, the results also show that parameter efficiency does not consistently translate into comparable accuracy, particularly for domain-shifted and fine-grained tasks. Overall, the findings demonstrate that LoRA provides substantial parameter savings across model architectures, although its effectiveness depends on the task and dataset characteristics. 5.5 Comparison with State-of-the-Art Methods Table 11 compares the proposed SentiMatrix models, using LoRA and AdaLoRA adaptation, with representative state-of-the-art (SOTA) approaches reported in the literature across the four sentiment analysis paradigms considered in this study. The comparison includes accuracy, precision, recall, and macro-F1 where these metrics are available. However, the cited studies do not always use identical datasets, data splits, model architectures, or evaluation protocols; the results are interpreted with respect to comparability rather than treating all reported values as strictly equivalent benchmarks. Intent-Based Sentiment Analysis. For the SST-2 2-class task, SentiMatrix LoRA achieves 93.28% accuracy, 93.42% precision, 93.64% recall, and 93.51% macro-F1. Although below the reported RoBERTa and LoRA + Dataset Pruning (Hosseini et al., 2024) results, SentiMatrix achieves competitive performance without dataset pruning or task-specific architectural modifications. For the Twitter US Airline 3-class task, SentiMatrix achieves 72.84% accuracy and 72.10% macro-F1, below RoBERTa-LSTM (Tan et al., 2022) (91.77%), BERT-DCNN (Jain et al., 2023) (87.10%), and DJC\(^\S \) (He, 2024) (87.22%). These methods employ task-specific architectures, whereas SentiMatrix uses general-purpose LoRA adaptation. The RoBERTa-RT\(^\ddagger \) (Barbieri et al., 2020) and TweetEval\(^\ddagger \) (Barbieri et al., 2020) results use the SemEval-2017 Twitter Sentiment dataset and macro-averaged recall; therefore, they are included only for contextual comparison. Aspect-Based Sentiment Analysis. For ABSA, SentiMatrix LoRA achieves 80.74% accuracy and 79.84% macro-F1 on the combined Laptop+Restaurant setting, exceeding the AdaLoRA-DeBERTa baseline (76.18% accuracy, 71.00% macro-F1) evaluated under the same protocol. Although the accuracy is below LCF-BERT\(^\dagger \) (Zeng et al., 2019) (87.14%) and LSA+DeBERTa-v3\(^\dagger \) (Yang & Li, 2024) (90.33%), these external results are based on the Restaurant-only subset of SemEval-2014 and are therefore not directly comparable. LoRA nevertheless remains competitive with FFT, achieving 82.15% versus 84.22% accuracy on Laptop and 78.45% versus 81.11% on Restaurant, while updating substantially fewer parameters. Fine-Grained Sentiment Analysis. For the Yelp 5-class task, SentiMatrix LoRA achieves 50.87% accuracy and 51.53% macro-F1, while AdaLoRA achieves 59.35% and 59.98%, respectively. C-LSTM+GloVe 100d\(^\dagger \) (Rafay & Suleman, 2020) reports higher performance; however, it uses a different Yelp review dataset configuration and evaluation split. The result is therefore retained as a contextual reference rather than a directly comparable SOTA benchmark. For the E-Commerce 5-Class task, SentiMatrix LoRA achieves 67.63% accuracy and 47.37% macro-F1, while AdaLoRA achieves 68.16% and 49.21%, respectively. BERT\(^\dagger \) (Bellar et al., 2024) reports higher accuracy, precision, and recall; however, differences in dataset configuration, architecture, and reported F-score limit direct comparison. The granularity analysis further shows substantial performance gains when reducing five classes to three. LoRA accuracy increases from 67.63% to 84.96% on E-Commerce and from 50.87% to 77.85% on Yelp, highlighting the increased difficulty of fine-grained sentiment classification. Emotion Detection. For the 6-class emotion detection task, SentiMatrix LoRA-DistilBERT achieves 93.22% accuracy and 90.20% macro-F1, compared with 90.08% and 86.54% for AdaLoRA. The Enriched-Pattern CNN\(^\dagger \) (Saravia et al., 2018) reports 81.00% accuracy on the same underlying Emotion dataset but employs a different model architecture and learning strategy. Thus, this comparison provides contextual evidence rather than isolating the contribution of LoRA alone. LoRA nevertheless provides stronger performance than AdaLoRA while requiring substantially fewer trainable parameters than full fine-tuning. Overall Comparison. Across the evaluated tasks, SentiMatrix demonstrates that parameter-efficient adaptation can achieve competitive sentiment classification performance with substantially fewer trainable parameters than full fine-tuning. However, performance varies across tasks and datasets, and comparisons with external methods must account for differences in dataset configurations, label settings, architectures, and evaluation metrics. Accordingly, the SOTA comparisons are interpreted primarily as contextual positioning of SentiMatrix rather than evidence of universal superiority. Direct numerical comparisons are strongest when the dataset and evaluation protocol are consistent, while results involving different dataset configurations or reported metrics are retained for broader contextual reference. 5.6 Error Analysis We analyze representative misclassification cases from Twitter and review-based datasets to identify common sources of model errors. In Twitter data, errors frequently arise from sarcasm, irony, implicit sentiment, and informal expressions, where the literal wording may differ from the intended sentiment. For example, statements such as “Great, another delay” may be misclassified because of their positive surface polarity despite conveying negative sentiment. In review datasets, errors often occur in mixed-sentiment sentences containing both positive and negative aspects, such as “The service was excellent, but the delivery was very slow,” making the overall sentiment difficult to determine. These cases highlight the challenges of capturing contextual, implicit, and aspect-dependent sentiment across different datasets and adaptation strategies. 5.7 Limitations Despite the comprehensive evaluation across multiple sentiment analysis paradigms and datasets, several limitations should be acknowledged. First, the study is restricted to English-language sentiment analysis datasets; therefore, the findings may not directly generalize to multilingual or low-resource settings. Second, the evaluation focuses on encoder-based Transformer architectures, including BERT, RoBERTa, DistilBERT, and DeBERTa, and does not consider decoder-only or large generative language models. Consequently, the observed adaptation behavior may not extend directly to large-scale LLMs. Third, although multiple sentiment analysis paradigms are considered, the study does not evaluate broader NLP tasks such as question answering, summarization, or text generation. Fourth, the LoRA and AdaLoRA configurations use closely matched, but not strictly identical, trainable parameter budgets. The difference arises from the additional parameters required by AdaLoRA for rank-importance estimation and dynamic allocation. Thus, the comparison does not constitute a strictly capacity-matched evaluation. Fifth, a comprehensive rank-sensitivity analysis and a strictly controlled training-time protocol are not included in the current study. Therefore, the observed performance and efficiency patterns should not be interpreted as definitive evidence of an underlying regularization mechanism. Rather, they provide empirical evidence consistent with this hypothesis. Confirming such a mechanism would require additional controlled experiments across multiple adapter ranks, strictly matched parameter budgets, and standardized timing conditions that minimize hardware and training-procedure variability. Sixth, while LoRA and AdaLoRA share identical target modules and matched rank for the majority of datasets, scaling factor and dropout differ for several configurations, and learning rate, batch size, and epoch count were not independently verified as identical across the two methods for every dataset. The comparison should therefore be read as closely aligned rather than a fully hyperparameter-matched, capacity-controlled evaluation. Finally, the 80%/10%/10% splits used for the fine-grained datasets were not explicitly stratified by rating class, which may affect the representation of minority rating categories across the validation and test partitions. This limitation may partially influence comparisons involving label granularity. Future work should employ stratified partitioning and imbalance-aware training objectives, while maintaining the same backbone architecture across granularity settings to provide a more controlled assessment. Further investigation using multilingual datasets, generative language models, broader NLP tasks, and systematic rank and capacity controls would provide additional evidence regarding the generalizability and underlying mechanisms of parameter-efficient fine-tuning. 6 Conclusion This paper presented SentiMatrix, a unified evaluation framework for comparing pretrained baseline inference, Full Fine-Tuning (FFT), LoRA-based parameter-efficient fine-tuning (PEFT), and AdaLoRA-based PEFT across four sentiment analysis paradigms and multiple benchmark datasets. The results demonstrate that LoRA can achieve competitive performance with substantially lower computational and parameter costs than FFT. Across the evaluated settings, LoRA reduces the number of trainable parameters by more than 97% while providing substantial reductions in training-time resource requirements, with the exact efficiency gains varying across architectures and datasets. These efficiency measurements reflect training-time characteristics; inference-time efficiency depends on deployment configuration and is not directly evaluated in this study. Results averaged across multiple runs further demonstrate that the observed performance trends remain consistent across different experimental conditions. The ablation analyses provide several key observations. First, LoRA generally maintains competitive performance despite updating only a small fraction of the underlying model parameters, although its effectiveness varies with dataset characteristics, model architecture, and task difficulty. Second, FFT retains a clear advantage in several domain-shifted and challenging settings, indicating that parameter-efficient adaptation does not uniformly outperform full fine-tuning. Third, the comparison between five-class and collapsed three-class sentiment settings shows substantial accuracy improvements after label simplification. However, different backbone architectures are used across these settings; these gains should be interpreted as evidence of the combined effect of label simplification and model differences rather than as an isolated effect of label granularity. Finally, the observed performance patterns are consistent with the possibility that low-rank adaptation provides a form of implicit regularization in some settings; however, establishing this mechanism conclusively requires further controlled experiments. The findings are limited to the encoder-based Transformer architectures and sentiment analysis benchmarks evaluated in this study and should not be directly generalized to decoder-only or generative large language models. In particular, differences in architecture, optimization objectives, and adaptation dynamics may lead to different PEFT behavior in large generative models. Future work will extend SentiMatrix to generative large language models, including LLaMA and Mistral (Šmíd et al., 2024), and investigate PEFT strategies such as LoRA, AdaLoRA, and QLoRA across the same sentiment analysis paradigms. Further studies will examine multilingual and low-resource settings to evaluate whether the observed efficiency–performance trade-offs remain consistent under cross-lingual and domain shifts. Additional controlled experiments involving systematic rank-sensitivity analysis, strictly matched parameter budgets, and standardized training-time protocols will also be important for determining the factors underlying the observed adaptation behavior. Finally, integrating psychology-informed features with parameter-efficient adaptation on emotion detection benchmarks such as CARER (Tan et al., 2025) represents another promising direction for extending the framework. Data Availability No datasets were generated or analysed during the current study. Code Availability Source code is publicly available at: https://github.com/ELTE-DSED/senti-matrix. References Alamoudi, E. S., & Alghamdi, N. S. (2021). Sentiment classification and aspect-based sentiment analysis on yelp reviews using deep learning and word embeddings. Journal of Decision Systems, 30(2–3), 259–281. Almufareh, M. F., Jhanjhi, N., Khan, N. A., Almuayqil, S. N., Humayun, M., & Javed, D. (2024). Bertsent: Transformer-based model for sentiment analysis of penta-class tweet classification. IEEE Access, 12, 196803–196817. Barbieri, F., Camacho-Collados, J., Espinosa-Anrés, L., Neves, M., & Navigli, R. (2020). Tweeteval: Unified benchmark and comparative evaluation for tweet classification. Findings of the Association for Computational Linguistics: EMNLP 2020 (pp. 1644–1650) Bellar, O., Baina, A., & Ballafkih, M. (2024). Sentiment analysis: Predicting product reviews for E-commerce recommendations using deep learning and transformers. Mathematics, 12(15), 2403. https://doi.org/10.3390/math12152403 Delbrouck, J.-B., Tits, N., Brousmiche, M., & Dupont, S. (2020) A transformer-based joint-encoding for emotion recognition and sentiment analysis. In Second Grand-Challenge and Workshop on Multimodal Language (Challenge-HML), pp. 1–7 Demszky, D., Movshovitz-Attias, D., Ko, J., Cowen, A., Nemade, G., R., & S. (2020). Goemotions: A dataset of fine-grained emotions. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (pp. 4040–4054) Dettmers, T., Pagnoni, A., Holtzman, A., & Zettlemoyer, L. (2023). Qlora: Efficient finetuning of quantized llms. Advances in Neural Information Processing Systems, 36, 10088–10115. Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (long and Short Papers), pp. 4171–4186 Ding, N., Qin, Y., Yang, G., Wei, F., Yang, Z., Su, Y., Hu, S., Chen, Y., Chan, C.-M., Chen, W., et al. (2023). Delta tuning: A comprehensive study of parameter efficient methods for pre-trained language models. AI Open, 4, 1–20. Dordević, N., & Stojković, S. (2024). Traditional and parameter-efficient fine-tuning of llms for sentiment analysis in the english and serbian language. 2024 11th International Conference on Electrical, Electronic and Computing Engineering (IcETRAN) (pp. 1–4). IEEE. Duru, I., & Sunar, A. S. (2025). Transformer and pre-transformer model-based sentiment prediction with various embeddings: A case study on amazon reviews. Entropy, 27(12), 1202. https://doi.org/10.3390/e27121202 Ekman, P. (1992). An argument for basic emotions. Cognition & emotion, 6(3–4), 169–200. He, J., Zhou, C., Ma, X., Berg-Kirkpatrick, T., & Neubig, G. (2022) Towards a unified view of parameter-efficient transfer learning. In Proceedings of the International Conference on Learning Representations. https://openreview.net/forum?id=0RDcd5Axok He, L. (2024). Enhanced twitter sentiment analysis with dual joint classifier integrating roberta and bert architectures. Frontiers in Physics, 12, 1477714. He, P., Gao, J., & Chen, W. (2023) DeBERTaV3: Improving DeBERTa using ELECTRA-style pre-training with gradient-disentangled embedding sharing. In: Proceedings of the Eleventh International Conference on Learning Representations. https://openreview.net/forum?id=sE7-XhLxHA He, P., Liu, X., Gao, J., C., & W. (2020). Deberta: Decoding-enhanced bert with disentangled attention arXiv:2006.03654 arXiv preprint. Hoang, M., Bihorac, O. A., & Rouces, J. (2019). Aspect-based sentiment analysis using bert. Proceedings of the 22nd Nordic Conference on Computational Linguistics (pp. 187–196) Hosseini, S. M. T., Mozafari, S. H., Clark, J., Meyer, B., & Gross, W. (2024). Parameter efficient fine-tuning of transformer-based language models using dataset pruning. 2024 58th Asilomar Conference on Signals, Systems, and Computers (pp. 1248–1252). https://doi.org/10.1109/IEEECONF60004.2024.10942686 Houlsby, N., Giurgiu, A., Jastrzebski, S., Morrone, B., De Laroussilhe, Q., Gesmundo, A., Attariyan, M., & Gelly, S. (2019). Parameter-efficient transfer learning for nlp. International Conference on Machine Learning (pp. 2790–2799). PMLR. Huang, H., Zavareh, A. A., & Mustafa, M. B. (2023). Sentiment analysis in e-commerce platforms: A review of current techniques and future directions. Ieee Access, 11, 90367–90382. Hu, E.J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., & Chen, W. (2022) Lora: Low-rank adaptation of large language models. In Proceedings of the International Conference on Learning Representations, vol. 1. https://openreview.net/forum?id=nZeVKeeFYf9 Hu, R., Rui, L., Zeng, P., Chen, L., F., & X. (2018). Text sentiment analysis: A review. Ieee 4th International Conference on Computer and Communications (iccc) (pp. 2283–2288). IEEE. Jain, P. K., Quamer, W., Saravanan, V., & Pamula, R. (2023). Employing BERT-DCNN with sentic knowledge base for social media sentiment analysis. Journal of Ambient Intelligence and Humanized Computing, 14, 10417–10429. https://doi.org/10.1007/s12652-022-03698-z Jiang, Q., Chen, L., Xu, R., Ao, X., & Yang, M. (2019) A challenge dataset and effective models for aspect-based sentiment analysis. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pp. 6280–6285 Jia, W., Shan, J., Wang, J., & Mengyang, Z. (2026). Implicit aspect-based sentiment analysis: A systematic review. ACM Computing Surveys, 58(10), 1–37. Kiritchenko, S., Zhu, X., & Mohammad, S. M. (2014). Sentiment analysis of short informal texts. Journal of Artificial Intelligence Research, 50, 723–762. Li, C., Li, L., & Qi, J. (2018) A self-attentive model with gate mechanism for spoken language understanding. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp. 3824–3833 Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., & Stoyanov, V. (2019). Roberta: A robustly optimized bert pretraining approach arXiv:1907.11692 arXiv preprint. Maas, A., Daly, R. E., Pham, P. T., Huang, D., Ng, A. Y., & Potts, C. (2011). Learning word vectors for sentiment analysis. Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies (pp. 142–150) Mohammad, S., Bravo-Marquez, F., Salameh, M., & Kiritchenko, S. (2018) Semeval-2018 task 1: Affect in tweets. In: Proceedings of the 12th International Workshop on Semantic Evaluation, pp. 1–17 Naithani, K., & Raiwani, Y. P. (2023). Realization of natural language processing and machine learning approaches for text-based sentiment analysis. Expert Systems, 40(5), 13114. Naseem, U., Razzak, I., Musial, K., & Imran, M. (2020). Transformer based deep intelligent contextual embedding for twitter sentiment analysis. Future Generation Computer Systems, 113, 58–69. https://doi.org/10.1016/j.future.2020.06.050 Pang, B., & Lee, L. (2008). Opinion mining and sentiment analysis. Foundations and Trends in Information Retrieval, 2(1–2), 1–135. https://doi.org/10.1561/1500000011 Phan, M. H., & Ogunbona, P. O. (2020). Modelling context and syntactical features for aspect-based sentiment analysis. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (pp. 3211–3220) Plutchik, R. (1980). A general psychoevolutionary theory of emotion. In R. Plutchik & H. Kellerman (Eds.), Theories of Emotion (pp. 3–33). New York, NY, USA: Academic Press. Pontiki, M., Galanis, D., Papageorgiou, H., Androutsopoulos, I., Manandhar, S., Al-Smadi, M., Al-Ayyoub, M., Zhao, Y., Qin, B., & De Clercq, O. (2016). Semeval-2016 task 5: Aspect based sentiment analysis. Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016) (pp. 19–30) Pota, M., Ventura, M., Catelli, R., & Esposito, M. (2021). An effective BERT-based pipeline for twitter sentiment analysis: A case study in italian. Sensors, 21(1), 133. https://doi.org/10.3390/s21010133 Rafay, A., Suleman, M., A., & A. (2020). Robust review rating prediction model based on machine and deep learning: Yelp dataset. 2020 3rd International Conference on Computing, Mathematics and Engineering Technologies (iCoMET). Karachi, Pakistan: IEEE. Rane, A., & Kumar, A. (2018). Sentiment classification system of twitter data for us airline service analysis. IEEE 42nd Annual Computer Software and Applications Conference (COMPSAC) (Vol. 1, pp. 769–773). IEEE. Sanh, V., Debut, L., Chaumond, J., & Wolf, T. (2019). Distilbert, a distilled version of bert: Smaller, faster, cheaper and lighter arXiv:1910.01108 arXiv preprint. Saravia, E., Liu, H.-C.T., Huang, Y.-H., Wu, J., & Chen, Y.-S. (2018). Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing (pp. 3687–3697) Carer: Contextualized affect representations for emotion recognition. Šmíd, J., Přibáň, P., & Král, P. (2024). LLaMA-based models for aspect-based sentiment analysis. Proceedings of the 14th Workshop on Computational Approaches to Subjectivity, Sentiment, and Social Media Analysis (pp. 63–70). Bangkok, Thailand: Association for Computational Linguistics. https://doi.org/10.18653/v1/2024.wassa-1.6 Socher, R., Perelygin, A., Wu, J., Chuang, J., Manning, C. D., Ng, A. Y., & Potts, C. (2013). Recursive deep models for semantic compositionality over a sentiment treebank. Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing (pp. 1631–1642) Sun, C., Qiu, X., Xu, Y., & Huang, X. (2019). How to fine-tune bert for text classification? China National Conference on Chinese Computational Linguistics (pp. 194–206). Springer. Tan, J.J., Kwan, B.-H., Ng, D.W.-K., & Hum, Y.C. (2025) Psychology-informed natural language understanding: Integrating personality and emotion-aware features for comprehensive sentiment analysis and depression detection. Pertanika J. Sci. Technol 33(10.47836) Tan, K. L., Lee, C. P., Anbananthen, K. S. M., & Lim, K. M. (2022). Roberta-lstm: A hybrid model for sentiment analysis with transformer and recurrent neural network. IEEE Access, 10, 21517–21525. https://doi.org/10.1109/ACCESS.2022.3152828 Tur, G., & De Mori, R. (Eds.). (2011). Spoken Language Understanding: Systems for Extracting Semantic Information from Speech. Hoboken, NJ, USA: Wiley. Valipour, M., Rezagholizadeh, M., Kobyzev, I., & Ghodsi, A. (2023). Dylora: Parameter-efficient tuning of pre-trained models using dynamic search-free low-rank adaptation. Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics (pp. 3274–3287) Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, Ł, & Polosukhin, I. (2017). Attention is all you need. In Advances in Neural Information Processing Systems, 30, 5998–6008. Wang, A., Pruksachatkun, Y., Nangia, N., Singh, A., Michael, J., Hill, F., Levy, O., & Bowman, S. (2019). Superglue: A stickier benchmark for general-purpose language understanding systems. Advances in neural information processing systems (p. 32) Wang, H. (2026). Aspect-based sentiment analysis (ABSA) for academic linguistics articles. Discover Artificial Intelligence, 6(1), 105. https://doi.org/10.1007/s44163-025-00803-x Wang, L., Chen, S., Jiang, L., Pan, S., Cai, R., Yang, S., & Yang, F. (2025). Parameter-efficient fine-tuning in large language models: a survey of methodologies. Artificial Intelligence Review, 58(8), 227. Wang, Y., Huang, M., Zhu, X., & Zhao, L. (2016). Attention-based lstm for aspect-level sentiment classification. Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing (pp. 606–615) Wankhade, M., Rao, A. C. S., & Kulkarni, C. (2022). A survey on sentiment analysis methods, applications, and challenges. Artificial Intelligence Review, 55(7), 5731–5780. Yadollahi, A., Shahraki, A. G., & Zaiane, O. R. (2017). Current state of text sentiment analysis from opinion to emotion mining. ACM Computing Surveys (CSUR), 50(2), 1–33. Yang, H., & Li, K. (2024). Modeling aspect sentiment coherency via local sentiment aggregation. Findings of the Association for Computational Linguistics: EACL 2024 (pp. 182–195) Yang, Z., Dai, Z., Yang, Y., Carbonell, J., Salakhutdinov, R.R., & Le, Q.V. (2019) Xlnet: Generalized autoregressive pretraining for language understanding. Advances in neural information processing systems 32 Zeng, B., Yang, H., Xu, R., Zhou, W., & Han, X. (2019). Lcf: A local context focus mechanism for aspect-based sentiment classification. Applied Sciences, 9(16), 3389. Zhang, L., Wang, S., & Liu, B. (2018). Deep learning for sentiment analysis: A survey. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 8(4), 1253. https://doi.org/10.1002/widm.1253 Zhang, Q., Chen, M., Bukharin, A., He, P., Cheng, Y., Chen, W., & Zhao, T. (2023) Adaptive budget allocation for parameter-efficient fine-tuning. In International Conference on Learning Representations (ICLR) Zhang, W., Li, X., Deng, Y., Bing, L., & Lam, W. (2022). A survey on aspect-based sentiment analysis: Tasks, methods, and challenges. IEEE Transactions on Knowledge and Data Engineering, 35(11), 11019–11038. Zhang, X., Zhao, J., & LeCun, Y. (2015) Character-level convolutional networks for text classification. Advances in neural information processing systems 28 Zimbra, D., Abbasi, A., Zeng, D. D., & Chen, H. (2018). The state-of-the-art in twitter sentiment analysis: A review and benchmark evaluation. ACM Transactions on Management Information Systems, 9(2), 5. https://doi.org/10.1145/3185045 Acknowledgements The authors gratefully acknowledge the support of the Faculty of Excellence fellowship programme, Faculty of Informatics, Eötvös Loránd University, Budapest, Hungary. The authors also acknowledge the open-source communities behind the HuggingFace Transformers library, PyTorch, and the benchmark dataset providers whose resources made this research possible. Funding Open access funding provided by Eötvös Loránd University. The work was carried out as part of the Faculty of Excellence fellowship programme at Eötvös Loránd University (Work no. E19019/19). Author information Authors and Affiliations Contributions Md Easin Arafat: Conceptualization, Methodology, Validation, Writing - Original Draft. Muhammad Usman Akmal: Methodology, Software, Visualization, Writing - Original Draft. Ali S. Abosinnee: Validation, Formal Analysis, Writing - Review & Editing. Tamás Orosz: Writing - Review & Editing, Supervision. Corresponding author Ethics declarations Conflict of interest The authors declare that they have no conflict of interest. Ethical Approval and Consent to Participate This study does not involve human participants, animal subjects, or any clinical data. Therefore, ethics approval and consent to participate are not applicable. Additional information Editors: Lan Du, Benjamin C. M. Fung, Hady W. Lauw, Longbing Cao. Publisher's Note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. Supplementary Information Below is the link to the electronic supplementary material. Rights and permissions Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. About this article Cite this article Arafat, M.E., Akmal, M.U., Abosinnee, A.S. et al. SentiMatrix: Parameter-Efficient Fine-Tuning of Encoder-Based Transformers for Multidimensional Sentiment Analysis. Mach Learn 115, 229 (2026). https://doi.org/10.1007/s10994-026-07161-4 Received: Revised: Accepted: Published: Version of record: DOI: https://doi.org/10.1007/s10994-026-07161-4