A Hybrid and Low-Parameter Model Based on Transformer and GRU Architectures for Persian Sentiment Analysis

Document Type : Original Article

Authors

1 PhD student, Department of Computer Engineering, Islamic Azad University, Ferdows Branch, South Khorasan, Iran

2 Associate Professor, Department of Computer Engineering, Islamic Azad University, Ferdows Branch, South Khorasan, Iran

3 Assistant Professor, Department of Mathematics, Faculty of Science, University of Jiroft, Kerman, Iran

Abstract

Sentiment analysis, as a subset of text classification, is a vital tool in areas such as information management, data analysis, and performance improvement. The complexity of Persian texts, due to features such as sarcasm, polysemy, and language combinations, poses unique challenges for sentiment recognition. In this study, a new model is introduced that extracts local and global features of the text by combining GRU and ParsBERT. The main focus of this study was to reduce the computational complexity and training time while maintaining the desired level of accuracy, so that the model can be used in resource-constrained environments. This model improves the parameters and training time by reducing the encoder layers and adding GRU. Experiments on the Snapfood and Taqcheh datasets showed that this model has higher accuracy than the baseline models and its training is also faster. The proposed model achieved F1 score of 76.76 in the niche dataset and 16.86 in the Snapfood dataset, while the number of parameters was significantly reduced. These results indicate that the proposed model is efficient and optimal for sentiment analysis of Persian texts.

Keywords

Main Subjects

References

[1]     H. P. Luhn, "The automatic creation of literature abstracts," IBM Journal of Research and Development, vol. 2, no. 2, pp. 159–165,Apr. 1958.
[2]     Z. B. Nezhad and M. A. Deihimi, "A combined deep learning model for Persian sentiment analysis," IIUM Engineering Journal, vol. 20, no. 1, pp. 129–139, 2019.
[3]     S. A. H. Minoofam, A. Bastanfard, and M. R. Keyvanpour, "TrCLA: A transfer learning approach to reduce negative transfer for cellular learning automata," IEEE Transactions on Neural Networks and Learning Systems, 2021.
[4]     C. N. Dang, M. N. Moreno-García, and F. De la Prieta, "Hybrid deep learning models for sentiment analysis," Complexity, vol. 2021.
[5]     H. Sadr and M. Nazari Soleimandarabi, "ACNN-TL: Attention-based convolutional neural network coupling with transfer learning and contextualized word representation for enhancing the performance of sentiment classification," The Journal of Supercomputing, vol. 78, no. 7, pp. 10149–10175, 2022.
[6]     Lakizadeh and Z. Zinaty, "A novel hierarchical attention-based method for aspect-level sentiment classification," Journal of AI and Data Mining, vol. 9, no. 1, pp. 87–97, 2021.
[7]     S. M. Rezaeinia, R. Rahmani, A. Ghodsi, and H. Veisi, "Sentiment analysis based on improved pretrained word embeddings," Expert Systems with Applications, vol. 117, pp. 139–147, 2019.
[8]     Q. Zhang, Z. Zhang, M. Yang, and L. Zhu, "Exploring coevolution of emotional contagion and behavior for microblog sentiment analysis: A deep learning architecture," Complexity, vol. 2021, 2021.
[9]     M. Li, L. Chen, J. Zhao, and Q. Li, "Sentiment analysis of Chinese stock reviews based on BERT model," Applied Intelligence, vol. 51, no. 7, pp. 5016–5024, 2021.
[10] M. B. Dastgheib, S. Koleini, and F. Rasti, "The application of deep learning in Persian documents sentiment analysis," International Journal of Information Science and Management(IJISM),vol.18,no.1,pp.115,2020.
[11] Keramatfar, H. Amirkhani, and A. Jalaly Bidgoly, "Multi-thread hierarchical deep model for context-aware sentiment analysis," Journal of Information Science, p. 0165551521990617, 2021.
[12] M. Farahani, M. Gharachorloo, M. Farahani, and M. Manthouri, "ParsBERT: Transformer-based model for Persian language understanding," Neural Processing Letters, vol. 53, no. 6, pp. 3831–3847, 2021.
[13] S. S. Sadeghi, H. Khotanlou, and M. R. Mahand, "Automatic Persian text emotion detection using cognitive linguistic and deep learning," Journal of AI and Data Mining, vol. 9, no. 2, pp. 169–179, 2021.
[14] D. Xue et al., "Deep learning-based personality recognition from text posts of online social networks," Applied Intelligence, vol. 48, no. 11, pp. 4232–4246, 2018.
[15] D. Xu, Z. Tian, R. Lai, X. Kong, Z. Tan, and W. Shi, "Deep learning based emotion analysis of microblog texts," Information Fusion, vol. 64, pp. 1–11, 2020.
[16] V. Lorenzo, M. N. Franco, A. Pappalardo, and R. Prevete, "Dynamic hard pruning of neural networks at the edge of the internet," Journal of Network and Computer Applications, vol. 200, p. 103330, 2022.
[17] N. Park, X. Liu, G. Yang, and A. Simoulin, "Memory-efficient fine-tuning of transformers via token selection," in Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing (EMNLP), Association for Computational Linguistics, 2024.
[18] Y. Z. Vakili, A. Fallah, and S. Zakeri, "Enhancing sentiment analysis of Persian tweets: A transformer-based approach," in 2024 10th International Conference on Web Research(ICWR),Tehran,Iran,2024,pp.226230.
[19] L. Mathew and B. V. R., "Efficient classification techniques in sentiment analysis using transformers," in Advances in Electronics, Communication and Computing, Singapore: Springer, 2022, pp. 795–803.
[20] J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, "BERT: Pre-training of deep bidirectional transformers for language understanding," arXiv preprint arXiv:1810.04805, 2018.
[21] Karimi, S., & Jafarinejad, F. (2024). Aspect-Based Sentiment Analysis using the Attentional Encoder Network. Signal and Data Processing, 20(4), 121-128.
[22] Mir, M., & Noferesti, S. (2024). Using Data Augmentation Techniques for Sentiment Analysis of Users’ Opinions on Reopening of Schools During the Covid-19 Epidemic. Signal and Data Processing, 21(2), 3-14.
Applied and basic Machine intelligence research
Volume 4, Issue 1
May 2026
Pages 129-145

Files

Share

How to cite

Statistics