استفاده از مدل‌های یادگیری انتقالی برای بهبود تشخیص احساسات بصری در شبکه‌های اجتماعی

نوع مقاله : مقاله پژوهشی

نویسندگان

1 گروه هوش مصنوعی، دانشکده و پژوهشکده هوش مصنوعی و علوم شناختی، دانشگاه جامع امام حسین (ع)، تهران، ایران

2 پژوهشگر دانشگاه جامع امام حسین (ع)، تهران، ایران

چکیده

تجزیه و تحلیل احساسات افراد از محتوای رسانه‌های اجتماعی از طریق متن، گفتار و تصاویر، در انواع مختلفی از برنامه‌ها و کاربردها مورد نیاز است. اکثر مطالعات تحقیقاتی اخیر در زمینه تجزیه و تحلیل احساسات، بر داده‌های متنی تمرکز داشته‌اند. با این حال، کاربران رسانه‌های اجتماعی، عکس‌ها و فیلم‌های مشابه بیشتری نسبت به متن به اشتراک می‌گذارند. به‌عبارت دیگر، تصاویر بهترین روش برای انتقال احساسات به دیگران هستند. از این رو، تمرکز بر توسعه یک مدل تحلیل احساسات بر اساس تصاویر در رسانه‌های اجتماعی اهمیت دارد. در این مقاله، از مدل یادگیری انتقال DenseNet-121 برای تحلیل احساسات بر اساس تصاویر استفاده خواهیم کرد. برای پیاده‌سازی این روش، از تصاویر موجود در مجموعه داده Image Sentiment استفاده خواهیم نمود. این مجموعه داده شامل آدرس‌های اینترنتی تصاویر به‌همراه قطبیت‌های احساسی آن‌ها است. بر اساس نتایج به‌دست‌آمده، دقت مدل پیشنهادی در این مقاله برابر با 89 % است که در مقایسه با کارهای پیشین در زمینه تجزیه و تحلیل احساسات بصری، مدل پیشنهادی، بهبود ۵ تا ۱۰ درصدی را نشان‌می‌دهد.

کلیدواژه‌ها

موضوعات

عنوان مقاله [English]

Improving Visual Sentiment Analysis in Social Networks using Transfer Learning Models

نویسندگان [English]

  • Mohammad Roustaei 1
  • Meysam Mirzaee 2
1 Department of Artificial Intelligence, Faculty and Research Institute of Artificial Intelligence and Cognitive Sciences, Imam Hossein University, Tehran, Iran
2 Researcher at Imam Hossein University , Tehran, Iran
چکیده [English]

Analyzing individuals' emotions from the content of social media through text, speech, and images is necessary for various types of applications and purposes. Most recent research studies in the field of sentiment analysis have focused on textual data. However, social media users share more images and videos compared to text. In other words, images are the most effective way to convey emotions to others. Therefore, focusing on the development of a sentiment analysis model based on images in social media is important. In this article, we will use the DenseNet-121 transfer learning model to analyze emotions based on images. To implement this approach, we will utilize the images available in the Image Sentiment dataset. This dataset includes internet links to images along with their emotional polarities. Based on the obtained results, the accuracy of the proposed model in this article is 89%, which, compared to previous work in the field of visual sentiment analysis, shows a 5% to 10% improvement.

کلیدواژه‌ها [English]

  • Visual Sentiment Analysis
  • Transfer Learning
  • Deep Learning
  • Social Networks

مراجع

Karray, F.; Alemzadeh, M.; Abou Saleh, J.; Nours Arab, M. Human-Computer Interaction: Overview on State of the Art. Int. J. Smart Sens. Intell. Syst. 2008, 1, 137–159.
Auxier, B.; Anderson, M. Social media use in 2021. Pew Res. Cent. 2021.
Sivarajah, U.; Kamal, M.M.; Irani, Z.; Weerakkody, V. Critical analysis of Big Data challenges and analytical methods. J. Bus. Res. 2017, 70, 263–286.
Bansal, B.; Srivastava, S. On predicting elections with hybrid topic based sentiment analysis of tweets. Procedia Comput. Sci. 2018, 135, 346–353.
El Alaoui, I.; Gahi, Y.; Messoussi, R.; Chaabi, Y.; Todoskoff, A.; Kobi, A. A novel adaptable approach for sentiment analysis on big social data. J. Big Data 2018, 5, 12.
Drus, Z.; Khalid, H. Sentiment Analysis in Social Media and Its Application: Systematic Literature Review. Procedia Comput. Sci. 2019, 161, 707–714.
Zhao, H.; Liu, Z.; Yao, X.; Yang, Q. A machine learning-based sentiment analysis of online product reviews with a novel term weighting and feature selection approach. Inf. Processing Manag. 2021, 58, 102656.
Dashtipour, K.; Gogate, M.; Adeel, A.; Larijani, H.; Hussain, A. Sentiment Analysis of Persian Movie Reviews Using Deep Learning. Entropy 2021, 23, 596.
Farisi, A.A.; Sibaroni, Y.; Faraby, S.A. Sentiment analysis on hotel reviews using Multinomial Naïve Bayes classifier. J. Phys. Conf. Ser. 2019, 1192, 012024.
Melton, C.A.; Olusanya, O.A.; Ammar, N.; Shaban-Nejad, A. Public sentiment analysis and topic modeling regarding COVID-19 vaccines on the Reddit social media platform: A call to action for strengthening vaccine confidence. J. Infect. Public Health 2021, 14, S1876034121002288.
Mishra, N.; Jha, C.K. Classification of Opinion Mining Techniques. Int. J. Comput. Appl. 2012, 56, 1–6.
Kim, M.; Lee, S.M.; Choi, S.; Kim, S.Y. Impact of visual information on online consumer review behavior: Evidence from a hotel booking website. J. Retail. Consum. Serv. 2021, 60, 102494.
Xiao, Z.; Wang, L.; Du, J.Y. Improving the Performance of Sentiment Classification on Imbalanced DatasetsWith Transfer Learning. IEEE Access 2019, 7, 28281–28290.
Praveen Gujjar, J.; Prasanna Kumar, H.R.; Chiplunkar, N.N. Image Classification and Prediction using Transfer Learning in Colab Notebook. Glob. Transit. Proc. 2021, 2, S2666285X21000960.
Zhang, Q.; Yang, Q.; Zhang, X.; Bao, Q.; Su, J.; Liu, X.Waste image classification based on transfer learning and convolutional neural network. Waste Manag. 2021, 135, 150–157.
Dilshad, S.; Singh, N.; Atif, M.; Hanif, A.; Yaqub, N.; Farooq, W.A.; Ahmad, H.; Chu, Y.; Masood, M.T. Automated image classification of chest X-rays of COVID-19 using deep transfer learning. Results Phys. 2021, 28, 104529.
Siersdorfer, S.; Minack, E.; Deng, F.; Hare, J. Analyzing and predicting sentiment of images on the social web. In Proceedings of the International Conference on Multimedia MM ’10, Firenze, Italy, 25–29 October 2010; pp. 715–718.
Rao, T.; Xu, M.; Liu, H.; Wang, J.; Burnett, I. Multi-scale blocks based image emotion classification using multiple instance learning. In Proceedings of the 2016 IEEE International Conference on Image Processing (ICIP), Phoenix, AZ, USA, 25–28 September 2016; pp. 634–638.
Datta, R.; Joshi, D.; Li, J.; Wang, J.Z. Studying Aesthetics in Photographic Images Using a Computational Approach. In Computer Vision—ECCV 2006; Leonardis, A., Bischof, H., Pinz, A., Eds.; Springer: Berlin/Heidelberg, Germany, 2006; Volume 3953, pp. 288–301.
Marchesotti, L.; Perronnin, F.; Larlus, D.; Csurka, G. Assessing the aesthetic quality of photographs using generic image In Proceedings of the 2011 International Conference on Computer Vision, Barcelona, Spain, 6–13 November 2011; pp. 1784–1791.
Borth, D.; Chen, T.; Ji, R.; Chang, S.-F. SentiBank: Large-scale ontology and classifiers for detecting sentiment and emotions in visual content. In Proceedings of the 21st ACM International Conference on Multimedia—M ’13, Barcelona, Spain, 21–25 October 2013; pp. 459–460.
Yuan, J.; Mcdonough, S.; You, Q.; Luo, J. Sentribute: Image sentiment analysis from a mid-level perspective. In Proceedings of the Second InternationalWorkshop on Issues of Sentiment Discovery and Opinion Mining—WISDOM ’13, Chicago, IL, USA, 11 August 2013; pp. 1–8.
Zhao, Z.; Zhu, H.; Xue, Z.; Liu, Z.; Tian, J.; Chua, M.C.H.; Liu, M. An image-text consistency driven multimodal sentiment analysis approach for social media. Inf. Processing Manag. 2019, 56, 102097.
Fernandez, D.; Woodward, A.; Campos, V.; Giro-i-Nieto, X.; Jou, B.; Chang, S.-F. More cat than cute? Interpretable Prediction of Adjective-Noun Pairs. In Proceedings of the Workshop on Multimodal Understanding of Social, Affective and Subjective Attributes, Mountain View, CA, USA, 27 October 2017; pp. 61–69.
Yang, J.; She, D.; Sun, M.; Cheng, M.-M.; Rosin, P.L.; Wang, L. Visual Sentiment Prediction Based on Automatic Discovery of Affective Regions. IEEE Trans. Multimed. 2018, 20, 2513–2525.
Wang, J.; Fu, J.; Xu, Y.; Mei, T. Beyond Object Recognition: Visual Sentiment Analysis with Deep Coupled Adjective and Noun Neural Networks. In Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence (IJCAI-16), New York, NY, USA, 9–15 July 2016; pp. 3484–3490.
Song, K.; Yao, T.; Ling, Q.; Mei, T. Boosting image sentiment analysis with visual attention. Neurocomputing 2018, 312, 218–228.
Ortis, A.; Farinella, G.M.; Torrisi, G.; Battiato, S. Visual Sentiment Analysis Based on on Objective Text Description of Images. In Proceedings of the 2018 International Conference on Content-Based Multimedia Indexing (CBMI), La Rochelle, France, 4–6 September 2018; pp. 1–6.
Xu, J.; Huang, F.; Zhang, X.; Wang, S.; Li, C.; Li, Z.; He, Y. Sentiment analysis of social images via hierarchical deep fusion of content and links. Appl. Soft Comput. 2019, 80, 387–399.
Huang, F.; Zhang, X.; Zhao, Z.; Xu, J.; Li, Z. Image-text sentiment analysis via deep multimodal attentive fusion. Knowl. Based Syst. 2019, 167, 26–37.
Deng, J.; Dong, W.; Socher, R.; Ki, L.; Li, K.; Fei-Fei, K. ImageNet: A large-scale hierarchical image database. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Miami Beach, FL, USA, 20–25 June 2009.
Simonyan, K.; Zisserman, A. Very deep convolutional networks for large-scale image recognition. arXiv 2014, arXiv:1409.1556.
He, K.; Zhang, X.; Ren, S.; Sun, J. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, 27–30 June 2016; pp. 770–778.
Usha Kingsly Devi, K., & Gomathi, V. (2023). Deep Convolutional Neural Networks with Transfer Learning for Visual Sentiment Analysis. Neural Processing Letters, 55(4), 5087-5120.
Chandrasekaran, G., Antoanela, N., Andrei, G., Monica, C., & Hemanth, J. (2022). Visual sentiment analysis using deep learning models with social media data. Applied Sciences, 12(3), 1030.
An, J., Zainon, W. M. N. W., & Ding, B. (2023). Leveraging Vision-Language Pre-Trained Model and Contrastive Learning for Enhanced Multimodal Sentiment Analysis. Intelligent Automation & Soft Computing, 37(2).
پژوهش های نظری و کاربردی هوش ماشینی
دوره 1، شماره 2
آذر 1402
صفحه 105-117

فایل ها

هم رسانی

ارجاع به این مقاله

آمار