ترکیب داده‌های چندوجهی برای تشخیص افسردگی در توئیتر

نوع مقاله : مقاله پژوهشی

نویسندگان

گروه علوم کامپیوتر، دانشکده ریاضی، آمار و علوم کامپیوتر، دانشگاه تهران، تهران، ایران

چکیده

افسردگی یکی از شایع‌ترین بیماری‌های روانی دنیای امروز است که می توان آن را با استفاده از اطلاعات موجود در شبکه‌های اجتماعی به طور مؤثری شناسایی کرد. استفاده از شبکه‌های اجتماعی برای مقاصد مختلف در سال‌های گذشته افزایش داشته است، زیرا این شبکه‌ها بیانگر اطلاعات مهمی هم از افراد و هم از جامعه هستند. پژوهشگران تلاش کرده‌اند تا افسردگی را با استفاده از وجوه مختلف اطلاعات مثل عکس، متن و صوت شناسایی کنند؛ اما بیشتر پژوهش‌ها تمرکز بر این موضوع داشتند که فقط از یک نوع اطلاعات مثل متن یا عکس برای تشخیص استفاده کنند که به نتایج قابل‌توجهی دست یافته‌اند. در این پژوهش یک مدل هوش مصنوعی چندوجهی از نوع شبکه‌های عمیق معرفی می‌شود که اطلاعات متن و عکس را با هم تحلیل کرده و افسردگی را تشخیص می‌دهد. این پژوهش از کدگذار متنی Bert و ResNet برای استخراج ویژگی استفاده می‌کند. این مدل نسبت به مدل‌های مشابه با استفاده از مقدار بسیار کمتری از مجموعه‌داده‌ی اصلی، دقت را نزدیک به ۵ درصد ارتقا داده است و به ۸۷/۸۹ درصد رسانده است.

کلیدواژه‌ها

موضوعات

عنوان مقاله [English]

Multimodal Data Fusion for Depression Detection on Twitter

نویسندگان [English]

  • Abolfazl Nadi
  • Reza Rezaei
Computer Science Department, Faculty of Mathematics, Statistics and Computer Science, University of Tehran, Tehran, Iran
چکیده [English]

Depression is one of the most significant and prevalent mental health disorders in today’s world. Early detection of depression is critical, and this study aims to identify depression in individuals using information derived from social media. The use of social media for various purposes has grown in recent years, as these platforms provide valuable insights into both individuals and society. Social media can be effectively utilized to detect depression. Researchers have attempted to identify depression using various types of data, such as images, text, and audio. Most studies have focused on using only one type of data, such as text or images, for detection. While these methods have achieved notable results, they have limitations in accuracy that can be addressed by incorporating new methods and integrating multiple data modalities into the model. In this study, we propose a multimodal model that analyzes text and images together to detect depression. Compared to similar models, our approach achieves an approximate 5% improvement in accuracy, reaching 89.87%, while utilizing significantly less of the original dataset.

کلیدواژه‌ها [English]

  • Depression Detection
  • Multi-Modal Data Fusion
  • Sentiment Analysis
  • Social Networks

مراجع

[1] A. Jitimon, T. Suda, and A. Thara, "Big data analytics on social networks for real-time depression detection," *Journal of Big Data*, vol. 9, p. 69, 2022.
 
[2] G. Shelley, S. Archana, and R. Jayanthi, "Multimodal, multiview and multitasking depression detection framework endorsed with auxiliary sentiment polarity and emotion detection," Jan. 19, 2023.
 
[3] C. B. Krishna, T. T. Marijn, L.-L. Lorenzo, A. R. Lauren, and B. Johan, "Individuals with depression express more distorted thinking on social media," *Nature Human Behaviour*, vol. 5, pp. 458–466, 2021.
 
[4] S. C. Gullal, H. Sherzod, S. Abdul, M.-B. Eric, O. Christian, and R. E., "MM-Claims: A Dataset for Multimodal Claim Detection in Social Media," *arXiv preprint arXiv:2205.01989*, 2022.
 
[5] B. Marouane, K. Mohammed, and B.-H. Abderrahim, "A comprehensive survey on sentiment analysis: Approaches, challenges and trends," 2023.
 
[6] F. Cummins, D. Doyle, and G. McCarthy, "Towards Automatic Text-Based Estimation of Depression Through Symptom Prediction," 2022. [Online]. Available: https://example-link1.com
 
[7] A. Ibitoye, O. Bamidele, and K. Dahunsi, "Deep Learning for Depression Detection from Textual Data," 2021. [Online]. Available: https://example-link2.com
 
[8] D. L. Danxia, F. X. Lin, A. Farooq, S. Muhammad, and G. Jing, "Detecting and measuring depression on social media using a machine learning approach: Systematic review," *JMIR Mental Health*, vol. 9, p. e27244, 2022.
 
[9] Z. Li, A. Zhengyi, C. Wenchuan, Z. Jiawei, Z. Fang, and H. Bin, "MHA: A multimodal hierarchical attention model for depression detection in social media," *Health Information Science and Systems*, vol. 11, p. 6, 2023.
 
[10] M. A. H. Yazda, M. Saeid, B. Goonmeet, R. William, S. Amit, A. Hassan, T. Krishnaprasad, M. John, M. Annie, and H. Pascal, "Multimodal mental health analysis in social media," *PLoS One*, vol. 15, p. e0226248, 2020.
 
[11] A.-M. Bucur, A. Cosma, P. Rosso, and L. P. Dinu, "It's just a matter of time: Detecting depression with time-enriched multimodal transformers," in *European Conference on Information Retrieval*, Springer, Jan. 13, 2023, pp. 200–215.
 
[12] O. Adekanmbi, M. A. Adekanmbi, M. O. Adekanmbi, and O. O. Adekanmbi, "Algorithmic teenagers’ depression detection on social media and automated instant engagement using therapy bot powered by multimodal deep learning and psychotherapy intervention," 2023.
 
[13] G. Shen, J. Jia, L. Nie, F. Feng, C. Zhang, T. Hu, T.-S. Chua, and W. Zhu, "Depression detection via harvesting social media: A multimodal dictionary learning solution," *IJCAI*, pp. 3838–3844, 2017.
 
[14] T. Gui, L. Zhu, Q. Zhang, M. Peng, X. Zhou, K. Ding, and Z. Chen, "Cooperative multimodal approach to depression detection in Twitter," in *Proceedings of the AAAI Conference on Artificial Intelligence*, vol. 33, no. 01, pp. 110–117, 2019.
 
[15] P. Xu, X. Zhu, and D. A. Clifton, "Multimodal learning with transformers: A survey," *IEEE Transactions on Pattern Analysis and Machine Intelligence*, vol. 45, no. 10, pp. 12113–12132, 2023.
 
[16] A.-M. Bucur, A. Cosma, P. Rosso, and L. P. Dinu, "It's just a matter of time: Detecting depression with time-enriched multimodal transformers," in *European Conference on Information Retrieval*, Springer, pp. 200–215, 2023.
 
[17] P. Mann, A. Paes, and E. H. Matsushima, "See and read: detecting depression symptoms in higher education students using multimodal social media data," in *Proceedings of the International AAAI Conference on Web and Social Media*, vol. 14, pp. 440–451, 2020.
 
[18] J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, "BERT: Pre-training of deep bidirectional transformers for language understanding," in *Proceedings of NAACL-HLT*, pp. 4171–4186, 2019.
 
[19] K. He, X. Zhang, S. Ren, and J. Sun, "Deep residual learning for image recognition," in *Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)*, pp. 770–778, 2016.
 
[20] H. Liu, Y. You, and D. Darrow, "Deep Incubation: Training Large Models by Divide-and-Conquering," *arXiv preprint arXiv:2212.04129*, 2022.
 
[21] J. Wang, Z. Liu, and T. Zhang, "Dataset Pruning: Reducing Training Data by Examining Generalization Influence," *arXiv preprint arXiv:2205.09329*, 2022.
 
[22] T. F. Qu, Y. Zhang, and J. Liu, “Depression Detection Using Emotion Artificial Intelligence,” in 2019 IEEE 10th International Conference on Awareness Science and Technology (iCAST), Morioka, Japan, 2019, pp. 1-5.
 
[23] S. Orabi, P. Buddhitha, M. H. Orabi, and D. Inkpen, “Deep Learning for Depression Detection of Twitter Users,” in Proceedings of the Fifth Workshop on Computational Linguistics and Clinical Psychology: From Keyboard to Clinic, New Orleans, LA, USA, 2018, pp. 88-97.
 
[24] F. Ren and Y. Wu, “Predicting Depression Trends on Twitter Using Sentiment Analysis,” in 2019 IEEE 13th International Conference on Semantic Computing (ICSC), Newport Beach, CA, USA, 2019, pp. 230-233.
 
پژوهش های نظری و کاربردی هوش ماشینی
دوره 2، شماره 1
شهریور 1403
صفحه 120-130

فایل ها

هم رسانی

ارجاع به این مقاله

آمار