رده‌بندی تصور گفتار مبتنی بر سیگنال‌های مغزی با افزودن داده‌ها و شبکه تطبیق دامنه هماوردانه با استفاده از یادگیری عمیق در کاربرد رابط مغزـرایانه

نوع مقاله : مقاله پژوهشی

نویسندگان

1 دانشجوی دکتری، گروه مهندسی کامپیوتر، واحد علوم و تحقیقات، دانشگاه آزاد اسلامی، تهران، ایران

2 دانشیار، گروه مهندسی کامپیوتر و برق، دانشگاه خوارزمی، تهران، ایران

3 دانشیار، گروه علمی مهندسی کامپیوتر، واحد علوم و تحقیقات، دانشگاه آزاد اسلامی، تهران، ایران

10.22034/abmir.2025.22852.1105

چکیده

رابط مغز -رایانه مبتنی بر گفتار راهبُردهای ارتباط صوتی مؤثری را برای کنترل دستگاه‌ها از طریق دستورات گفتاری که از سیگنال‌های مغزی تفسیر می‌شوند، ارائه می‌کنند. یکی از چالش‌های مهم در مسئله رابط مغز-رایانه رده‌بندی سیگنال‌های مغزی مبتنی بر الکتروانسفالوگرافی است. الکتروانسفالوگرافی یک سیگنال مغزی غیرتهاجمی است که از سطح پوست سر از طریق الکترودها ضبط می‌شود. سیگنال‌های به‌دست‌آمده با استفاده از تجهیزات آسان و ارزان دارای وضوح مکانی نسبتاً پایین و وضوح زمانی بالا هستند که برای دستیابی به نتایج بهینه مناسب‌ترین روش استخراج و رده‌بندی ویژگی‌ها باید استفاده شود. همچنین جمع‌آوری داده‌های کافی برای آزمودنی جدید زمان و تلاش زیادی می‌طلبد که در این مقاله افزودن داده‌ها با استفاده از مدل مولد هماوردانه به‌منظور بهبود عملکرد رده‌بندی سیگنال‌های مغزی پیشنهادشده است. همچنین مدلی برای رده‌بندی تصور گفتار مبتنی بر سیگنال‌های مغزی آزمودنی جدید با یادگیری انتقالی با استفاده از روش مولد هماوردانه مبتنی بر تطبیق دامنه متمایزکننده هماوردانه ارائه‌شده است. به‌منظور شناسایی تصور گفتار از پایگاه داده KaraOne استفاده‌شده است. روش پیشنهادی با سایر روش‌های جدید بر اساس معیارهای صحت و کاپا مورد ارزیابی قرار گرفتند. طبق نتایج به‌دست‌آمده روش پیشنهادی با صحت ۸۶ ٪ و 21/60 ٪ به ترتیب تصور کلمات و واج‌ها را رده‌بندی می‌کند. مدل پیشنهادی، مستقل از سیگنال‌های مغزی هر فرد است که با آموزش مدل بر روی سیگنال‌های مغزی افزوده‌شده آزمودنی‌ها در دامنه منبع می‌توان به‌طور مؤثر سیگنال‌ها را در دامنه هدف بدون نیاز به داده‌های برچسب‌گذاری شده از آزمودنی جدید رده‌بندی کرد.

کلیدواژه‌ها

موضوعات

عنوان مقاله [English]

Speech Imagery Classification based on Brain Signals with Data Augmentation and Adversarial Domain Adaptation Network using Deep Learning in Brain-Computer Interface Application

نویسندگان [English]

  • Marzieh Anjerani 1
  • Mir Mohsen Pedram 2
  • Mitra Mirzarezaee 3
1 Ph.D. Student, Department of Computer Engineering, SR.C., Islamic Azad University, Tehran, Iran
2 Associate Professor, Department of Electrical and Computer Engineering, Faculty of Engineering, Kharazmi University, Tehran, Iran
3 Associate Professor, Department of Computer Engineering, SR.C., Islamic Azad University, Tehran, Iran
چکیده [English]

Speech-based brain-computer interfaces provide effective voice communication strategies for controlling devices through spoken commands interpreted from brain signals. One of the major challenges in the brain-computer interface problem is the classification of brain signals based on electroencephalography. Electroencephalography is a non-invasive brain signal that is recorded from the scalp surface through electrodes. The signals obtained using easy and cheap equipment have relatively low spatial resolution and high temporal resolution, which requires the most appropriate feature extraction and classification method to achieve optimal results. Also, Collecting sufficient data for a new subject requires a lot of time and effort, so in this paper, data augmentation using a generative adversarial model is proposed to improve the performance of brain signal classification. Also, a model for classifying speech imagery based on brain signals of a new subject with transfer learning using a generative adversarial method based on adversarial domain adaptation method is presented. In order to identify speech imagery, the KaraOne database was used. The proposed method was evaluated with other new methods based on accuracy and kappa criteria. According to the results, the proposed method classifies word imagery and phonemes with 86% and 60.21% accuracy, respectively. The proposed model is independent of each individual's brain signals, which can be effectively classified in the target domain by training the model on the augmented brain signals of the subjects in the source domain without the need for labeled data from the new subject.

کلیدواژه‌ها [English]

  • Speech Imagery
  • Data Augmentation
  • Adversarial Discriminative Domain Adaptation
  • Deep Learning
  • Transfer Learning
  • Convolutional Neural Network
  • Brain-Computer Interface

مراجع

[1]     M. Bisla and R. Anand, "EEG Based Brain Computer Interface System for Decoding Covert Speech using Deep Neural Networks," in 2023 IEEE 12th International Conference on Communication Systems and Network Technologies (CSNT), Bhopal, India, 2023.
[2]     C. S. DaSalla, H. Kambara, M. Sato and Y. Koike, "Single-trial classification of vowel speech imagery using common spatial patterns," Neural Networks, vol. 22, no. 9, pp. 1334-1339, 2009.
[3]     K. Brigham and B. V. Kumar, "Imagined Speech Classification with EEG Signals for Silent Communication: A Preliminary Investigation into Synthetic Telepathy," in 2010 4th International Conference on Bioinformatics and Biomedical Engineering, Chengdu, China, 2010.
[4]     S. Deng, R. Srinivasan, T. Lappas and M. D’Zmura, "EEG classification of imagined syllable rhythm using Hilbert spectrum methods," Journal of neural engineering, vol. 7, no. 4, 2010.
[5]     X. Chi, J. B. Hagedorn, D. Schoonover and M. D'Zmura, "EEG-Based Discrimination of Imagined Speech Phonemes," International Journal of Bioelectromagnetism, vol. 13, no. 4, pp. 201-206, 2011.
[6]     L. Wang, X. Zhang,. X. Zhong and Y. Zhang, "Analysis and classification of speech imagery EEG for BCI," Biomedical Signal Processing and Control, vol. 8, no. 6, pp. 901-908, 2013.
[7]     M. Matsumoto and J. Hori, "Classification of silent speech using support vector machine and relevance vector machine," Applied Soft Computing, vol. 20, pp. 95-102, 2014.
[8]     S. Iqbal, Y. U. Khan and O. Farooq, "EEG based classification of imagined vowel sounds," in 2015 2nd International Conference on Computing for Sustainable Global Development (INDIACom), New Delhi, India, 2015.
[9]     C. H. Nguyen, G. K. Karavas and P. Artemiadis, "Inferring imagined speech using EEG signals: a new approach using Riemannian manifold features," Journal of Neural Engineering, vol. 15, 2017.
[10] Balaji, A. Haldar, K. Patil, T. S. Ruthvik, V. CA, M. Jartarkar and V. Baths, "EEG-based classification of bilingual unspoken speech using ANN," in 2017 39th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Jeju, Korea (South), 2017.
[11] R. Sereshkeh, R. Trott, A. Bricout and T. Chau, "EEG Classification of Covert Speech Using Regularized Neural Networks," IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 25, no. 12, pp. 2292 - 2300, 2017.
[12] R. Sereshkeh, R. Trott, A. Bricout and T. Chau, "Online EEG classification of covert speech for brain–computer interfacing," International journal of neural systems, vol. 27, no. 8, p. 1750033, 2017.
[13] P. Saha and S. Fels, "Hierarchical Deep Feature Learning For Decoding Imagined Speech From EEG," in Proceedings of the AAAI Conference on Artificial Intelligence, 2019.
[14] J. T. Panachakel, A. Ramakrishnan and T. Ananthapadmanabha, "A novel deep learning architecture for decoding imagined speech from eeg," arXiv preprint arXiv:2003.09374, 2020.
[15] S. Zhao and F. Rudzicz, "Classifying phonological categories in imagined and articulated speech," in 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), South Brisbane, QLD, Australia, 2015.
[16] P. Sun and J. Qin, "Neural Networks based EEG-Speech Models," arXiv preprint arXiv:1612.05369, 2016.
[17] Cooney, R. Folli and D. Coyle, "Mel Frequency Cepstral Coefficients Enhance Imagined Speech Decoding Accuracy from EEG," in 2018 29th Irish Signals and Systems Conference (ISSC), Belfast, UK, 2018.
[18] A.-L. Rusnac and O. Grigore, "Generalized Brain Computer Interface System for EEG Imaginary Speech Recognition," in 2020 24th International Conference on Circuits, Systems, Communications and Computers (CSCC), Chania, Greece, 2020.
[19] A.-L. Rusnac and O. Grigore, "Convolutional Neural Network applied in EEG imagined phoneme recognition system," in 2021 12th International Symposium on Advanced Topics in Electrical Engineering (ATEE), Bucharest, Romania, 2021.
[20] P. Saha, S. Fels and M. Abdul-Mageed, "Deep learning the EEG manifold for phonological categorization from active thoughts," in ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Brighton, UK, 2019.
[21] P. Saha,. M. Abdul-Mageed and S. Fels, "SPEAK YOUR MIND! Towards Imagined Speech Recognition With Hierarchical Deep Learning," arXiv:1904.05746, 2019.
[22] J. T. Panachakel, A. G. Ramakrishnan and T. V. Ananthapadmanabha, "Decoding imagined speech using wavelet features and deep neural networks," in 2019 IEEE 16th India Council International Conference (INDICON), Rajkot, India, 2019.
[23] A.-L. Rusnac and O. Grigore, "CNN Architectures and Feature Extraction Methods for EEG Imaginary Speech Recognition," Sensors, vol. 22, no. 13, p. 4679, 2022.
[24] A.-L. Rusnac and O. Grigore, "Imaginary Speech Recognition Using a Convolutional Network with Long-Short Memory," Applied Sciences, vol. 12, no. 22, p. 11873, 2022.
[25] Q. Heting and G. Nuo, "Research on the Classification Algorithm of Imaginary Speech EEG Signals Based on Twin Neural Network," in 2022 7th International Conference on Signal and Image Processing (ICSIP), Suzhou, China, 2022.
[26] V. S. Rathod, A. Tiwari and O. G. Kakde, "Wading corvus optimization based text generation using deep CNN and BiLSTM classifiers," Biomedical Signal Processing and Control, vol. 78, p. 103969, 2022.
[27] Alizadeh and H. Omranpour, "EM-CSP: An efficient multiclass common spatial pattern feature method for speech imagery EEG signals recognition," Biomedical Signal Processing and Control, vol. 84, p. 104933, 2023.
[28] K. Zhang, G. Xu, Z. Han, K. Ma, X. Zheng, L. Chen, N. Duan and S. Zhang, "Data Augmentation for Motor Imagery Signal Classification Based on a Hybrid Neural Network," Sensors, vol. 20, no. 16, p. 4485, 2020.
J. Xu, H. Zheng, J. Wang, D. Li and X. Fang, "Recognition of EEG Signal Motor Imagery Intention Based on Deep Multi-View Feature Learning," Sensors, vol. 20, no. 12, p. 3496, 2020
پژوهش های نظری و کاربردی هوش ماشینی
دوره 2، شماره 2
اسفند 1403
صفحه 163-184

فایل ها

هم رسانی

ارجاع به این مقاله

آمار