Domain adaptive English aspect word extraction method based on bidirectional long and short-term memory network and multi-head attention mechanism

Tianxiao  Wang

doi:10.6180/jase.202512_28(12).0013

Domain adaptive English aspect word extraction method based on bidirectional long and short-term memory network and multi-head attention mechanism

Tianxiao WangThis email address is being protected from spambots. You need JavaScript enabled to view it.

College of International Business, Shenyang Normal University, Shenyang 11034 China

Received: February 24, 2025
Accepted: March 23, 2025
Publication Date: April 23, 2025

Copyright The Author(s). This is an open access article distributed under the terms of the Creative Commons Attribution License (CC BY 4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are cited.

Download Citation: ||https://doi.org/10.6180/jase.202512_28(12).0013

English aspect word extraction is a core task of aspect level sentiment analysis. With the continuous development of social networks, more users tend to make decisions based on comment text, and pay more attention to the details of comment text. Therefore, it is of great significance to accurately extract aspect words from massive review texts for users to make quick decisions. Because the annotation corpus is extremely time-consuming, labor-intensive and costly, there are relatively few aspect word datasets in public currently, which affects the effective training of neural network models. To solve this problem, this paper proposes a novel domain adaptive English aspect word extraction method based on bidirectional long and short-term memory network and multi-head attention mechanism. Firstly, the dual labels of aspect emotion and aspect extraction are used for labeling. Secondly, by parallel aspect extraction and aspect emotion classification task channel, it uses the BERT, bidirectional long and short term memory networks (Bi-LSTM) and multi-head self-attention to extract deeper semantic information, near and far feature information. Finally, conditional random field (CRF) classifier and Softmax classifier are used for classification. In order to verify the effectiveness of the new method, experiments are carried out on Laptop, Restaurant and Device datasets respectively. Experimental results show that the proposed method performs better than multiple baseline models.

Keywords: English aspect word extraction; bidirectional long and short-term memory network; multi-head attention mechanism; BERT

[1] M.Tubishat, N. Idris, and M. A. Abushariah, (2018) “Implicit aspect extraction in sentiment analysis: Review, taxonomy, oppportunities, and open challenges" Information Processing & Management 54(4): 545–563. DOI: 10.1016/j.ipm.2018.03.008.
[2] M. Babaali, A. Fatemi, and M. A. Nematbakhsh, (2024) “Aspect extraction with enriching word represen tation and post-processing rules" Expert Systems with Applications 252: 124174. DOI: 10.1016/j.eswa.2024.124174.
[3] S. Poria, E. Cambria, and A. Gelbukh, (2016) “Aspect extraction for opinion mining with a deep convolutional neural network" Knowledge-Based Systems 108: 42–49. DOI: 10.1016/j.knosys.2016.06.009.
[4] Z. Chen, A. Mukherjee, B. Liu, M. Hsu, M. Castellanos, and R. Ghosh. “Exploiting domain knowledge in aspect extraction”. In: Proceedings of the 2013 Con ference on Empirical Methods in Natural Language Pro cessing. 2013, 1655–1667. DOI: 10.18653/v1/d13-1172.
[5] Y. Zhao, H. Li, S. Yin, and Y. Sun, (2018) “A New Chinese Word Segmentation Method Based on Maximum Matching." J. Inf. Hiding Multim. Signal Process. 9(6): 1528–1535.
[6] J. Z. Maitama, N. Idris, A. Abdi, L. Shuib, and R. Fauzi, (2020) “A systematic review on implicit and ex plicit aspect extraction in sentiment analysis" IEEE Ac cess 8: 194166–194191. DOI: 10.1109/ACCESS.2020. 3031217.
[7] T. Wang, Y. Cai, H.-f. Leung, R. Y. Lau, Q. Li, and H. Min, (2014) “Product aspect extraction supervised with online domain knowledge" Knowledge-Based Systems 71: 86–100. DOI: 10.1016/j.knosys.2014.05.018.
[8] Z. Jin, M. Tao, X. Wu, and H. Zhang, (2024) “Span based dependency-enhanced graph convolutional network for aspect sentiment triplet extraction" Neurocomputing 564: 126966. DOI: 10.1016/j.neucom.2023.126966.
[9] M. M. A. Busst, K. S. M. Anbananthen, S. Kannan, J. Krishnan, and S. Subbiah, (2024) “Ensemble BiL STM: A Novel Approach for Aspect Extraction From Online Text" IEEE Access 12: 3528–3539. DOI: 10.1109/ ACCESS.2023.3349203.
[10] Y. Liu, J. Shi, F. Huang, J. Hou, and C. Zhang, (2024) “Unveiling consumer preferences in automotive reviews through aspect-based opinion generation" Journal of Retailing and Consumer Services 77: 103605. DOI: 10.1016/j.jretconser.2023.103605.
[11] W. Zou, W. Zhang, W. Wu, and Z. Tian, (2024) “A multi-task shared cascade learning for aspect sentiment triplet extraction using bert-mrc" Cognitive Computation 16(4): 1554–1571. DOI: 10.1007/s12559-024 10247-7.
[12] Y. Shi, L. Li, H. Li, A. Li, and Y. Lin, (2024) “Aspect level sentiment analysis of customer reviews based on neural multi-task learning" Journal of Theory and Practice of Engineering Science 4(04): 1–8. DOI: 10.53469/ jtpes.2024.04(04).01.
[13] J. Yu, H. Li, S.-L. Yin, and S. Karim, (2020) “Dynamic gesture recognition based on deep learning in human-to computer interfaces" Journal of Applied Science and Engineering 23(1): 31–38. DOI: 10.6180/jase.202003_ 23(1).0004.
[14] H. Wei, A. Zhou, Y. Zhang, F. Chen, W. Qu, and M. Lu, (2022) “Biomedical event trigger extraction based on multi-layer residual BiLSTM and contextualized word representations" International Journal of Machine Learning and Cybernetics: 1–13. DOI: 10.1007/s13042-021-01315-7.
[15] M. Rhanoui, M. Mikram, S. Yousfi, and S. Barzali, (2019) “A CNN-BiLSTM model for document-level sentiment analysis" Machine Learning and Knowledge Extraction 1(3): 832–847. DOI: 10.3390/make1030048.
[16] A. Jisi, S. Yin, et al., (2021) “A new feature fusion network for student behavior recognition in education" Jour nal of Applied Science and Engineering 24(2): 133 140. DOI: 10.6180/jase.202104_24(2).0002.
[17] R. Pecoraro, V. Basile, and V. Bono, (2022) “Local multi-head channel self-attention for facial expression recognition" Information 13(9): 419. DOI: 10.3390/info13090419.
[18] S. Yin, H. Li, A. A. Laghari, T. R. Gadekallu, G. A. Sampedro, and A. Almadhor, (2024) “An Anomaly Detection Model Based on Deep Auto-Encoder and Capsule Graph Convolution via Sparrow Search Algorithm in 6G Internet of Everything" IEEE Internet of Things Journal 11(18): 29402–29411. DOI: 10.1109/JIOT.2024.3353337.
[19] S. Yin, H. Li, A. A. Laghari, L. Teng, T. R. Gadekallu, and A. Almadhor, (2024) “FLSN-MVO: Edge Computing and Privacy Protection Based on Federated Learning Siamese Network With Multi-Verse Optimization Algorithm for Industry 5.0" IEEE Open Journal of the Communications Society: DOI: 10.1109/OJCOMS.2024.3520562.
[20] X. Li, L. Bing, P. Li, and W. Lam. “A unified model for opinion target extraction and target sentiment prediction”. In: Proceedings of the AAAI conference on artificial intelligence. 33. 01. 2019, 6714–6721. DOI: 10. 1609/aaai.v33i01.33016714.
[21] S. Mensah, K. Sun, and N. Aletras, (2021) “An empirical study on leveraging position embeddings for target-oriented opinion words extraction" arXiv preprint arXiv:2109.01238: DOI: 10.48550/arXiv.2109.01238.
[22] K. M. Karao˘glan and O. Fındık, (2022) “Extended rule-based opinion target extraction with a novel text preprocessing method and ensemble learning" Applied Soft Computing 118: 108524. DOI: 10.1016/j.asoc.2022.108524.
[23] Q.Zhao,F. Yang, D. An, and J. Lian, (2024) “Modeling structured dependency tree with graph convolutional networks for aspect-level sentiment classification" Sensors 24(2): 418. DOI: 10.3390/s24020418.
[24] L. Xiao, X. Wu, J. Xu, W. Li, C. Jin, and L. He, (2024) “Atlantis: Aesthetic-oriented multiple granularities fusion network for joint multimodal aspect-based sentiment analysis" Information Fusion 106: 102304. DOI: 10.1016/j.inffus.2024.102304.
[25] L. Yang, J. Li, E. Ci, L. Zhang, Z. Li, and P. Wang, (2025) “Label Drop for Multi-Aspect Relation Modeling in Universal Information Extraction" arXiv preprint arXiv:2502.12614: DOI: 10.48550/arXiv.2502.12614.
[26] J. Ouyang, C. Xuan, B. Wang, and Z. Yang, (2024) “Aspect-based sentiment classification with aspect-specific hypergraph attention networks" Expert Systems with Applications 248: 123412. DOI: 10.1016/j.eswa.2024.123412.
[27] L. Wu, Y. Luo, B. Zhu, G. Liu, R. Wang, and Q. Yu. “Graph neural network framework for sentiment analysis using syntactic feature”. In: 2024 5th Interna tional Conference on Big Data & Artificial Intelligence & Software Engineering (ICBASE). IEEE. 2024, 531–535. DOI: 10.1109/ICBASE63199.2024.10762511.
[28] P. Zhu, B. Wang, K. Tang, H. Zhang, X. Cui, and Z. Wang, (2024) “A knowledge-guided graph attention network for emotion-cause pair extraction" Knowledge Based Systems 286: 111342. DOI: 10.1016/j.knosys.2023.111342.
[29] V.Ganganwar and R.Rajalakshmi, (2024) “Employing synthetic data for addressing the class imbalance in aspect based sentiment classification" Journal of Information and Telecommunication 8(2): 167–188. DOI: 10.1080/24751839.2023.2270824.