Improving the Cross-Domain Classification of Short Text Using the Deep Transfer Learning Framework

Document Type : Research Paper


1 MA., Department of Information Technology Management, Faculty of Social Sciences and Economics, Alzahra University, Tehran, Iran.

2 Associate Prof., Department of Management, Faculty of Social Sciences and Economics, Alzahra University, Tehran, Iran.



With the advent of user-generated text information on the Internet, text sentiment analysis plays an essential role in online business transactions. The expression of feelings and opinions depends on the domains, which have different distributions. In addition, each of these domains or so-called product groups has its vocabulary and peculiarities that make analysis difficult. Therefore, different methods and approaches have been developed in this area. However, most of the analysis involved a single-domain and few studies on cross-domain mood classification using deep neural networks have been performed. The aim of this study was therefore to examine the accuracy and transferability of deep learning frameworks for the cross-domain sentiment analysis of customer ratings for different product groups as well as the cross-domain sentiment classification in five categories “very positive”, “positive”, “neutral”, “negative” and “very negative”. Labels were extracted and weighted using the Long Short-Term Memory (LSTM) Recurrent Neural Network. In this study, the RNN LSTM network was used to implement a deep transfer learning framework because of its significant results in sentiment analysis. In addition, two different methods of text representation, BOW and CBOW were used. Based on the results, using deep learning models and transferring weights from the source domain to the target domain can be effective in cross-domain sentiment analysis.


Main Subjects

Abdullah, N. A., Feizollah, A., Sulaiman, A., & Anuar, N. B. (2019). Challenges and Recommended Solutions in Multi-Source and Multi-Domain Sentiment Analysis. IEEE Access, 7, 144957-144971.
Alami, N., Meknassi, M., & En-nahnahi, N. (2019). Enhancing unsupervised neural networks-based text summarization with word embedding and ensemble learning. Expert systems with applications, 123, 195-211.
Al-Moslmi, T., Omar, N., Abdullah, S., & Albared, M. (2017). Approaches to cross-domain sentiment analysis: A systematic literature review. IEEE access, 5, 16173-16192.
Babu, N. V., & Kanaga, E. G. M. (2022). Sentiment analysis in social media data for depression detection using artificial intelligence: a review. SN Computer Science, 3, 1-20.
Başarslan, M. S., & Kayaalp, F. (2023). Sentiment analysis with ensemble and machine learning methods in multi-domain datasets. Turkish Journal of Engineering7(2), 141-148.
Bengio, Y. (2012, June). Deep learning of representations for unsupervised and transfer learning. JMLR: Workshop and Conference Proceedings 27, 17–37.
Bhavitha, B. K., Rodrigues, A. P., & Chiplunkar, N. N. (2017, March). Comparative study of machine learning techniques in sentimental analysis. In 2017 International conference on inventive communication and computational technologies (ICICCT) (pp. 216-221). IEEE.
Blei, D. M. (2012). Probabilistic topic models. Communications of the ACM, 55(4), 77-84.
Chen, J., Feng, J., Sun, X., & Liu, Y. (2020). Co-Training Semi-Supervised Deep Learning for Sentiment Classification of MOOC Forum Posts. Symmetry, 12(1), 8-32.
Chen, T., Xu, R., He, Y., & Wang, X. (2017a). Improving sentiment analysis via sentence type classification using BiLSTM-CRF and CNN. Expert Systems with Applications, 72, 221-230.
Chen, L., Guan, Z., He, J., & Peng, J. (2017b). A survey on sentiment classification. J. Comput. Res. Dev54(6), 1150-1170.
Dahal, B., Kumar, S. A., & Li, Z. (2019). Topic modeling and sentiment analysis of global climate change tweets. Social Network Analysis and Mining, 9(24), 1-20.
Dang, N. C., Moreno-García, M. N., & De la Prieta, F. (2020). Sentiment analysis based on deep learning: A comparative study. Electronics9(3), 483-512.
Gulati, K., Kumar, S. S., Boddu, R. S. K., Sarvakar, K., Sharma, D. K., & Nomani, M. Z. M. (2022). Comparative analysis of machine learning-based classification models using sentiment classification of tweets related to COVID-19 pandemic. Materials Today: Proceedings, 51, 38-41
Haque, T. U., Saber, N. N., & Shah, F. M. (2018, May). Sentiment analysis on large scale Amazon product reviews. In 2018 IEEE International Conference on Innovative Research and Development (ICIRD) (pp. 1-6). IEEE.
Huq, M. R., Ahmad, A., & Rahman, A. (2017). Sentiment analysis on Twitter data using KNN and SVM. International Journal of Advanced Computer Science and Applications, 8(6),19-25.
Inzalkar, S., & Sharma, J. (2015). A survey on text mining-techniques and application. International Journal of Research in Science & Engineering, 24, 1-14.
Kingma, D. P., & Ba, J. (2014). Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980.
Li, Z., Wei, Y., Zhang, Y., & Yang, Q. (2018, April). Hierarchical attention transfer network for cross-domain sentiment classification. In Proceedings of the AAAI Conference on Artificial Intelligence (Vol. 32, No. 1).
Li, Z., Zhang, Y., Wei, Y., Wu, Y., & Yang, Q. (2017, August). End-to-End Adversarial Memory Network for Cross-domain Sentiment Classification. In Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence (IJCAI-17) (pp. 2237-2243).
Liddy, E. D. (2001). Natural language processing. In Encyclopedia of Library and Information Science, 2nd Ed. NY. Marcel Decker, Inc.
Lim, S., & Lee, D. (2017). Stable improved softmax using constant normalization. Electronics Letters, 53(23), 1504-1506.
Lin, C., & He, Y. (2009, November). Joint sentiment/topic model for sentiment analysis. In Proceedings of the 18th ACM conference on Information and knowledge management (pp. 375-384).
Lin, S., Xie, H., Yu, L. C., & Lai, K. R. (2017, December). SentiNLP at IJCNLP-2017 Task 4: Customer feedback analysis using a Bi-LSTM-CNN model. In Proceedings of the IJCNLP 2017, Shared Tasks (pp. 149-154).
Ioffe, S., & Szegedy, C. (2015). Batch normalization: Accelerating deep network training by reducing internal covariate shift. arXiv preprint arXiv:1502.03167.
Liu, Y., Jin, J., Ji, P., Harding, J. A., & Fung, R. Y. (2013). Identifying helpful online reviews: a product designer’s perspective. Computer-Aided Design, 45(2), 180-194.
McTear, M., Callejas, Z., & Griol, D. (2016). The dawn of the conversational interface. In The Conversational Interface (pp. 11-24). Springer, Cham.
Medhat, W., Hassan, A., & Korashy, H. (2014). Sentiment analysis algorithms and applications: A survey. Ain Shams engineering journal5(4), 1093-1113.
Monali, P., & Sandip, K. (2014). A concise survey on text data mining. International Journal of Advanced Research in Computer and Communication Engineering3(9), 8040-8043.
Pal, S., Ghosh, S., & Nag, A. (2018). Sentiment analysis in the light of LSTM recurrent neural networks. International Journal of Synthetic Emotions (IJSE), 9(1), 33-39.
Pandey, A. C., Rajpoot, D. S., & Saraswat, M. (2017). Twitter sentiment analysis using hybrid cuckoo search method. Information Processing & Management, 53(4), 764-779.
Peng, M., Zhang, Q., Jiang, Y. G., & Huang, X. J. (2018, July). Cross-domain sentiment classification with target domain specific information. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (pp. 2505-2513). Melbourne, Australia.
Piryani, R., Madhavi, D., & Singh, V. K. (2017). Analytical mapping of opinion mining and sentiment analysis research during 2000–2015. Information Processing & Management, 53(1), 122-150.
Qi, J., Zhang, Z., Jeon, S., & Zhou, Y. (2016). Mining customer requirements from online reviews: A product improvement perspective. Information & Management53(8), 951-963.
Salas-Zárate, M. D. P., Medina-Moreira, J., Lagos-Ortiz, K., Luna-Aveiga, H., Rodriguez-Garcia, M. A., & Valencia-Garcia, R. (2017). Sentiment analysis on tweets about diabetes: an aspect-level approach. Computational and mathematical methods in medicine2017.
Severyn, A., & Moschitti, A. (2015, August). Twitter sentiment analysis with deep convolutional neural networks. In Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval (pp. 959-962).
Sivic, J., & Zisserman, A. (2008). Efficient visual search of videos cast as text retrieval. IEEE transactions on pattern analysis and machine intelligence, 31(4), 591-606.
Snoek, J., Larochelle, H., & Adams, R. P. (2012). Practical bayesian optimization of machine learning algorithms. In Advances in neural information processing systems (pp. 2951-2959).
Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., & Salakhutdinov, R. (2014). Dropout: a simple way to prevent neural networks from overfitting. The journal of machine learning research15(1), 1929-1958.,
Sun, S., Luo, C., & Chen, J. (2017). A review of natural language processing techniques for opinion mining systems. Information fusion, 36, 10-25.
Sundström, J. (2018). Sentiment analysis of Swedish reviews and transfer learning using convolutional neural networks. UPTEC STS. Retrieved from
Tang, D., Qin, B., & Liu, T. (2015). Deep learning for sentiment analysis: successful approaches and future challenges. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 5(6), 292-303.
Vidya, S. (2018). Cross Domain Sentiment Classification Using Natural Language Processing.‏ International Journal of Scientific Research in Computer Science, Engineering and Information Technology. 3(3), 348-353.
Wei, X., Lin, H., Yang, L., & Yu, Y. (2017). A convolution-LSTM-based deep neural network for cross-domain MOOC forum post classification. Information8(3), 92-108.
Yadav, A., & Vishwakarma, D. K. (2023). A deep multi-level attentive network for multimodal sentiment analysis. ACM Transactions on Multimedia Computing, Communications and Applications19(1), 1-19.
Yang, L., Li, Y., Wang, J., & Sherratt, R. S. (2020). Sentiment analysis for E-commerce product reviews in Chinese based on sentiment lexicon and deep learning. IEEE access8, 23522-23530.
Yang, X., Zhang, T., Xu, C., & Yang, M. H. (2015). Boosted multi-feature learning for cross-domain transfer. ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM)11(3), 1-18.
Yuan, Z., Wu, S., Wu, F., Liu, J., & Huang, Y. (2018). Domain attention model for multi-domain sentiment classification. Knowledge-Based Systems155, 1-10.
Yuan, Y., & Zhou, Y. (2015). Twitter sentiment analysis with recursive neural networks. CS224D course projects.
Zhang, L., Wang, S., & Liu, B. (2018). Deep learning for sentiment analysis: A survey. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery8(4), e1253.
Zhang, X., & Zheng, X. (2016, July). Comparison of text sentiment analysis based on machine learning. In 2016 15th international symposium on parallel and distributed computing (ISPDC) (pp. 230-233). IEEE.
Zhou, G., Zhou, Y., Guo, X., Tu, X., & He, T. (2015). Cross-domain sentiment classification via topical correspondence transfer. Neurocomputing159, 298-305.
Zhao, C., Wang, S., & Li, D. (2017, September). Deep transfer learning for social media cross-domain sentiment classification. In Chinese National Conference on Social Media Processing (pp. 232-243). Springer, Singapore.
Zhu, F., Dong, X., Song, R., Hong, Y., & Zhu, Q. (2017, November). A multiple learning model based voting system for news headline classification. In National CCF Conference on Natural Language Processing and Chinese Computing (pp. 797-806). Springer, Cham.
Ziser, Y., & Reichart, R. (2018, June). Pivot based language modeling for improved neural domain adaptation. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) (pp. 1241-1251).
Zola, Paola, Paulo Cortez, Costantino Ragno, and Eugenio Brentari. (2019). Social Media Cross-Source and Cross-Domain Sentiment Classification. International Journal of Information Technology & Decision Making (IJITDM). 18(5), 1469-1499.