A Fine Tuned Universal Language Model Fine-Tuning (ULMFiT) Approach for Airline Twitter Sentiment Analysis
AbstractMore researches have shown that real-time Twitter data can be used to predict market movement of securities and other financial instruments. The Universal Language Model Fine-Tuning (ULMFiT) is a new approach which is based on training a language model and transferring its knowledge to a final classifier. We propose to fine tune the ULMFiT model by optimizing the parameters and training the model in a deterministic approach to increase the reproducibility. In this paper, we performed multi-class classification using Fine-Tuned ULMFiT, Naive Bayes, SVM, Logistic Regression, Random Forest, Decision Tree, K-Nearest Neighbors, on the Twitter US Airline data set from Kaggle. A model is built firstly for six major U.S. airlines that performs sentiment analysis on customer reviews so that the airlines can have fast and concise feedback. Recommendations is made secondly on the most important aspect of services they could improve given customers complains. Significant accuracy has achieved, which shows that our models are reliable for future prediction. Also, the accuracy of different models is compared, and results show that Random Forest is the best approach.
Esi Adeborna, Keng Siau, “An Approach To Sentiment Analysis-The Case of Airline Quality Rating,” PACIS 2014 Proceedings. Paper 363.
Hung T. Vo, Hai C. Lam, Duc Dung Nguyen, Nguyen Huynh Tuong, “Topic Classification and Sentiment Analysis for Vietnamese Education Survey System”, Asian Journal of Computer Science and Information Technology 6:3, May (2016).
Soumi Sarkar, Taniya Seal, “Sentiment Analysis- An Objective View”, Journal of Research, Volume 02, Issue 02, April 2016.
Ann Devitt, Khurshid Ahmad, “Sentiment Analysis and The Use of Extrinsic Datasets in Evaluation,” International Conference on Language Resources and Evaluation,2008.
Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I. and Salakhutdinov, R. (2014). Dropout: A Simple Way to Prevent Neural Networks from Overfitting. J Mach Learn Res 15, 1929-1958. J. Clerk Maxwell, A Treatise on Electricity and Magnetism, 3rd ed., vol. 2. Oxford: Clarendon, 1892, pp.68-73.
Ian Goodfellow, Yoshua Bengio & Aaron Courville (2016). Deep Learning. MIT PressK. Elissa,
Zagoruyko, S., & Komodakis, N. (2016). Wide Residual Networks. Proceedings of the British Machine Vision Conference 2016. doi:10.5244/c.30.87
He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep Residual Learning for Image Recognition. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). doi:10.1109/cvpr.2016.90
J. Howard and S. Ruder, “Universal language model fine-tuning for text classification,” in Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), vol. 1, 2018, pp. 328–339
M. C. Mozer, “A focused backpropagation algorithm for temporal,” Backpropagation: Theory, architectures, and applications, p. 137, 1995
M. E. Peters, W. Ammar, C. Bhagavatula, and R. Power, “Semisupervised sequence tagging with bidirectional language models,” arXiv preprint arXiv:1705.00108, 2017.
S Kamal, N. Dey, A.S Ashour, s. Ripon, V.E. Balas and M. Kaysar, “FbMapping: An automated system for monitoring facebook data”,Neural Network World, 2017.
J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei, “Imagenet: A large-scale hierarchical image database,” in Computer Vision and Pattern Recognition, 2009. CVPR 2009. IEEE Conference on. Ieee, 2009, pp. 248–255.
J. Donahue, Y. Jia, O. Vinyals, J. Hoffman, N. Zhang, E. Tzeng, and T. Darrell, “Decaf: A deep convolutional activation feature for generic visual recognition,” in International conference on machine learning, 2014, pp. 647–655.
J. Long, E. Shelhamer, and T. Darrell, “Fully convolutional networks for semantic segmentation,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2015, pp. 3431–3440.
M. D. d. S. Wanderley, L. d. A. e Bueno, C. Zanchettin, and A. L. Oliveira, “The impact of dataset complexity on transfer learning over convolutional neural networks,” in International Conference on Artificial Neural Networks. Springer, 2017, pp. 582–589.
J. Pennington, R. Socher, and C. Manning, “Glove: Global vectors for word representation,” in Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), 2014, pp. 1532–1543.
T. Mikolov, I. Sutskever, K. Chen, G. S. Corrado, and J. Dean, “Distributed representations of words and phrases and their compositionality,” in Advances in neural information processing systems, 2013, pp. 3111–3119.
P. Bojanowski, E. Grave, A. Joulin, and T. Mikolov, “Enriching word vectors with subword information,” Transactions of the Association for Computational Linguistics, vol. 5, pp. 135–146, 2017.
M. Rei, “Semi-supervised multitask learning for sequence labeling,” arXiv preprint arXiv:1704.07156, 2017.
L. Liu, J. Shang, X. Ren, F. F. Xu, H. Gui, J. Peng, and J. Han, “Empower sequence labeling with task-aware neural language model,” in Thirty-Second AAAI Conference on Artificial Intelligence, 2018.
K. Hashimoto, C. Xiong, Y. Tsuruoka, and R. Socher, “A joint manytask model: Growing a neural network for multiple nlp tasks,” arXiv preprint arXiv:1611.01587, 2016.
B. McCann, J. Bradbury, C. Xiong, and R. Socher, “Learned in translation: Contextualized word vectors,” in Advances in Neural Information Processing Systems, 2017, pp. 6294–6305.
S. Min, M. Seo, and H. Hajishirzi, “Question answering through transfer learning from large fine-grained supervision data,” arXiv preprint arXiv:1702.02171, 2017.
A. Severyn and A. Moschitti, “Unitn: Training deep convolutional neural network for twitter sentiment classification,” in Proceedings of the 9th IJCNN 2019. International Joint Conference on Neural Networks. Budapest, Hungary. 14-19 July 2019 - 6 - paper N-20204.pdfinternational workshop on semantic evaluation (SemEval 2015), 2015,pp. 464–469.
Copyright (c) 2020 International Journal of Sciences: Basic and Applied Research (IJSBAR)
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
Authors who submit papers with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).
- By submitting the processing fee, it is understood that the author has agreed to our terms and conditions which may change from time to time without any notice.
- It should be clear for authors that the Editor In Chief is responsible for the final decision about the submitted papers; have the right to accept\reject any paper. The Editor In Chief will choose any option from the following to review the submitted papers:A. send the paper to two reviewers, if the results were negative by one reviewer and positive by the other one; then the editor may send the paper for third reviewer or he take immediately the final decision by accepting\rejecting the paper. The Editor In Chief will ask the selected reviewers to present the results within 7 working days, if they were unable to complete the review within the agreed period then the editor have the right to resend the papers for new reviewers using the same procedure. If the Editor In Chief was not able to find suitable reviewers for certain papers then he have the right to accept\reject the paper.B. sends the paper to a selected editorial board member(s). C. the Editor In Chief himself evaluates the paper.
- Author will take the responsibility what so ever if any copyright infringement or any other violation of any law is done by publishing the research work by the author
- Before publishing, author must check whether this journal is accepted by his employer, or any authority he intends to submit his research work. we will not be responsible in this matter.
- If at any time, due to any legal reason, if the journal stops accepting manuscripts or could not publish already accepted manuscripts, we will have the right to cancel all or any one of the manuscripts without any compensation or returning back any kind of processing cost.
- The cost covered in the publication fees is only for online publication of a single manuscript.