Global Advanced Research Journal of Educational Research and Reviews (ISSN: 2315-5132) Vol. 11(8) PP. 325-336, November 2023
Available online http://garj.org/garjerr
Copyright © 2023 Global Advanced Research Journals

 

Full Length Research Paper

Hate Speech Detection in Twitter: Natural Language Processing Exploration

Kelly Ochuko Egode¹*, Linda Oraegbunam², Adedamola Samuel Oyatunji³ and Ojore Solomon Akwue⁴

¹MSc Artificial Intelligence and Data Science, University of Hull, UK,
²MSc Applied Artificial Intelligence and Data Analytics, University of Bradford, UK,
³MSc Computer Science, Ulster University, UK.
⁴MSc Big Data and Business Intelligence, School of Computer Science, The Universidad International Isabel I de Castilla, Barcelona, Spain.

*Corresponding Author E-mail: egodekelly@yahoo.co.uk

Accepted 15 November, 2023

Abstract

The proliferation of social media platforms, particularly Twitter, has led to a significant rise in hate speech propagation, posing serious challenges to information dissemination and societal harmony. This paper proposes a novel approach leveraging state-of-the-art natural language processing (NLP) and deep learning techniques to automatically detect and prevent hate speech in real-time on Twitter. By employing machine learning algorithms and deep learning models such as Simple Recurrent Neural Network (SimpleRNN), Long Short-Term Memory Network (LSTM), and Gated Recurrent Unit (GRU), this study aims to surpass existing methods in hate speech classification. Utilising a dataset from Kaggle, the research conducts sentiment analysis and hate speech detection, addressing challenges such as data pre-processing and class imbalance. Various resampling techniques and model architectures are explored to optimise performance metrics including accuracy, precision, recall, F1-score, and area under the precision-recall curve (pr_auc score). The results indicated that while the Naïve Bayes algorithm achieved high precision, deep learning models, particularly best-performing LSTM architecture 2 - include accuracy: 0.950, precision: 0.633, recall: 0.674, F1-score: 0.653, pr_auc score: 0.622, and roc_auc score: 0.870, exhibited promising performance, albeit slightly below baseline expectations. Challenges such as limited training data and imbalanced datasets were identified as key factors impacting model performance. In conclusion, this research underscores the feasibility of leveraging NLP and deep learning for hate speech detection on social media platforms like Twitter. Future work entails exploring advanced models like BERT and ensemble methods to further enhance classification accuracy and mitigate the impact of data scarcity and imbalance.

Keywords: Machine Learning, Deep Learning, Recurrent Neural Network, Natural Language Processing, Hate Speech. 

References

Al-Makhadmeh Z, Tolba Amr (2020). Automatic hate speech detection using killer natural language processing optimizing ensemble deep learning approach. Computing. Archives for Informatics and Numerical Computation, 102 (2): 501-522.

Chuluunsaikhan T, Ryu G, Yoo K, Rah H, Nasridinov A (2020). Incorporating deep learning and news topic modeling for forecasting pork prices: The case of south korea. Agriculture (Basel), 10 (11): 1-22.

Chung, J, Gulcehre C, Cho K, Bengio Y (2014). Empirical evaluation of gated recurrent neural networks on sequence modeling.

Dey N, Mishra R, Fong SJ, Santosh KC, Tan S, Crespo RG (2020). COVID-19: Psychological and psychosocial impact, fear, and passion. Digital Government: Research and Practice, 2 (1): 1-4.

Gaydhani A, Doma V, Kendre S, Bhagwat L (2018). Detecting hate speech and offensive language on twitter using machine learning: An n-gram and tfidf based approach. arXiv Preprint arXiv:1809.08651, .

Greevy E, Smeaton AF (2004). Classifying racist texts using a support vector machine. Proceedings of the 27th annual international ACM SIGIR conference on Research and development in information retrieval.

Jelodar H, Wang Y, Orji R, Huang H (2020). Deep sentiment classification and topic discovery on novel coronavirus or COVID-19 online discussions: NLP using LSTM recurrent neural network approach.

Kaggle (2020). Twitter Sentiment Analysis - Analytics Vidya [Blog post] Practice Problem by Analytics Vidya Vinayak Dhage- March 2020. Available online: https://www.kaggle.com/datasets/dv1453/twitter-sentiment-analysis-analytics-vidya?select=train_E6oV3lV.csv [Accessed: 25/03/2022]

Mandl T, Modha S, Majumder P, Patel D, Dave M, Mandlia C, Patel A (2019). Overview of the hasoc track at fire 2019: Hate speech and offensive content identification in indo-european languages. Proceedings of the 11th forum for information retrieval evaluation.

Omnicore (2022) Twitter by the Numbers: Stats, Demographics and Fun Facts [Blog post] Salman Aslam-February 22, 2022. Available online: https://www.omnicoreagency.com/twitter-statistics/ [Accessed : 25/05/2022]

Oyebode O, Ndulue C, Adib A, Mulchandani D, Suruliraj B, Orji FA, Chambers CT, Meier S, Orji R (2021). Health, psychosocial, and social issues emanating from the COVID-19 pandemic based on social media comments: Text mining and thematic analysis approach. J. MIR Med. Informatics. 9 (4): e22734.

Pitsilis GK, Ramampiaro H, Langseth H (2018). Effective hate-speech detection in twitter data using recurrent neural networks. Applied Intelligence (Dordrecht, Netherlands), 48 (12): 4730-4742.

Raza H, Faizan M, Hamza A, Mushtaq A, Akhtar N (2019). Scientific text sentiment analysis using machine learning techniques. Int. J. Adv. Comp. Sci. Appl. 10 (12):157-165.

Rustam F, Khalid M, Aslam W, Rupapara V, Mehmood A, Choi GS (2021a). A performance comparison of supervised machine learning models for covid-19 tweets sentiment analysis. PloS One. 16 (2): e0245909.

Rustam F, Khalid M, Aslam W, Rupapara V, Mehmood A, Choi GS (2021b). A performance comparison of supervised machine learning models for covid-19 tweets sentiment analysis. PloS One, 16 (2): e0245909.

Hochreiter S, Schmidhuber J (1997). Long short-term memory. Scikit-learn: Machine Learning in Python, Pedregosa et al., JMLR 12, pp. 2825-2830, 2011. Available Online :https://scikit-learn.org/stable/auto_examples/datasets/plot_iris_dataset.html [Accessed: 11/04/2022]

Staudemeyer RC, Morris ER (2019). Understanding LSTM--a tutorial into long short-term memory recurrent neural networks. arXiv Preprint arXiv:1909.09586, .

Tom Davidson (2017). hate-speechand-offensive-language. https: //github.com/t-davidson/ hate-speech-and-offensive-language. Accessed: 2021-03-29.

Van Den Broeck J, Cunningham SA, Eeckels R, Herbst K (2005). Data cleaning: Detecting, diagnosing, and editing data abnormalities. PLoS Med. 2 (10): 966.

Zagidullina A, Patoulidis G, Bokstaller J (2021). Model bias in NLP -- application to hate speech classification using transfer learning techniques.

 Zhang Aston, Lipton Zachary C, Li Mu, Smola Alexander J (2021). Dive Into Deep Learning. zhang2021dive. Online Source: https://d2l.ai/chapter_recurrent-modern/gru.html [Accessed : 25/05/2022]

 

Download Full Text - PDF