Machine Learning Approach to Improve Data Connectivity in Text-based Personality Prediction using Multiple Data Sources Mapping

Abstract

This paper considers the task of personality prediction using social media text data. Personality datasets with conventional personality labels are few, and collecting them is challenging due to privacy concerns and the high expense of hiring expert psychologists to label them. Pertaining to a smaller number of labelled samples available, existing studies usually adds a sentiment, statistical NLP features to the text data to improve the accuracy of the personality detection model. To overcome these concerns, this research proposes a new methodology to generate a large amount of labelled data that can be used by deep learning algorithms. The model has three components: general data representation, data mapping and classification. The model applies Personality correlation descriptors to incorporate correlation information and further use this information in generating dataset mapping algorithm. Experimental results clearly demonstrate that the proposed method beats strong baselines across a variety of evaluation metrics. The results had the highest accuracy of 86.24% and 0.915 F1 measure score on the combined MBTI and Essays dataset. Moreover, the new dataset constructed contains 3,84,089 labelled samples on the combined dataset and can be further considered for personality prediction using the famous Five Factor Model thereby alleviating the problem of limited labelled samples for the purpose of personality detection.