BiSplit-DistilBERT: A Lightweight Early-Exit Transformer for Fake News Detection with Cross-Domain Evaluation on BoolQ Question-Answering Data Benchmarked against BERT, RoBERTa, and DeBERTa

Abstract

In the age where people communicate more on digital mediums, authenticity of content is a benchmark. At the same time light weighted models are more preferable. By considering both things a light weighted model is developed in this study by modifying the classification layer of DistiBERT using Bi-Split method. The idea behind Bi-Split method is that prediction is possible by only the first half of the sentence in such cases. This generates an adaptive early-exit approach, in which the model determines whether to end prematurely or proceed with further processing to gain further insight. Upon reaching a predefined threshold, a prediction is made, otherwise the remaining text is analysed to maintain accuracy. The model is tested on four benchmark datasets, GossipCop, PolitiFact, ISOT, and BoolQ, using Accuracy, Precision, Recall, F1-score, and AUC. Accuracies of 99, 94.8, 91.5 and 85% are achieved, showing better performance than baseline models, with SHAP-based interpretability.