Performance Evaluation of Parametric and Non-Parametric Machine Learning Models using Statistical Analysis for RT-IoT2022 Dataset

Abstract

With the vast growth of the Internet of Things (IoT) applications, the number of devices connected to the IoT is increasing exponentially. On the other hand, cybercriminals are generating sophisticated new cyber attacks to exploit IoT devices. However, conventional Intrusion Detection Systems (IDS) that rely on an alert-based approach fail to detect these novel attacks. Machine Learning (ML) based IDS has the potential to spot even small mutations and new threats. This study investigates the statistical tests like Kolmogorov-Smirnov (KS) test, skewness test, kurtosis test, Pearson‘s Correlation Coefficient (PCC) and Information Gain Ratio techniques on the recently introduced RT-IoT2022 dataset. The aim is to determine the optimal machine learning algorithms for detecting vulnerabilities within this dataset. The results of skewness and kurtosis tests identify the features having outliers and the KS test indicates that the proposed dataset exhibits non-parametric characteristics. Subsequently, Pearson‘s Correlation Coefficient (PCC) and Information Gain Ratio techniques are applied to analyze the correlation between features and the categories of the target attacks. Further, parametric and non-parametric ML models are tested to validate the results of statistical tests. With the non-parametric Decision Tree algorithm achieving the highest accuracy of 99.85% among all other models, we conclude that non-parametric ML models are optimal for detecting mutant vulnerabilities.