Real-time anomaly detection for network traffic patterns using machine learning

Abstract

With the increasing complexity and volume of network traffic, ensuring the security and stability of computer networks is paramount. Traditional rule-based approaches for detecting anomalies in network traffic have limitations in handling evolving threats and detecting previously unseen patterns. To address this challenge, we propose a real-time anomaly detection system leveraging machine learning techniques. This project presents a real-time network anomaly detection system using machine learning and packet analysis. The system employs the Isolation Forest algorithm, a popular unsupervised method for anomaly detection, to identify irregular network behavior. Network packets are captured live using the Pyshark library, with relevant features such as packet length and source port extracted and preprocessed. The processed data is scaled using StandardScaler before being fed into the Isolation Forest model. The synthetic training dataset is generated to include a mix of normal and anomalous data points, with a focus on having a higher proportion of normal data points, reflecting typical network conditions. After training, the model is evaluated using accuracy, precision, recall, and F1-score, with results plotted for visualization. The system operates in a continuous loop, capturing live network traffic, preprocessing the data, and predicting anomalies in real-time. Detected anomalies are visualized, and performance metrics are displayed. Additionally, the system is integrated with an email notification feature that alerts administrators when anomalies are detected, providing key metrics in the notification. This system is valuable for real-time network monitoring and security, helping detect potential threats as they occur, with the added capability of immediate administrator notification.