TITLE: NetFlow Datasets for Machine Learning-based Network Intrusion Detection Systems 

ABSTRACT: Machine Learning (ML)-based Network Intrusion Detection Systems (NIDSs) have
proven to become a reliable intelligence tool to protect networks against
cyberattacks. Network data features has a great impact on the performances of
ML-based NIDSs. However, evaluating ML models often are not reliable, as each
ML-enabled NIDS is trained and validated using different data features that may
do not contain security events. Therefore, a common ground feature set from
multiple datasets is required to evaluate an ML model's detection accuracy and
its ability to generalise across datasets. This paper presents NetFlow features
from four benchmark NIDS datasets known as UNSW-NB15, BoT-IoT, ToN-IoT, and
CSE-CIC-IDS2018 using their publicly available packet capture files. In a
real-world scenario, NetFlow features are relatively easier to extract from
network traffic compared to the complex features used in the original datasets,
as they are usually extracted from packet headers. The generated Netflow
datasets have been labelled for solving binary- and multiclass-based learning
challenges. Preliminary results indicate that NetFlow features lead to similar
binary-class results and lower multi-class classification results amongst the
four datasets compared to their respective original features datasets. The
NetFlow datasets are named NF-UNSW-NB15, NF-BoT-IoT, NF-ToN-IoT,
NF-CSE-CIC-IDS2018 and NF-UQ-NIDS are published at
http://staff.itee.uq.edu.au/marius/NIDS_datasets/ for research purposes.
Machine Learning (ML)-based Network Intrusion Detection Systems (NIDSs) have
proven to become a reliable intelligence tool to protect networks against
cyberattacks. Network data features has a great impact on the performances of
ML-based NIDSs. However, evaluating ML models often are not reliable, as each
ML-enabled NIDS is trained and validated using different data features that may
do not contain security events. Therefore, a common ground feature set from
multiple datasets is required to evaluate an ML model's detection accuracy and
its ability to generalise across datasets. This paper presents NetFlow features
from four benchmark NIDS datasets known as UNSW-NB15, BoT-IoT, ToN-IoT, and
CSE-CIC-IDS2018 using their publicly available packet capture files. In a
real-world scenario, NetFlow features are relatively easier to extract from
network traffic compared to the complex features used in the original datasets,
as they are usually extracted from packet headers. The generated Netflow
datasets have been labelled for solving binary- and multiclass-based learning
challenges. Preliminary results indicate that NetFlow features lead to similar
binary-class results and lower multi-class classification results amongst the
four datasets compared to their respective original features datasets. The
NetFlow datasets are named NF-UNSW-NB15, NF-BoT-IoT, NF-ToN-IoT,
NF-CSE-CIC-IDS2018 and NF-UQ-NIDS are published at
http://staff.itee.uq.edu.au/marius/NIDS_datasets/ for research purposes.
NetFlow Datasets for Machine Learning-based Network Intrusion Detection Systems
http://arxiv.org/abs/2011.09144v1