Security Data Collection and Data Analytics in the Internet: A Survey

Xuyang Jing, Zheng Yan, Witold Pedrycz

Research output: Contribution to journalReview Articlepeer-review

33 Citations (Scopus)
319 Downloads (Pure)


Attacks over the Internet are becoming more and more complex and sophisticated. How to detect security threats and measure the security of the Internet arises a significant research topic. For detecting the Internet attacks and measuring its security, collecting different categories of data and employing methods of data analytics are essential. However, the literature still lacks a thorough review on security-related data collection and analytics on the Internet. Therefore, it becomes a necessity to review the current state of the art in order to gain a deep insight on what categories of data should be collected and which methods should be used to detect the Internet attacks and to measure its security. In this paper, we survey existing studies about security-related data collection and analytics for the purpose of measuring the Internet security. We first divide the data related to network security measurement into four categories: 1) packet-level data; 2) flow-level data; 3) connection-level data; and 4) host-level data. For each category of data, we provide a specific classification and discuss its advantages and disadvantages with regard to the Internet security threat detection. We also propose several additional requirements for security-related data analytics in order to make the analytics flexible and scalable. Based on the usage of data categories and the types of data analytic methods, we review current detection methods for distributed denial of service flooding and worm attacks by applying the proposed requirements to evaluate their performance. Finally, based on the completed review, a list of open issues is outlined and future research directions are identified.

Original languageEnglish
Article number8428412
Pages (from-to)586 - 618
Number of pages33
JournalIEEE Communications Surveys and Tutorials
Issue number1
Early online date2018
Publication statusPublished - 1 Jan 2019
MoE publication typeA2 Review article in a scientific journal


  • Computer crime
  • Data analysis
  • data analytics
  • Data collection
  • data collection
  • DDoS flooding attacks
  • Grippers
  • Internet
  • Protocols
  • security measurement.
  • Security-related data
  • worm attacks

Fingerprint Dive into the research topics of 'Security Data Collection and Data Analytics in the Internet: A Survey'. Together they form a unique fingerprint.

Cite this