Analysing and scoring cybersecurity vulnerability reports using deep learning

Abstract

Vulnerability analysis plays a crucial role in cybersecurity, as it helps identify and mitigate potential threats to systems and networks. It is essential for organizations to understand the severity of vulnerabilities and prioritize their remediation efforts. This thesis focuses on the analysis and scoring of cybersecurity vulnerability reports using deep learning techniques. By leveraging large language models and machine learning algorithms, this thesis aims to improve the accuracy and efficiency of vulnerability classification and severity assessment.

The research presented in this thesis includes the development of models to estimate the CVSS of vulnerabilities based on their descriptions, comparing the performance of AI models with human experts. For CVSS 3.1 severity level prediction task, the USE model achieves an accuracy of 0.73. The USE model outperforms human experts with a macro mean squared error of 0.128, which is four times lower than that of human experts (0.537). Meanwhile, quality of vulnerability descriptions is also analyzed to understand its impact on the performance of machine learning models. A BERT-based model (‘DesQ’) is proposed to assess the quality of vulnerability descriptions, achieving an accuracy of 0.76. A dataset of vulnerability descriptions with quality labels is curated and analyzed to understand the distribution of description quality across different vendors and reporting agencies. The analyzer BART model (‘EVAL’) identifies missing essential vulnerability aspects. Trained on a specialized quality dataset of 450 manually labeled entries, EVAL identifies Vulnerability Impact as the most frequently missing component across analyzed weaknesses. This thesis also explores the use of explainable AI techniques to provide insights into the decision-making process of machine learning models, enhancing their interpretability and trustworthiness in mapping root causes to vulnerabilities.