tgoop.com/pwnai/694
Last Update:
📝 Vulnerability Detection in Popular Programming Languages with Language Models
Vulnerability detection is crucial for maintaining software security, and recent research has explored the use of Language Models (LMs) for this task. While LMs have shown promising results, their performance has been inconsistent across datasets, particularly when generalizing to unseen code. Moreover, most studies have focused on the C/C++ programming language, with limited attention given to other popular languages. This paper addresses this gap by investigating the effectiveness of LMs for vulnerability detection in JavaScript, Java, Python, PHP, and Go, in addition to C/C++ for comparison. We utilize the CVEFixes dataset to create a diverse collection of language-specific vulnerabilities and preprocess the data to ensure quality and integrity. We fine-tune and evaluate state-of-the-art LMs across the selected languages and find that the performance of vulnerability detection varies significantly. JavaScript exhibits the best performance, with considerably better and more practical detection capabilities compared to C/C++. We also examine the relationship between code complexity and detection performance across the six languages and find only a weak correlation between code complexity metrics and the models' F1 scores.
💡 Key Findings:
• The paper investigates the effectiveness of Language Models (LMs) for vulnerability detection in popular programming languages, including JavaScript, Java, Python, PHP, and Go, in addition to C/C++. This expands the scope beyond previous studies that focused mainly on C/C++.
• The authors utilize the CVEFixes dataset and preprocess it to create language-specific subsets for evaluation. They fine-tune and evaluate state-of-the-art LMs on these subsets to assess their performance in detecting vulnerabilities.
• The results show that JavaScript exhibits the best performance, with considerably better and more practical detection capabilities compared to C/C++. The performance of vulnerability detection varies significantly across the selected languages.
• The paper also analyzes the relationship between code complexity and vulnerability detection performance and finds only a weak correlation between code complexity metrics and the models' F1 scores.
• The main practical implication of this work is the potential use of LMs for vulnerability detection in popular programming languages, particularly JavaScript. The curated dataset, scripts, and experimental results are publicly released to support open science and replication of the findings.
• The limitations of the work are discussed in the paper, and future work could involve exploring other programming languages and investigating techniques to improve the generalization of LMs to unseen code.
👥 Authors: Syafiq Al Atiiq, Kevin Dahlén, Christian Gehrmann
📅 Published: 2024-12-20
🔗 ArXiv
#AI #Detection #Popular #Security #Vulnerability
📂 AI Security papers | 📱 AI Security channels
BY PWN AI
Share with your friend now:
tgoop.com/pwnai/694