Evaluating ChatGPT for Smart Contracts Vulnerability Correction

The growing number of exploits and hacks on the Ethereum blockchain has led to the development of powerful smart contract vulnerability detection tools and the frequent patching of the smart contract’s programming languages (such as Solidity). At the same time, an ever-increasing number of people are interested in blockchain and smart contract-related topics and willing to build and deploy their own Decentralized Applications (dApp). However, learning a new programming language and its best practices as long as how to actually deploy a smart contract on the blockchain is a difficult task even for experienced developers. Recently, ChatGPT, a new user-friendly deep learning tool, has been released to improve the ability of non-skilled users to write high-quality code and in general, to boost the performances of developers in key tasks related to code writing (i.e., writing functions, explaining runtime errors, fixing bugs, etc.). This paper aims to measure the capabilities of ChatGPT in fixing vulnerable smart contracts and to assess the effectiveness of this tool, determining whether it can be a valuable aid for those who want to correct their own smart contract or want to reuse existing ones by first checking their status and eventually fix their vulnerability. In particular, we asked ChatGPT to fix 143 smart contracts with well-known labeled vulnerabilities. We considered a vulnerability as "fixed" if the code corrected by ChatGPT no longer contained the vulnerability (for this purpose, we exploited Slither, one of the state-of-the-art tools for smart contracts vulnerability detection to check the status of the original and the corrected smart contracts). As a result we obtained that ChatGPT was able to fix bugs and vulnerable smart contracts on average the 57.1% of the time with an increase of +1.4% when a description of the bug was provided in addition to the smart contract’s source code.