ChatGPT, be my teaching assistant! Automatic correction of SQL exercises

The use of Large Language Models (LLMs) such as OpenAI ChatGPT to enhance teachers’ and learners’ experience has become established. The impressive capabilities of ChatGPT in solving Text2SQL problems prompts their use in database courses to solve SQL exercises. In this paper, we dig deep into ChatGPT abilities applied to SQL exercises. We quantitatively and qualitatively evaluate the performance of a ChatGPT-as-a- SQL-assistant on benchmark data, with particular attention paid to its ability to correctly detect syntactic and semantic errors, provide insightful judgment explanations, and assign grades comparable to those of human teachers. Furthermore, we also analyze the benefits of leveraging few-shot learning to adapt LLM responses to the expectation.