GPTutor is designed to help beginners learn to code - or professionals understand the code of others. According to the team, the system provides better explanations than ChatGPT.
AI-based coding tools such as GitHub CoPilot are now widely used, and the coding capabilities of ChatGPT and especially GPT-4 are being used by non-professionals to develop their own applications. But the tools also have great potential as digital tutors to teach beginners how to code, or to help professionals understand others' existing code more quickly.
Researchers from Taiwan and Canada present the Visual Studio Code Extension GPTutor, a ChatGPT-based programming tool for explaining code.
GPTutor bypasses problems of ChatGPT & CoPilot
According to the team, GPTutor provides better results than CoPilot or ChatGPT when explaining code. The team identified three issues as common limitations of existing systems:
- Superficial insights: They only provide insights based on the code in the current file and may miss domain logic, especially when analyzing object-oriented code that imports objects from other files.
- Unnecessary or irrelevant information: Other systems often provide unnecessary explanations for all code, even for a specific line of code, resulting in unnecessary or even fictitious information.
- Lack of up-to-date knowledge: Language models such as ChatGPT have been trained with data up to 2021, and therefore cannot provide satisfactory explanations for new, unpopular, or private libraries beyond that time period.
GPTutor avoids these problems by enriching the underlying model - OpenAI's GPT-3.5-turbo was used in the tests - with additional information. The extension uses various APIs in Visual Studio Code to extract the programming language and code of the current file, as well as the source code behind selected functions. This data is then sent to a language model that uses the available information to explain the code. If only one line of code is selected, only that line is explained.
GPTutor provides better explanations than ChatGPT
In tests, the team says it has shown that GPTutor provides more relevant explanations on average than CoPilot or ChatGPT. It is less likely to produce irrelevant or even incorrect information, and by relying on external code, it can provide information that no other system can.
In the future, the team plans to further improve GPTutor's performance through real-time engineering. For example, they say, a heuristic search could be used to identify relevant code in the code base. After transforming the code into many possible prompts, GPTutor could offer different explanations to identify user preferences and provide them with personalized explanations and a better user experience.
GPTutor is available in the Visual Studio Code Extension Marketplace and the source code is available on GitHub.
