The Code Interpreter is probably the most interesting ChatGPT plugin of OpenAI and opens up completely new capabilities for the Chatbot.
At the end of March, OpenAI introduced a groundbreaking new feature for ChatGPT: Plugins. One of them is a so-called Code Interpreter. With it, the language model can not only generate code, but also execute it independently.
As with Auto-GPT, the busy developer community has found exciting use cases for this technology in a very short time. Especially for data journalism and similar data-based analysis, the tool seems to open up completely new possibilities. This is also due to the possibility of uploading and downloading files up to 100 MB in size.
We have compiled some of the most exciting demonstrations of ChatGPT's Code Interpreter.
As mentioned earlier, one strength of the Code Interpreter lies in the visualization of data. The software is not limited to any particular type of display or graphic. The possibilities range from simple bar graphs to more complex and lesser-known graphs.
So it looks like ChatGPT with Code Interpreter can do basically any visualization technique you ask. And you can tell it to make charts "more beautiful"
(I actually asked it to generate artificial data to create the charts to show what they could do, which the AI happily did😬) pic.twitter.com/CsmRO8vDe1
— Ethan Mollick (@emollick) May 2, 2023
Clean data and generate synthetic data
ChatGPT can not only process the data, but also clean it or generate synthetic data in case of doubt. The AI is not completely flawless: "If you give it something with a lot of text, it seems to tokenize that and the data you are cleaning can come back somewhat mangled," notes data scientist Kathryn Cramer on Twitter.
The code interpreter feature on ChatGPT is the most mind blowing thing I've seen yet.
All I did was upload a CSV of SF crime data and ask it to visualize trends(!!) pic.twitter.com/pkFdPqgAzb
— John Backus (@backus) April 29, 2023
Analyze bitcoin rates
The code interpreter can also analyse bitcoin prices, as demonstrated by Twitter user @TechMemeKing. Once a dataset has been uploaded, the ChatGPT plugin accepts various requests to display the data as a line or bar chart, for example. "I did this with an iPhone while lying in bed .... This is a game changer for exploratory data analysis," he comments.
Graph the log price of bitcoin
Seasonally decompose the price since 2011
Give me the bitcoin price variance vs average by day as well as the average price by month
Graph the log distribution of prices
Graph common moving averages
Analyzing your own musical tastes
Twitter user @SHL0MS went a step further and analyzed a Spotify playlist of about 300 hours of all his favorite songs. ChatGPT not only provided the visualization, but the AI also helped with data retrieval and explained how to use the Spotify API accordingly.
analyzing a 300hr Spotify favorites playlist with ChatGPT Code Interpreter
i used to use a Jupyter notebook some guy made for PCA analysis but it broke
GPT showed me how to export from the Spotify API, ran multidimensional PCA and t-SNE analyses, and summarized my taste for me pic.twitter.com/cVmUvuuRdM
— █̶̳̘͛̄̃͒̄̃͜█̴͇̱̅͒̅█̵̻̣̝͒̈̄̈͝͝█̴̞̜̻̝͍̂̽͜█̷̢͔̭̝̼̾̍̚ (@SHL0MS) May 1, 2023
Create HTML page
While ChatGPT visualised the data as images in the previous examples, Patrick Blumenthal found a way to output the result as an HTML page. After uploading a raw dataset, the code interpreter generated a fully functional HTML page. It was originally a world map, but ChatGPT was able to crop the map to North America on request.
OpenAI's new Code Interpreter model is incredible for data visualization. I fed it an unpolished dataset of UFO sightings and it generated a fully-functioning HTML heat map for me.
Here's a static version (it easily lets me create as many versions as I want) pic.twitter.com/mDVU2Rp8BR
— Patrick Blumenthal (@PatrickJBlum) April 29, 2023
Create animated GIFs
In addition to data processing, the Code Interpreter has limited multimedia capabilities. For example, a short prompt can be used to create a GIF animation.
Make a 512x512 GIF with falling green Matrix letters. Assume no fonts. 30 frames 5 fps. No talk, just go.
ChatGPT Code Interpreter (alpha) renders an animated GIF: pic.twitter.com/2txArURcu1
— Riley Goodside (@goodside) April 28, 2023
The ability to create GIFs from scratch can of course be combined with the data analysis capabilities described above. Like here to show all lighthouse sites in the USA.
Create a gif of a map of the lighthouse locations, where the map is very dark but each lighthouse twinkles
This was kind of delightful: I uploaded a CSV file of every lighthouse location in the US.
"ChatGPT Code Interpreter: Create a gif of a map of the lighthouse locations, where the map is very dark but each lighthouse twinkles." A couple seconds later... pic.twitter.com/f14JLWQCyB
— Ethan Mollick (@emollick) May 2, 2023
Simple video editing
However, the Code Interpreter handles not only GIFs, but other file formats as well. For example, Prompt Engineer Riley Goodside uploaded a GIF and asked ChatGPT to convert it to an MP4 with a dramatic zoom. ChatGPT doesn't store data for long, but asks for a re-upload if in doubt.
I'll upload a gif and you give me a 5s mp4 with a dramatic slow zoom-in. No talk; just go.
came back 1hr later to make a 10s version and check if my upload was still in the mount dir. it wasn’t, but GPT was smart enough to figure out what happened and ask for a re-upload: pic.twitter.com/52FEyb5ZuI
— Riley Goodside (@goodside) April 30, 2023
Create colour palette from image
ChatGPT can create a colour palette from an image with the code interpreter enabled. Thanks to the upload and download functions, it extracts colours from an image and then creates a "palette.png". If space is at a premium, it can also automatically compress large images, making the language model behind ChatGPT a more practical tool for designers.
Create a script that extract the top 5 colors from a picture then make into a palette.png to download, wait for the image upload
ChatGPT Code Interpreter is incredible! 🌟
Its file uploading & downloading capabilities make it a super flexible tool for various tasks. 💼
Here it extracts colors from an image to create a palette.png 🎨
And automatically compresses large images when running out of memory. pic.twitter.com/i25yPujtBV
— Pietro Schirano (@skirano) May 1, 2023
Convert image to text with OCR
The code interpreter generally improves the multimodal capabilities of ChatGPT. This also means that it is possible to quickly convert images into text files. An essential keyword for this capability is OCR (Optical Character Recognition), which scanners are capable of. ChatGPT will ask you to upload the file and then process it to produce the text.
OCR an image for me and generate a text file
Multimodal AI is here 🤯
GPT-4 can now turn your images into a text file in a snap with the new code interpreter model.
Witness the OCR magic in action 🔥 pic.twitter.com/c9mnJfiweg
— Shubham Saboo (@Saboo_Shubham_) May 5, 2023
"When I realized that ChatGPT understands a lot of mathematical data and CI can generate downloadable files, I asked ChatGPT to generate a Shepard tone (a sound illusion that seems like it’s constantly getting higher in pitch," Andrew Mayne reports on his blog. Initially, ChatGPT displayed this sound as a graph, but could also output it as a WAV file via a code interpreter.
Create a Shepard tone and save it as a wav
Create QR codes
Although there are many free services available on the internet: For the sake of completeness, it should be mentioned that ChatGPT can generate QR codes with the code interpreter, which can then be downloaded as graphics for further use.
Create a QR code for OpenAI.com
Recognize faces in pictures
Since the code interpreter has access to many Python libraries, it can also be used to recognise faces, as Andrew Mayne demonstrates. In this case, he uses the Haar Cascade Classifier.
Convert images to ASCII
The code interpreter can open images and create graphics from them. In this way, for example, a drawn work of art becomes an ASCII graphic.
Code interpreter available for all Plus customers
Initially the code interpreter was only in closed beta, but since the beginning of July the plugin has been available to all paying Plus customers of ChatGPT. In order to use it, you will need to enable the interpreter in the "Beta Features" section of your settings, so that you can select it in GPT-4.