Content
summary Summary

With StableCode, Stability AI releases a language model for code generation as open-source software under the Apache license, version 2.0.

Ad

According to Stability AI, StableCode is based on three models: The three billion parameter base model, built on Eleuther's GPT-NeoX, was pre-trained with numerous programming languages from the BigCode dataset and then expanded to include additional programming languages such as Python, Go, Java, Javascript, C, Markdown, and C++, totaling 560 billion tokens.

An instruction model was built on top of the base model, which was refined with concrete application examples to "solve complex programming tasks" using the Alpaca formula. Stability AI used a total of 120,000 pairs of instructions and their solutions.

Stability AI's announcement does not include an evaluation of the model's performance against existing models such as Starcoder or Github Copilot.

Ad
Ad

StableCode 16K: code model with large context window

In addition to the standard model with a 4K context window, StableCode is also available in a 16K variant. The larger context window allows the model to view more code at once to solve a task, potentially generating better code.

According to Stability AI, the 16K model can view or edit the equivalent of up to five medium-sized Python files at once, which should be especially helpful for beginners. Both models can generate and complete single or multiple lines of code.

"People of every background will soon be able to create code to solve their everyday problems and improve their lives using AI, and we’d like to help make this happen," the company writes. Stability AI CEO Emad Mostaque also teases "very interesting variations" of StableCode in the pipeline, claiming that "programming will be transformed so that there will be 1b coders."

In addition to version 1.0 of its SD XL image model, Stability AI released its first open-source language model, StableLM, in April, as well as "Free Willy," a language model based on Meta's Llama v2 and refined with a synthetic dataset. It matches or exceeds the performance of the original model and, to some extent, GPT-3.5 (ChatGPT).

Ad
Ad
Join our community
Join the DECODER community on Discord, Reddit or Twitter - we can't wait to meet you.
Support our independent, free-access reporting. Any contribution helps and secures our future. Support now:
Bank transfer
Summary
  • Stability AI releases StableCode, an open-source language model for code generation that supports a variety of programming languages.
  • The StableCode model is available in 4K and 16K versions. The 16K version's larger context window is particularly useful for novice programmers, Stability AI says.
  • Stability AI also recently released its first open-source language model, StableLM, and "Free Willy," a language model based on Meta Llama v2 that partially matches and exceeds the performance of GPT-3.5 in benchmarks.
Sources
Online journalist Matthias is the co-founder and publisher of THE DECODER. He believes that artificial intelligence will fundamentally change the relationship between humans and computers.
Join our community
Join the DECODER community on Discord, Reddit or Twitter - we can't wait to meet you.