ServiceNow, Hugging Face, and Nvidia have released StarCoder2, a family of open-access code generation LLMs. StarCoder2 was developed in collaboration with the BigCode community as a successor to Starcoder, which was released in May 2023 and trained on 619 programming languages. StarCoder2 offers three model sizes: a 3 billion parameter model from ServiceNow, a 7 billion parameter model from Hugging Face, and a 15 billion parameter model from Nvidia.
StarCoder2 has been trained on the new Stackv2 code dataset, which is also available. New training methods are designed to help the model better understand low-resource programming languages, mathematics, and source code discussions. The model can be fine-tuned by companies for their own tasks.
Sources: