Google has expanded its open-source Gemma AI model family with a new Japanese version that packs GPT-3.5-level performance into a tiny package.
The company announced the release at Gemma Developer Day in Tokyo, noting that the model achieves strong Japanese language capabilities while maintaining its English language capabilities - all with just 2 billion parameters.
This is noteworthy because very small models in particular can experience what is known as "catastrophic forgetting," where a model forgets previously learned information as it acquires new knowledge. This can happen during fine-tuning for new languages.
Google has made the model weights immediately available through Kaggle and Hugging Face, along with training materials and examples to help developers get started. The new Japanese model is compact enough to run locally on mobile devices, making it particularly useful for developers working on edge computing applications.
To spur international development, Google announced a $150,000 competition called "Unlocking Global Communication with Gemma." The initiative aims to help developers adapt Gemma for their local languages, with projects already underway for Arabic, Vietnamese, and Zulu.
In India, developers are working on the "Navarasa" project to optimize the model for twelve Indian languages, while another team is exploring Korean dialect support through fine-tuning.
Going global with local AI
The Gemma 2 models, which Google open-sourced in late July, aim to maximize performance with few parameters. Like similar models from Meta and others, they are designed for efficiency and local processing. For example, the 2-billion-parameter version of Gemma 2 can outperform some much larger models, such as the 70-billion-parameter LLaMA-2.
Developers and researchers can access the Gemma-2-2B model and other Gemma models through various platforms, including Hugging Face, Google AI Studio, and Google Colab's free plan. The models are also available through the Vertex AI Model Garden.
