Chinese startup 01.AI, founded by computer scientist and AI pioneer Kai-Fu Lee, has achieved unicorn status in less than eight months with a valuation of more than $1 billion.
The "Yi-34B" AI model developed by 01.AI is open source without restrictions for non-commercial and research purposes. Lee writes on LinkedIn that China is currently behind in the LLM race, but that the new model "performs competitively at a global level."
The company has published benchmark results showing Yi-34B at or well above the level of Meta's Llama2-70B and Falcon-180B, which have more than twice and five times as many parameters, respectively. The smaller Yi-6B is at the level of the Llama2-34B.
Developers and researchers can use the models under the Apache 2.0 license. They are available on HuggingFace, ModelScope and Github. The models are intended "not just for China, but for the whole world," Lee told Bloomberg.
For "free commercial use," an application must be submitted through the website, which is a bit puzzling since the Apache 2.0 license also states that commercial use is free.
High-quality data improves LLM performance
According to the website, Yi-34B was trained from scratch on a "high-quality" corpus of three trillion tokens. Lee attributes the fact that Yi-34B outperforms much larger models to this high data quality. The smaller size also makes the models cheaper to run.
Other research also shows that data quality has a critical impact on LLM training. Lee expects that scaling up to this level will produce "dramatically better" AI models as early as next year, and he is teasing more models.
According to Lee, 01.AI's next proprietary model should be able to compete with OpenAI's GPT-4. Since the U.S. banned Nvidia from exporting to China, Lee says 01.AI has overdrawn its bank account and bought plenty of chips, which it will need for the foreseeable future.
Kai-Fu Lee joins the AI race
Alibaba Group Holding Ltd.'s cloud division, among others, participated in 01.AI's latest funding round, and the company's valuation has surpassed $1 billion in just eight months.
Kai-Fu Lee, CEO of venture capital firm Sinovation Ventures, will also serve as CEO of 01.AI, Bloomberg reports. Lee began building the team for 01.AI in March 2023, and the company didn't start operations until June. The startup employs more than 100 people, including experienced industry figures who have worked on Google Bard and TensorFlow.
Lee is a well-known figure in the AI industry with extensive experience in computer science and AI research. He holds a Ph.D. in computer science from Carnegie Mellon University and has held numerous leadership positions at renowned technology companies such as Apple, Microsoft, and Google.
