With the half-published, half-leaked LLaMA model, Meta has started an open-source movement for language models. Meta plans to keep it open in the future.
At an internal meeting and in the latest Lex Fridman podcast, Zuckerberg emphasizes that he still sees a lot of potential in open source. He counters criticism of this approach, especially regarding safety, by pointing to the still limited capabilities of the systems themselves, but also by comparing Meta to the much larger models of Google or OpenAI.
If AI does indeed reach superintelligence status, this stance will need to be reconsidered, but until then, open-sourcing AI systems is the best way to get feedback on safety and alignment with human needs, according to Zuckerberg. The feedback and close collaboration with outsiders would help Meta improve its technologies.
Internally, Zuckerberg has expressed similar sentiments about the open-source strategy, according to the New York Times, saying that "democratizing" AI has many benefits and also fits with the company's product vision of enabling as many AI systems as possible, rather than making just one to rule them all.
"Different people have different interests, and we’ll need a diverse array of AIs to represent all of these different interests," Zuckerberg said.
LLaMA v2 to be more than a research project
Meta is working on a successor to LLaMA, possibly "LLaMA v2". Of course it might get a new name. This is still up in the air.
Zuckerberg says the new model will become a core infrastructure product for Meta, serving many purposes in its own products and "hopefully many others." Meta itself is apparently planning chatbots in WhatsApp and Instagram, as well as an AI assistant for smart glasses and headsets.
Meta's focus is on safety and the integration of state-of-the-art technologies. Accountability and alignment are two key aspects of the development. In addition, training will be done with "more data from the company's own services" compared to LLaMA, Zuckerberg said.
It is still unclear whether LLaMA v2 will be open source again. Zuckerberg does not rule it out, telling Fridman that Meta is primarily discussing the "how" internally. LLaMA v1 was released under a research license and therefore may not be used for commercial projects, which Zuckerberg says is widely respected.
Meta's LLaMA has served as the basis for numerous iterations and evolutions of open-source chatbots and language models, such as Alpaca, Vicuna, or OpenAssistant.
