Cerebras and Opentensor have trained a powerful 3 billion parameter language model with an 8k context length window, called BTLM-3B-8k-base, on the Condor Galaxy 1 (CG-1) supercomputer. This new model outperforms similar models, achieves performance comparable to open 7B parameter models, can be quantized to fit on devices with as little as 3 GB of memory, and is licensed for commercial use. It requires 71% fewer training FLOPs and has a 58% smaller memory footprint for inference than comparable 7B models.
BTLM-3B-8k-base brings LLM capabilities to devices with just 3GB of memory
Support our independent, free-access reporting. Any contribution helps and secures our future. Support now:
News, tests and reports about VR, AR and MIXED Reality.Try one of the most anticipated Quest games of the year as Mannequin enters Open Alpha Quest 3 looks much better with this Vision Pro mod The first Somnium VR1 enthusiast PC VR headsets could ship in April MIXED-NEWS.com
Join our community
Join the DECODER community on Discord, Reddit or Twitter - we can't wait to meet you.