Cerebras and Opentensor have trained a powerful 3 billion parameter language model with an 8k context length window, called BTLM-3B-8k-base, on the Condor Galaxy 1 (CG-1) supercomputer. This new model outperforms similar models, achieves performance comparable to open 7B parameter models, can be quantized to fit on devices with as little as 3 GB of memory, and is licensed for commercial use. It requires 71% fewer training FLOPs and has a 58% smaller memory footprint for inference than comparable 7B models.
BTLM-3B-8k-base brings LLM capabilities to devices with just 3GB of memory
Support our independent, free-access reporting. Any contribution helps and secures our future. Support now:
News, tests and reports about VR, AR and MIXED Reality.Amazon unveils a new generation of its Alexa audio glasses VR Tennis Racket Club to be released in December for Meta Quest Leaked video gives glimpse of Meta Quest 3's passthrough quality MIXED-NEWS.com
Join our community
Join the DECODER community on Discord, Reddit or Twitter - we can't wait to meet you.