DBRX by Databricks

DBRX by Databricks is an advanced open-source large language model (LLM) designed for efficiency and quality, leading in AI benchmarks. It is specifically suitable for developers, data scientists, and researchers looking to leverage the power of LLMs for various applications. DBRX stands out due to its fine-grained mixture-of-experts (MoE) architecture, which allows for better tradeoffs between model quality and inference efficiency compared to dense models.
The model was trained using next-token prediction and utilizes a fine-grained MoE architecture with 132B total parameters, of which 36B parameters are active on any input. This approach enables DBRX to achieve state-of-the-art quality with marked improvements in training and inference performance. For instance, DBRX is up to 2x faster in inference than LLaMA2-70B and requires less compute for training, making it a more efficient option for users.
DBRX is particularly valuable for individuals and teams focusing on programming and mathematics, as it outperforms other open models in these areas. Its ability to generate text at up to 150 tokens per second when hosted on Databricks Model Serving makes it a powerful tool for applications requiring rapid and accurate text generation. With its open-source nature, DBRX also offers a community-driven approach to AI development, allowing users to contribute and benefit from collective advancements in the field.