Code Llama

Code Llama is a family of large language models for code, developed by fine-tuning Llama 2 using a higher sampling of code. It is designed for individuals, creators, researchers, and businesses of all sizes to experiment, innovate, and scale their ideas responsibly. The models are trained on sequences of 16k tokens and show improvements on inputs with up to 100k tokens. Code Llama was developed to provide state-of-the-art performance among open models, infilling capabilities, support for large input contexts, and zero-shot instruction following ability for programming tasks.
The Code Llama models come in multiple flavors, including foundation models, Python specializations, and instruction-following models, each with 7B, 13B, and 34B parameters. The 7B and 13B Code Llama and Code Llama - Instruct variants support infilling based on surrounding content. To get the expected features and performance for the 7B, 13B, and 34B variants, a specific formatting defined in chat_completion() needs to be followed.
Code Llama is most valuable for developers, researchers, and businesses that need to generate high-quality code, automate coding tasks, or improve their coding efficiency. It can be used for a wide range of applications, including code completion, code generation, and code review. With its large language models and infilling capabilities, Code Llama can help users save time, reduce errors, and improve their overall coding experience.