Just in LLAMA 4 with 10 Million Context!!!
AI Summary
Llama 4 Overview
- Release: Llama 4 is officially available for download with three variants: Behemoth, Maverick, and Scout.
Model Variants
- Llama 4 Behemoth
- Largest version, still under training.
- Expected to outperform existing models like Gemini 2.0 Pro.
- Parameters: 288 billion active parameters, 2 trillion total.
- Llama 4 Maverick
- 17 billion active parameters, comprising 128 experts.
- Achieves significant benchmarks, outperforming GPT-4 and Gemini 2.0 Flash.
- Context length: 1 million tokens.
- Llama 4 Scout
- Smallest variant with 17 billion active parameters and 16 experts.
- Features the highest context length: 10 million tokens.
- Aimed for single GPU usage.
Licensing and Access
- License: Restrictions exist; not available for entities with over 700 million monthly active users.
- Downloading requires filling out a form on the relevant website. Users can download models a limited number of times (five times within 48 hours).
Performance Claims
- Llama 4 models purportedly deliver better performance results at lower operational costs compared to competitors.
- A focus on multimodal capabilities and efficiency in computational resource use.
- Meta claims Llama 4 Maverick scored 1417 on LM Arena, making it top-tier in performance.
- A significant number of benchmarks suggest improvements over prior generations.
Conclusion
- Llama 4 represents a milestone in open-source AI by combining scalability and efficiency in model performance. Further details about its deployment and additional models to come soon.