META’s New Code LLaMA 70b BEATS GPT4 At Coding (Open Source)
AI Summary
Summary: Meta’s Code Llama 70B Release
- Release Announcement:
- Meta released Code Llama 70B, their most advanced coding model.
- It’s available for open-source use under the same license as previous models.
- Three versions: base model, Python-specific, and Instruct model.
- Code Llama 70B Instruct scores 67.8 on HumanEval, one of the highest for open models.
- Models support both research and commercial use.
- Mark Zuckerberg’s Statement:
- Emphasizes the importance of AI in coding.
- AI models help process information across domains.
- Anticipates including these advances in future models like Llama 3.
- Community Contributions:
- Defog Data released SQL Coder 70B, excelling in PostgreSQL text-to-SQL generation.
- SQL Coder 70B is fine-tuned on less than 20,000 hand-curated prompt completion pairs.
- Model Accessibility:
- Available for download after requesting access.
- Access is typically granted quickly.
- License allows for commercial use if changes are open-sourced.
- Testing Code Llama 70B:
- The model is massive, requiring over 30GB of RAM.
- Tested on a VM provided by Mast Compute with GPU acceleration.
- Demonstrated ability to write the Snake game in Python using Pygame.
- Some fine-tuning may be necessary for specific tasks.
- Personal Involvement:
- The author is an investor in LM Studio, which will use Code Llama 70B.
- Disclosures will be made transparent in the description.
- Performance and Limitations:
- While the model is not blazingly fast, it performs well for its size.
- Local testing may be limited by hardware capabilities.
- The generated Snake game code required some adjustments and did not run successfully on the first try.
- Other users have successfully generated a working Snake game using the model.
- Engagement:
- The author invites viewers to suggest tests for Code Llama 70B and to subscribe for future updates.