DeepCoder 14B This LOCAL Opensource AI Coding MODEL is CRAZY!
AI Summary
Overview
- Introduction to Deepcoder 14B, a coding-focused LLM.
- Specifically designed for code generation and understanding with 14 billion parameters.
Key Features
- Achieves 60.6% pass accuracy on Live Codebench, an 8% improvement over its base model.
- Outperforms OpenAI’s 03 mini model while having fewer parameters.
- Open weights available on Olma for easy accessibility.
Fine-Tuning Process
- Fine-tuned from Deepseek R1 distilled model using Reinforcement Learning.
- Utilized 24,000 unique problem test pairs from multiple sources.
- Training employed Generalized Reward Policy Optimization and Direct Preference Optimization techniques.
Performance Improvements
- Enhanced capabilities for handling long context outputs.
- Generalizes well at 64,000 token context length, maintaining a 60.6% accuracy.
- Use of overlong filtering during training aids performance on longer contexts.
Training Verification
- Employs code sandbox execution for reinforcement learning evaluation.
- Tested across 1024 different problems with multiple unit tests.
Accessibility
- Available on Olma; easy to set up and run locally.
- Users can select between the 1.5B and 14B parameter models.
User Experience
- Initial tests show improved performance over the previous distilled variant for complex tasks.
- However, responses can be slow, especially for simpler tasks due to deep reasoning processes.
- Weakness observed in tool calling/friction with interfaces but works well in conversational settings.