Function Calling with Local Models & LangChain - Ollama, Llama3 & Phi-3

---

---

AI Summary

• Introduction to Local Model Usage
  • Previously focused on cloud-based Llama 3 model (70B parameters)
  • Now exploring local use of Ollama and Llama 3 (8B parameters) for function calling
• Transition to Local Models
  • Aim to build agents locally without cloud dependency
  • Benefits include cost savings and no token limits
• Function Calling with Local Models
  • Past videos used proprietary cloud models for function calling
  • Current focus on local models for structured outputs and JSON
• Function Calling Leaderboard
  • Reference to a function calling leaderboard from a paper called “gorilla”
  • Llama 3 (70B) and Llama 3 (8B) models are on the leaderboard
• Setting Up Llama 3 with Ollama
  • Use chat Ollama for the model
  • Keep alive property is important for notebooks to avoid reloading the model
• Creating Prompt Templates
  • Example of a 500-word article prompt
  • Use of LangChain expression language for dynamic content generation
• Streaming vs. Invoke Responses
  • Streaming allows seeing the output as it’s generated
  • Invoke returns the entire output at once
• JSON Output with Llama 3
  • Define a JSON schema for structured output
  • Use JSON output parser to get dictionary responses
• Structured Responses with Ollama Functions
  • Use Pydantic to define structured output classes
  • Example of extracting information about characters’ heights and hair colors
• Testing with Phi-3 Model
  • Phi-3 is smaller than Llama 3 but can still produce structured outputs
  • Example of using Phi-3 for similar tasks as Llama 3
• Tool Use and Function Calling
  • Introduction to Ollama functions for binding tools to models
  • Example of a weather tool function call
• Comparison of Llama 3 and Phi-3 for Function Calling
  • Llama 3 is more capable but Phi-3 can still handle simple tasks
• Future Work
  • Plans to build agents and RAG systems using local models
  • Interest in community feedback on tool usage for agents
• Conclusion
  • Encouragement for comments and questions
  • Invitation to like and subscribe for future content updates