Agency Swarm Now Supports Open Source Models

---

---

AI Summary

Summary of Framework Disadvantages and Updates

Disadvantages

• Framework is fully dependent on OpenAI’s API.
• Does not support open source models natively.

Open Source Model Support

• Agency Swarm now supports open source models.
• Users can use models like Anthropic, Google Gemini, Mixr, or Llama 3 locally with AMA.
• Flexibility to combine different agents with different LLMs.

Assistance API V2 Features

• Retrieval tool replaced with file search.
• Enterprise-level RAG (Retrieval-Augmented Generation) out of the box.
• No need for third-party vector databases.
• Techniques include query rewriting, parallel searches, and result reranking.

New Parameters

• Control over the maximum number of tokens used.
• Max prompt tokens parameter added.
• Truncation strategy parameter to control prompt token truncation.
• Max completion tokens to limit generated tokens.
• Parameters can be set within each assistant or agency class.

Temperature Parameter

• Temperature parameter controls creativity vs. hallucinations.
• Default set to 0.3, but 0 recommended for coding use cases.

Tool Choice

• Select which tool the assistant will use.
• Can force the use of specific tools like send message.

Few-Shot Learning

• Add assistant messages to threads for few-shot learning.
• Useful for creating text in a specific style.

JSON Mode

• Force assistant to respond in JSON object format.
• Instructions must specify JSON use.

Fine-Tuned Models

• Use fine-tuned models within each assistant.
• Accessible in the playground under the model parameter.

Using Framework with Open Source Models

• Install Agency Swarm Lab repository.
• Set up API keys and models in the light LLM config file.
• Install AMA for local Llama 3 model.
• Run light LLM proxy server and open Assistant API backend.
• Create agents for different models.
• Set up agency structure in the agency.py file.
• Run demo gradio method for interface.

Function Calling Support

• Not all models support function calling.
• Check support using function_calling_support.py.

Documentation and Community

• Detailed setup instructions in the open source swarm directory.
• Community discussions on Discord.

Conclusion

• The framework now offers flexibility with open source models.
• New features and parameters enhance production applications.
• Users should be aware of function calling limitations with some models.