Deploying code agents without all the agonizing pain

---

---

AI Summary

Summary: Deployment of Lang Chain Code Assistant

• Initial State:
  • Notebook with Lang Chain code assistant.
  • Code checks for imports and execution within the notebook environment.
  • Not deployable due to security and environment constraints.
• Charles’ Contributions:
  • Refactored code for deployment.
  • Changed code execution to an isolated sandbox for security.
  • Wrapped the Lang Chain graph in app.py as a FastAPI app using Lang Serve.
  • Added Modal decorators for deployment on Modal’s platform.
  • Integrated Modal sandbox for secure code execution.
• Deployment Process:
  • Graph from the notebook is imported into app.py.
  • FastAPI app is created with Lang Serve, mapping graph invocation methods to HTTP endpoints.
  • Modal decorators enable deployment to Modal with sandboxing for code execution.
• Pain Points Addressed:
  • Security considerations with sandboxing and secrets management.
  • Development and production environment discrepancies.
  • Slower development cycles typical when interacting with production.
  • Monitoring and observability with Lang Smith.
  • Scalable infrastructure for evaluation and improving the system using production data.
• Benefits:
  • Secure, deployable code execution environment.
  • Simplified deployment process with minimal code changes.
  • Fast iteration and debugging with Modal’s development server (modal.serve).
  • User experience improvements with intermediate state streaming.
  • Observability tools for monitoring and evaluating model performance.
• Conclusion:
  • Charles successfully transformed a non-deployable notebook into a secure, deployable web application.
  • The collaboration resulted in a streamlined process for deploying Lang Chain applications with secure code execution.
  • The project demonstrated the ease of deployment and potential for future applications.