Create Phi-3 Chatbot with 20 lines of code (runs without wifi)

---

---

AI Nuggets

Instructions and Tips Extracted from the Video Transcript

Setting Up the Chatbot Environment

• Turn off Wi-Fi to demonstrate running in an air-gapped environment.
• Ensure everything is running on the local machine, including the backend and the model.
• Use a local host in the Chrome browser for the frontend.

Building the Chatbot

• The chatbot is built with a local model in approximately 20 lines of code.
• The UI is simple and not very interactive but functional.
• The chatbot uses the 53 model (GPT-3 variant) running locally.

Code Overview

• The code can be found in the LM (Language Model) repository under examples in the UI subfolder.
• Import Streamlit for the UI (pip install streamlit).
• The UI elements are derived from the Streamlit tutorial on building a chat UI. The link is provided in the video description.
• Configure the max output tokens for the model.
• Load the model from the LLMware catalog, with recommended chat-oriented models listed at the end of the video.
• Set parameters like temperature and sampling for a more interactive experience.
• The core UI element is handled by Streamlit, which manages the interaction between the user and the assistant.
• The key line of code uses a generator provided by LLMware, which Streamlit consumes to create the interactive streaming output.

Running the Streamlit Application

• Navigate to the root of your folder path.
• Run the Streamlit application with the command: streamlit run [your_file_name].py

Recommended Models for Chatbot

• Llama 2 chat GGF
• Llama 3 A GGF (instruct quantized version)
• Mraw models: OpenAI Mraw and Hugging Face Zephyr (both are 7 billion parameter models)
• Tiny Llama chat GGF (1 billion parameter model, quantized to a 670 MB binary)

Running the Chatbot

• Place the file you want to run in the root path of your folder directory.
• Use the command: streamlit run [your_file_name].py to start the chatbot.

Additional Resources

• Check out the example in the LLMware repository.
• For questions, comments, or issues, join the discussion on Discord.

Tips for Creating a YouTube Demo Tutorial

• Identify your target audience.
• Choose an interesting topic or skill.
• Plan your script before recording.
• Set up your camera properly.
• Edit your video for clarity and engagement.

Tips for Developing a Business Plan

• Seek insights and tips for getting a startup off the ground.

Tips for Basketball Offense Strategy

• Consider player positioning and movement.
• Focus on ball distribution.
• Utilize pick and roll and isolation plays.
• Implement off-ball actions and transition game.
• Use high low post plays.

Tips for Developing a Chatbot UI with HTML and CSS

• Start with basic HTML and CSS to create the chatbot UI.

Final Thoughts

• The chatbot is designed to be a simple framework that can be expanded with more interactive elements and rich interactions using LLMware.

URLs and Commands Mentioned

• Streamlit tutorial link (provided in the video description).
• Streamlit installation command: pip install streamlit
• Streamlit run command: streamlit run [your_file_name].py

(Note: The exact URLs for the Streamlit tutorial and the LLMware repository are not provided in the transcript. They should be available in the video description, which is not included in the provided text.)