Skip to content

deployment_guide.md

Latest commit

 

History

History
94 lines (79 loc) · 3.44 KB

deployment_guide.md

File metadata and controls

94 lines (79 loc) · 3.44 KB
Raw

Deployment Guide - Class Intelligence System (CIS)

1. Local LAN Deployment (Best for Class Demo)

To let other students/faculty access your local CIS instance on the same Wi-Fi network:

Backend

  1. Bind to 0.0.0.0: Run uvicorn with host 0.0.0.0. 
    uvicorn app.main:app --host 0.0.0.0 --port 8000
  2. Firewall: Ensure port 8000 is open on your firewall.
  3. Find IP: Run ip addr (Linux) or ipconfig (Windows) to find your local IP (e.g., 192.168.1.15).

Frontend

  1. Update API URL: In frontend/.env (or config), change localhost to your IP: 
    NEXT_PUBLIC_API_URL="http://192.168.1.15:8000"
  2. Run with Network Access: 
    npm run dev -- -H 0.0.0.0
    Or build for production: npm run build && npm start.

Local AI Service (BitNet/Ollama)

  • Must also run on 0.0.0.0 if accessing from another backend machine.
  • Recommended Command (Run this in your llama.cpp directory): 
    ./build/bin/llama-server \
  -m models/BitNet-b1.58-2B-4T/ggml-model-i2_s.gguf \
  -c 2048 \
  --host 0.0.0.0 \
  --port 8080 \
  -t 8

2. Cloud Deployment (Render/Railway)

Note: Cloud deployment generally requires Cloud LLMs (Gemini/OpenAI) because standard free-tier servers (Render) cannot run local LLMs like BitNet.

Steps

  1. Dockerize: Ensure Dockerfile is present in backend root.
  2. Push to GitHub: Commit your changes.
  3. Render Blueprint:
    • Create a Web Service for Backend (python 3.10).
    • Build Command: pip install -r requirements.txt.
    • Start Command: uvicorn app.main:app --host 0.0.0.0 --port $PORT.
    • Env Vars: AI_PROVIDER=google, GOOGLE_API_KEY=....
  4. Static Site for Frontend:
    • Build Command: npm run build.
    • Publish directory: .next or out.

3. Hybrid Mode (Local AI + Cloud Frontend)

  • Run Backend + AI locally (Ngrok tunnel).
  • Frontend on Vercel/Netlify pointing to Ngrok URL.
    • ngrok http 8000 -> gives public URL.

4. Google Colab Deployment (Free GPU)

This is the best way to run BitNet/Llama with high speed if you don't have a GPU locally.

Steps

  1. Download Notebook: Download cis_colab.ipynb from this project.
  2. Upload to Colab: Go to colab.research.google.com -> Upload.
  3. Select GPU Runtime (CRITICAL):
    • In Colab, go to top menu: Runtime -> Change runtime type.
    • Select T4 GPU.
    • Click Save.
    • If you don't do this, the notebook will fail with nvidia-smi not found.
  4. Upload Backend:
    • Zip your backend folder.
    • Upload it to the Colab files section (left sidebar).
    • Unzip it: !unzip backend.zip.
  5. Ngrok Token:
    • Sign up at ngrok.com.
    • Copy your Authtoken.
    • Paste it in the notebook code where it says YOUR_NGROK_AUTH_TOKEN.
  6. Run All Cells:
    • The notebook will install everything, download the model, and start the server.
    • Note on Build: The notebook is now optimized using Ninja and ccache. It builds only the llama-server target to significantly reduce installation time.
    • It will print a Public URL (e.g., https://xyz.ngrok-free.app).
  7. Connect Frontend:
    • Update your local frontend/.env: 
      NEXT_PUBLIC_API_URL="https://xyz.ngrok-free.app"
    • Run your frontend locally: npm run dev.