setting up the backend

This document will outline how to setup and run the Mourshid backend

Prerequisites

1. A machine with access to the internet running a Ubuntu Server or a similar linux distribution
2. The machine must have at lest 8 gigabytes of RAM

1. installing required software

1. The backend is written in spring boot and so JRE must be installed, it can be installed on linux using the following command

   sudo apt install default-jdk

2. LLM inference is done using the llama.cpp-server python binding so installing python is required, install it by running the following command

   sudo apt install python3

3. Data is persisted in the backend using a MySQL, in case you will be hosting the DB locally , install MySQL server on the local machine and start it as a service

   sudo apt install mysql-server
   #check if the service is up
   sudo systemctl status mysql
   #if the service is down start it
   sudo systemctl start mysql

4. to avoid requiring owning a domain and DNS hosting, a trial cloudflare tunnel can be used to access the backend remotely . Install cloudflare daemon using the command

   #Add Cloudflare's GPG key
   sudo mkdir -p --mode=0755 /usr/share/keyrings
   curl -fsSL https://pkg.cloudflare.com/cloudflare-main.gpg | sudo tee /usr/share/keyrings/cloudflare-main.gpg >/dev/null
   
   #Add the official Cloudflare repository to your sources list
   echo "deb [signed-by=/usr/share/keyrings/cloudflare-main.gpg] https://pkg.cloudflare.com/cloudflared $(lsb_release -cs) main" | sudo tee /etc/apt/sources.list.d/cloudflared.list
   
   #Update apt indexes and install cloudflared
   sudo apt-get update && sudo apt-get install cloudflared

2. running the spring backend

1. clone the backend code

   git clone https://github.com/Mourshid/backend.git

2. create in your mysql db (whether local or remote) a database to be used by the backend you can call it imourshid-db

3. create in the root directory of the backend (same level as the src directory) a .env file with file as follows

   DB_USER=<user name of your db>
   DB_PASSWORD=<pass word of your db>
   DB_URL=<url to local or remote db>
   JWT_SECRET=<64 char secret key for signing jwt>
   JWT_DURATION_MS=3600000
   JWT_REFRESH_DURATION_MS = 604800000
   GOAL_DECOMPOSER_URL = http://localhost:8000/v1/chat/completions
   

   the DB_URL will usually start with jdbc:mysql://

4. Run the backend using

   ./mvnw spring-boot:run

3. Running the LLM inference server

1. Create a directory to store LLM related files, you can call it llm

2. Go to the llm directory and create a python virtual environment, you can call it venv

   python -m venv venv

3. Activate the virtual environment

   source venv/bin/activate

4. Install llama.cpp-server python binding using pip

   pip install llama-cpp-python

5. download the fine-tuned and quantized model from this Hugging Face Repository

6. create a configuration file with the following content

   {
       "host": "localhost",
       "port": 8000,
       "models":[
           {
               "model":"./v2/phi-4-tuned-v2-Q8_0.gguf",
               "n_threads":4,
               "n_threads_batch":4,
               "n_gpu_layers": -1,
               "verbose": true
           }
       ]
   }

   the model field should be the path to the model downloaded in step 5 relative to where the llm directory created in step 1 is

7. Run the inference server

   python -m llama_cpp.server --config_file ./server_config.json

4. Allowing access through the internet

1. to avoid needing to own a domain and to have DNS hosting cloudflare tunnel could be used as temporary experimental domains to allow access to the server using the public internet. create a trial cloudflare tunnel and point it at the spring backend as follows

   cloudflared tunnel --url http://localhost:8080