Skip to content

Server Offline with either llama.cpp or HF-Waitress #30

Description

@Rotoslider

Ubuntu 22.04, Installed LARS today using instruction from your github readme. Using Conda environment.
llama-server --version version: 3912 (edc26566)
built with cc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0 for x86_64-linux-gnu
added to path in barshrc
model is in model directory.
if I run llama-server -m /home/asus/builds/LARS/lars_storage/models/dolphin-2.9.3-mistral-7B-32k-Q8_0.gguf -c 2048 from any where the server starts and I can chat with it.
main: server is listening on 127.0.0.1:8080 - starting the main loop
srv update_slots: all slots are idle

config.json
lars_server_log.log
llama_cpp_server_output_log.txt
The hf_waitress output log is blank and no json was created for it

What else do i need to give you to help troubleshoot?

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions