README.md

CompCon API Servers

Design Choices

1. All LLMs/VLMs/CLIPs serve as API with cache enabled, because loading a LLM/VLM/CLIP is expensive and we never modify them.
2. LLM functions in utils_llm.py, VLM functions in utils_vlm.py, CLIP functions in utils_clip.py, and others in utils_general.py.
3. Write unit tests to understand major functions.

LLM / VLMServer Configuration

1. Set up OpenAI API key: export OPENAI_API_KEY='[your key]'
2. Pip install environments: pip install vllm
3. Configure global variables in global_vars.py
4. Run python -m vllm.entrypoints.openai.api_server --model HuggingFaceM4/Idefics3-8B-Llama3 --port 8080 --max_model_len 5000
5. Run python -m serve.utils_llm or python -m serve.utils_vlm to test.

CLIP Server Configuration

1. Pip install environments: pip install open-clip-torch flask
2. Configure global variables in global_vars.py
3. Run python serve/clip_server.py
4. Run python -m serve.utils_clip to test.