Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ files/
outputs/
test_venv/
backup*/
demo/

# Python cache
__pycache__/
Expand Down
9 changes: 7 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,9 +20,15 @@ CNLLM 为中文大模型提供了一个**统一的 OpenAI 兼容接口层**与

- **统一接口** - 一套接口和参数调用不同中文大模型,返回 OpenAI API 标准响应
- **参数验证** - 对所有参数进行验证和明确反馈,尤其是厂商原生参数,并支持参数处理行为控制 (`drop_params`)
- **流式响应** - 通过 `repr()` 进行流式生命周期监测,以及通过 `.still/.think/.tools` 属性访问增量值自动累积
- **流式响应** - 通过 `repr()` 进行流式生命周期监测,以及通过 `.still/.think/.tools` 属性访问增量字段自动累积
- **批量能力** - 支持批量任务中单个请求的独立配置、实时批量进度统计 (`.status`),以及可配置的失败策略 (`stop_on_error`) 和内存管理 (`keep`).

**流式生命周期监控以及模型回复、思考内容、工具调用的自动累积演示:**

![Figure 2][repr]

[repr]: pics/repr.gif

### 开发者招募

欢迎开发者共同参与 CNLLM 的发展,创建 Pull Request 前请先提交 Issue 说明问题并讨论您的解决方案。
Expand Down Expand Up @@ -267,7 +273,6 @@ for chunk in resp:
# {'id': '...', 'object': '...', 'created': '...', 'model': '...', 'choices': [{'delta': {'content': '实时累积的模型回复', 'reasoning_content': '实时累积的推理过程'}, 'finish_reason': 'None'}]}
```


### 2.2 chat completions 批量调用

可通过`prompt`和`messages`参数输入并快速配置全局参数,也可以通过`requests`参数为单个请求进行独立配置。
Expand Down
8 changes: 7 additions & 1 deletion README_en.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,9 +20,15 @@ Through CNLLM, developers can seamlessly use Chinese LLMs in the OpenAI ecosyste

- **Unified Interface** - One set of interfaces and parameters to call different Chinese LLMs, returns OpenAI API standard format response
- **Parameter Validation** - Validation and explicit feedback for all parameters, especially vendor native parameters, with support for parameter handling behavior control (`drop_params`)
- **Streaming Response** - Streaming lifecycle monitoring via `repr()`, and automatic accumulation of incremental values via `.still`/`.think`/`.tools` property access
- **Streaming Response** - Streaming lifecycle monitoring via `repr()`, and automatic accumulation of incremental fields via `.still`/`.think`/`.tools` property access
- **Batch Capability** - Independent configuration for single requests in batch tasks, real-time batch progress statistics (`.status`), and configurable failure policy (`stop_on_error`) and memory management (`keep`).

**Streaming lifecycle monitoring and automatic accumulation demonstration for model responses, reasoning content, and tool calls:**

![Figure 2][repr]

[repr]: docs/pics/repr.gif

### Collaboration Opportunities

Welcome developers to participate in CNLLM's development. Please submit an Issue to discuss your solution before creating a Pull Request.
Expand Down
Binary file added pics/repr.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading