diff --git a/.gitignore b/.gitignore index 96ef2f4..5191c3f 100644 --- a/.gitignore +++ b/.gitignore @@ -3,6 +3,7 @@ files/ outputs/ test_venv/ backup*/ +demo/ # Python cache __pycache__/ diff --git a/README.md b/README.md index 946dfde..225b4a4 100644 --- a/README.md +++ b/README.md @@ -20,9 +20,15 @@ CNLLM 为中文大模型提供了一个**统一的 OpenAI 兼容接口层**与 - **统一接口** - 一套接口和参数调用不同中文大模型,返回 OpenAI API 标准响应 - **参数验证** - 对所有参数进行验证和明确反馈,尤其是厂商原生参数,并支持参数处理行为控制 (`drop_params`) -- **流式响应** - 通过 `repr()` 进行流式生命周期监测,以及通过 `.still/.think/.tools` 属性访问增量值自动累积 +- **流式响应** - 通过 `repr()` 进行流式生命周期监测,以及通过 `.still/.think/.tools` 属性访问增量字段自动累积 - **批量能力** - 支持批量任务中单个请求的独立配置、实时批量进度统计 (`.status`),以及可配置的失败策略 (`stop_on_error`) 和内存管理 (`keep`). +**流式生命周期监控以及模型回复、思考内容、工具调用的自动累积演示:** + +![Figure 2][repr] + +[repr]: pics/repr.gif + ### 开发者招募 欢迎开发者共同参与 CNLLM 的发展,创建 Pull Request 前请先提交 Issue 说明问题并讨论您的解决方案。 @@ -267,7 +273,6 @@ for chunk in resp: # {'id': '...', 'object': '...', 'created': '...', 'model': '...', 'choices': [{'delta': {'content': '实时累积的模型回复', 'reasoning_content': '实时累积的推理过程'}, 'finish_reason': 'None'}]} ``` - ### 2.2 chat completions 批量调用 可通过`prompt`和`messages`参数输入并快速配置全局参数,也可以通过`requests`参数为单个请求进行独立配置。 diff --git a/README_en.md b/README_en.md index a194ba0..83a66ba 100644 --- a/README_en.md +++ b/README_en.md @@ -20,9 +20,15 @@ Through CNLLM, developers can seamlessly use Chinese LLMs in the OpenAI ecosyste - **Unified Interface** - One set of interfaces and parameters to call different Chinese LLMs, returns OpenAI API standard format response - **Parameter Validation** - Validation and explicit feedback for all parameters, especially vendor native parameters, with support for parameter handling behavior control (`drop_params`) -- **Streaming Response** - Streaming lifecycle monitoring via `repr()`, and automatic accumulation of incremental values via `.still`/`.think`/`.tools` property access +- **Streaming Response** - Streaming lifecycle monitoring via `repr()`, and automatic accumulation of incremental fields via `.still`/`.think`/`.tools` property access - **Batch Capability** - Independent configuration for single requests in batch tasks, real-time batch progress statistics (`.status`), and configurable failure policy (`stop_on_error`) and memory management (`keep`). +**Streaming lifecycle monitoring and automatic accumulation demonstration for model responses, reasoning content, and tool calls:** + +![Figure 2][repr] + +[repr]: docs/pics/repr.gif + ### Collaboration Opportunities Welcome developers to participate in CNLLM's development. Please submit an Issue to discuss your solution before creating a Pull Request. diff --git a/pics/repr.gif b/pics/repr.gif new file mode 100644 index 0000000..07201b4 Binary files /dev/null and b/pics/repr.gif differ