Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 32 additions & 2 deletions programs/lfx-mentorship/2026/02-Jun-Aug/project_ideas.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,9 +18,11 @@

## Proposed Project ideas

### Volcano/AgentCube
### Volcano

#### Support multi-AgentCube Capability
#### AgentCube

##### Support multi-AgentCube Capability

- Description: At present, AgentCube launches a single agent for both Code Interpretation and agent runtime. However, we have now entered the era of multi-agent collaboration. Consequently, AgentCube aims to support multi-AgentCube orchestration, enabling multiple agents to collaborate on a single task, with AgentCube providing unified management of the lifecycle of these agents.
- Expected Outcome:
Expand All @@ -33,3 +35,31 @@
- Zhonghu xu(@hzxuzhonghu, zhhxu2011@gmail.com),
- ZhenCheng Li(@LiZhenCheng9527, lizhencheng6@huawei.com),
- Upstream Issue: https://github.com/volcano-sh/agentcube/issues/301

#### kthena

##### Kthena Router supports APIs for third-party models

- Description: At present, the Kthena router provides robust support for routing within the cluster. However, during discussions with customers, some users have expressed a need for the ability to integrate with external large language model APIs. Consequently, the Kthena community plans to use this LFX to enable the router to access third-party LLM APIs.
- Expected OUtcome:
1. Proposal
2. Code(Include unit test. It would be even better if end-to-end testing were included. However, external LLM APIs are unlikely to offer stable, long-term free access, so end-to-end testing is not a prerequisite.)
3. User Guide and related documents.
- Recommended: Go, Kubernetes, Network, LLM.
- Mentor(s): Zengzeng Yao(@yaozengzeng, yaozengzeng@huawei.com)
- Upstream Issue: https://github.com/volcano-sh/kthena/issues/939

##### Kthena Router Benchmark

- Description: Kthena Router is the LLM routing component in the Volcano/Kthena project, responsible for forwarding inference requests to in-cluster and (soon) third-party model backends. As Kthena gradually matures, the community needs a reproducible benchmark to understand the router's performance characteristics (throughput, latency, tail latency, resource usage) under realistic LLM traffic patterns, and to detect regressions across releases. This project asks the mentee to design and implement a benchmark framework for Kthena Router, produce a complete report. Where the benchmark surfaces clear bottlenecks or inefficiencies, the mentee will also work with maintainers to land code-level optimizations upstream.
- Expected Outcome:
- A reusable benchmark framework for Kthena Router (load generator, scenario configs, metrics collection, result aggregation), runnable locally and in CI.
- A defined set of test scenarios covering typical LLM routing patterns (varying QPS, prompt/response sizes, concurrency, number of backends, routing strategies).
- End-to-end test procedure documented as a runbook (cluster setup, mock/real backends, how to run, how to interpret results).
- A benchmark report with metrics (throughput, TPOT, TTFT, GPU/CPU/memory) and analysis of bottlenecks.
- Upstream PR(s) implementing optimizations for any clear hotspots identified, with before/after numbers.
- Recommended Skills: Go, Kubernetes, performance benchmarking and profiling (pprof), basic familiarity with LLM inference and HTTP/gRPC routing.
- Mentor(s):
- Zengzeng Yao (@yaozengzeng, yaozengzeng@huawei.com),
- Zhonghu Xu (@hzxuzhonghu, zhhxu2011@gmail.com),
- Upstream Issue: https://github.com/volcano-sh/kthena/issues/942
Loading