From bc6d3d9173a91b004014d731655b51b0c4f70ead Mon Sep 17 00:00:00 2001 From: LiZhenCheng9527 Date: Tue, 28 Apr 2026 20:04:40 +0800 Subject: [PATCH 1/2] add [lfx-mentorship-2026-June-August] Kthena LFX project declaration Signed-off-by: LiZhenCheng9527 --- .../lfx-mentorship/2026/02-Jun-Aug/project_ideas.md | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/programs/lfx-mentorship/2026/02-Jun-Aug/project_ideas.md b/programs/lfx-mentorship/2026/02-Jun-Aug/project_ideas.md index c4f273ec..6030b308 100644 --- a/programs/lfx-mentorship/2026/02-Jun-Aug/project_ideas.md +++ b/programs/lfx-mentorship/2026/02-Jun-Aug/project_ideas.md @@ -112,6 +112,19 @@ Alongside this, the mentee will improve documentation experience for contributor - Andy Anderson (@clubanderson, andy@clubanderson.com) - Upstream Issue: https://github.com/kubestellar/console/issues/4196 +### volcano/kthena + +#### Kthena Router supports APIs for third-party models + +- Description: At present, the Kthena router provides robust support for routing within the cluster. However, during discussions with customers, some users have expressed a need for the ability to integrate with external large language model APIs. Consequently, the Kthena community plans to use this LFX to enable the router to access third-party LLM APIs. +- Expected OUtcome: + 1. Proposal + 2. Code(Include unit test. It would be even better if end-to-end testing were included. However, external LLM APIs are unlikely to offer stable, long-term free access, so end-to-end testing is not a prerequisite.) + 3. User Guide and related documents. +- Recommended: Go, Kubernetes, Network, LLM. +- Mentor(s): Zengzeng Yao(@yaozengzeng, yaozengzeng@huawei.com) +- Upstream Issue: https://github.com/volcano-sh/kthena/issues/939 + ### Lima #### Improve Windows support (host and guest) From a548208f5c99b4c1e669b9a2d0704c0e06c14778 Mon Sep 17 00:00:00 2001 From: LiZhenCheng9527 Date: Wed, 29 Apr 2026 10:30:36 +0800 Subject: [PATCH 2/2] add kthena Router Benchmark project proposal Signed-off-by: LiZhenCheng9527 --- .../2026/02-Jun-Aug/project_ideas.md | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/programs/lfx-mentorship/2026/02-Jun-Aug/project_ideas.md b/programs/lfx-mentorship/2026/02-Jun-Aug/project_ideas.md index 6030b308..7b28513a 100644 --- a/programs/lfx-mentorship/2026/02-Jun-Aug/project_ideas.md +++ b/programs/lfx-mentorship/2026/02-Jun-Aug/project_ideas.md @@ -125,6 +125,21 @@ Alongside this, the mentee will improve documentation experience for contributor - Mentor(s): Zengzeng Yao(@yaozengzeng, yaozengzeng@huawei.com) - Upstream Issue: https://github.com/volcano-sh/kthena/issues/939 +#### Kthena Router Benchmark + +- Description: Kthena Router is the LLM routing component in the Volcano/Kthena project, responsible for forwarding inference requests to in-cluster and (soon) third-party model backends. As Kthena gradually matures, the community needs a reproducible benchmark to understand the router's performance characteristics (throughput, latency, tail latency, resource usage) under realistic LLM traffic patterns, and to detect regressions across releases. This project asks the mentee to design and implement a benchmark framework for Kthena Router, produce a complete report. Where the benchmark surfaces clear bottlenecks or inefficiencies, the mentee will also work with maintainers to land code-level optimizations upstream. +- Expected Outcome: + - A reusable benchmark framework for Kthena Router (load generator, scenario configs, metrics collection, result aggregation), runnable locally and in CI. + - A defined set of test scenarios covering typical LLM routing patterns (varying QPS, prompt/response sizes, concurrency, number of backends, routing strategies). + - End-to-end test procedure documented as a runbook (cluster setup, mock/real backends, how to run, how to interpret results). + - A benchmark report with metrics (throughput, TPOT, TTFT, GPU/CPU/memory) and analysis of bottlenecks. + - Upstream PR(s) implementing optimizations for any clear hotspots identified, with before/after numbers. +- Recommended Skills: Go, Kubernetes, performance benchmarking and profiling (pprof), basic familiarity with LLM inference and HTTP/gRPC routing. +- Mentor(s): + - Zengzeng Yao (@yaozengzeng, yaozengzeng@huawei.com), + - Zhonghu Xu (@hzxuzhonghu, zhhxu2011@gmail.com), +- Upstream Issue: https://github.com/volcano-sh/kthena/issues/942 + ### Lima #### Improve Windows support (host and guest)