feat: improve skill scores for agent-skills

skills/chdb-datastore/SKILL.md

@@ -1,17 +1,6 @@
 ---
 name: chdb-datastore
-description: >-
-  Drop-in pandas replacement with ClickHouse performance. Use
-  `import chdb.datastore as pd` (or `from datastore import DataStore`)
-  and write standard pandas code — same API, 10-100x faster on large
-  datasets. Supports 16+ data sources (MySQL, PostgreSQL, S3, MongoDB,
-  ClickHouse, Iceberg, Delta Lake, etc.) and 10+ file formats (Parquet,
-  CSV, JSON, Arrow, ORC, etc.) with cross-source joins. Use this skill
-  when the user wants to analyze data with pandas-style syntax, speed
-  up slow pandas code, query remote databases or cloud storage as
-  DataFrames, or join data across different sources — even if they
-  don't explicitly mention chdb or DataStore. Do NOT use for raw SQL
-  queries, ClickHouse server administration, or non-Python languages.
+description: "Drop-in pandas replacement with ClickHouse performance. Use `import chdb.datastore as pd` (or `from datastore import DataStore`) and write standard pandas code — same API, 10-100x faster on large datasets. Supports 16+ data sources (MySQL, PostgreSQL, S3, MongoDB, ClickHouse, Iceberg, Delta Lake, etc.) and 10+ file formats (Parquet, CSV, JSON, Arrow, ORC, etc.) with cross-source joins. Use this skill when the user wants to analyze data with pandas-style syntax, speed up slow pandas code, query remote databases or cloud storage as DataFrames, or join data across different sources — even if they don't explicitly mention chdb or DataStore. Do NOT use for raw SQL queries, ClickHouse server administration, or non-Python languages."
 license: Apache-2.0
 compatibility: Requires Python 3.9+, macOS or Linux. pip install chdb.
 metadata:
@@ -107,6 +96,8 @@ result = (orders
     .agg({"amount": "sum", "rating": "mean"})
     .sort_values("sum", ascending=False))
 print(result)
+# Debug: inspect the generated SQL if results look unexpected
+print(result.to_sql())
 ```
 
 More join examples → [examples.md](examples/examples.md)
@@ -120,6 +111,11 @@ target = DataStore("file", path="summary.parquet", format="Parquet")
 target.insert_into("category", "total", "count").select_from(
     source.groupby("category").select("category", "sum(amount) AS total", "count() AS count")
 ).execute()
+
+# Verify the write succeeded
+result = DataStore.from_file("summary.parquet")
+print(f"Written {len(result)} rows")
+print(result.head(5))
 ```
 
 ## Troubleshooting

skills/chdb-sql/SKILL.md

@@ -1,16 +1,6 @@
 ---
 name: chdb-sql
-description: >-
-  In-process ClickHouse SQL engine for Python — run ClickHouse SQL queries
-  directly on local files, remote databases, and cloud storage without a
-  server. Use when the user wants to write SQL queries against Parquet/CSV/
-  JSON files, use ClickHouse table functions (mysql(), s3(), postgresql(),
-  iceberg(), deltaLake() etc.), build stateful analytical pipelines with
-  Session, use parametrized queries, window functions, or other advanced
-  ClickHouse SQL features. Also use when the user explicitly mentions
-  chdb.query(), ClickHouse SQL syntax, or wants cross-source SQL joins.
-  Do NOT use for pandas-style DataFrame operations — use chdb-datastore
-  instead.
+description: "In-process ClickHouse SQL engine for Python — run ClickHouse SQL queries directly on local files, remote databases, and cloud storage without a server. Use when the user wants to write SQL queries against Parquet/CSV/JSON files, use ClickHouse table functions (mysql(), s3(), postgresql(), iceberg(), deltaLake() etc.), build stateful analytical pipelines with Session, use parametrized queries, window functions, or other advanced ClickHouse SQL features. Also use when the user explicitly mentions chdb.query(), ClickHouse SQL syntax, or wants cross-source SQL joins. Do NOT use for pandas-style DataFrame operations — use chdb-datastore instead."
 license: Apache-2.0
 compatibility: Requires Python 3.9+, macOS or Linux. pip install chdb.
 metadata:
@@ -69,6 +59,11 @@ sess = chs.Session("./analytics_db")   # persistent; Session() for in-memory
 
 sess.query("CREATE TABLE users ENGINE=MergeTree() ORDER BY id AS SELECT * FROM mysql('db:3306','crm','users','root','pass')")
 sess.query("CREATE TABLE events ENGINE=MergeTree() ORDER BY (ts,user_id) AS SELECT * FROM s3('s3://logs/events/*.parquet',NOSIGN)")
+
+# Verify tables were created and data loaded
+sess.query("SELECT count() FROM users", "Pretty").show()
+sess.query("SELECT count() FROM events", "Pretty").show()
+
 sess.query("""
     SELECT u.country, count() AS cnt, uniqExact(e.user_id) AS users
     FROM events e JOIN users u ON e.user_id = u.id

skills/clickhouse-architecture-advisor/SKILL.md

@@ -1,6 +1,6 @@
 ---
 name: clickhouse-architecture-advisor
-description: MUST USE when designing ClickHouse architectures, selecting between ingestion or modeling patterns, or translating best practices into workload-specific system designs. Complements clickhouse-best-practices with decision frameworks and explicit provenance labels.
+description: "Provides workload-aware architecture decisioning for ClickHouse — selects table engines, designs partition strategies, chooses ingestion patterns (Kafka vs batch vs async inserts), evaluates ReplacingMergeTree vs AggregatingMergeTree trade-offs, and plans data modeling for specific workloads (observability, SIEM, product analytics, IoT, financial services). Use when designing a ClickHouse architecture, selecting between ingestion or modeling patterns, planning schema design, choosing MergeTree variants, building a data pipeline, or evaluating OLAP trade-offs. Complements clickhouse-best-practices (implementation-level rule checks) with higher-level decision frameworks and provenance-labeled recommendations."
 license: Apache-2.0
 metadata:
   author: ClickHouse Inc
@@ -33,21 +33,17 @@ Before producing recommendations:
    - `field`
 5. Never present field guidance as official guidance
 6. If a recommendation is uncertain, say so explicitly
+7. Verify each recommendation's provenance by confirming the linked documentation actually supports the claim before presenting
 
 ## Provenance rules
 
-### `official`
-Use this when the recommendation is directly backed by official docs.
+Classify every recommendation with one of these labels:
 
-### `derived`
-Use this when the recommendation is not stated verbatim in docs but follows logically from documented ClickHouse behavior.
+- **`official`** — directly backed by official ClickHouse documentation.
+- **`derived`** — follows logically from documented behavior but is not stated verbatim.
+- **`field`** — experience-based, situational guidance. When using this label, include a disclaimer that the advice is heuristic, link a partially relevant official doc, and explain why the advice depends on workload context.
 
-### `field`
-Use this only for experience-based guidance that may be situational.
-When using `field`, include:
-- a disclaimer that the advice is heuristic
-- a relevant official doc if one partially applies
-- the reason the advice depends on workload context
+Never present `field` guidance as official guidance. If uncertain, say so explicitly.
 
 ## Read these rule files by scenario
 
@@ -110,6 +106,22 @@ high | medium | heuristic
 - concrete SQL, metric, or smoke test
 ```
 
+## Example: Observability Ingestion Decision
+
+```sql
+-- Validate ingestion throughput after choosing async inserts
+SELECT
+    table,
+    formatReadableSize(sum(bytes_on_disk)) AS size,
+    sum(rows) AS total_rows,
+    count() AS part_count
+FROM system.parts
+WHERE active AND table = 'otel_traces'
+GROUP BY table;
+```
+
+Use queries like this to validate that the chosen architecture meets throughput and storage targets. See `examples/` for full worked examples across workload types.
+
 ## Architecture-specific guidance
 
 Prefer decision frameworks over generic advice. Good responses should:

skills/clickhouse-best-practices/SKILL.md

@@ -1,6 +1,6 @@
 ---
 name: clickhouse-best-practices
-description: MUST USE when reviewing ClickHouse schemas, queries, or configurations. Contains 28 rules that MUST be checked before providing recommendations. Always read relevant rule files and cite specific rules in responses.
+description: "Validates ClickHouse table schemas, optimizes queries, and audits data ingestion patterns using 28 prioritized rules covering primary key design, data type selection, JOIN optimization, partition strategy, materialized views, and mutation avoidance. Use when reviewing CREATE TABLE statements, optimizing ClickHouse queries, selecting MergeTree engine variants, designing ORDER BY keys, tuning JOIN performance, choosing partition strategies, or applying ClickHouse best practices. Read relevant rule files from rules/ and cite specific rules in responses."
 license: Apache-2.0
 metadata:
   author: ClickHouse Inc
@@ -15,7 +15,7 @@ Comprehensive guidance for ClickHouse covering schema design, query optimization
 
 ## IMPORTANT: How to Apply This Skill
 
-**Before answering ClickHouse questions, follow this priority order:**
+**When answering ClickHouse questions, follow this priority order:**
 
 1. **Check for applicable rules** in the `rules/` directory
 2. **If rules exist:** Apply them and cite them in your response using "Per `rule-name`..."
@@ -133,100 +133,6 @@ Structure your response as follows:
 
 ---
 
-## Quick Reference
-
-### Schema Design - Primary Key (CRITICAL)
-
-- `schema-pk-plan-before-creation` - Plan ORDER BY before table creation (immutable)
-- `schema-pk-cardinality-order` - Order columns low-to-high cardinality
-- `schema-pk-prioritize-filters` - Include frequently filtered columns
-- `schema-pk-filter-on-orderby` - Query filters must use ORDER BY prefix
-
-### Schema Design - Data Types (CRITICAL)
-
-- `schema-types-native-types` - Use native types, not String for everything
-- `schema-types-minimize-bitwidth` - Use smallest numeric type that fits
-- `schema-types-lowcardinality` - LowCardinality for <10K unique strings
-- `schema-types-enum` - Enum for finite value sets with validation
-- `schema-types-avoid-nullable` - Avoid Nullable; use DEFAULT instead
-
-### Schema Design - Partitioning (HIGH)
-
-- `schema-partition-low-cardinality` - Keep partition count 100-1,000
-- `schema-partition-lifecycle` - Use partitioning for data lifecycle, not queries
-- `schema-partition-query-tradeoffs` - Understand partition pruning trade-offs
-- `schema-partition-start-without` - Consider starting without partitioning
-
-### Schema Design - JSON (MEDIUM)
-
-- `schema-json-when-to-use` - JSON for dynamic schemas; typed columns for known
-
-### Query Optimization - JOINs (CRITICAL)
-
-- `query-join-choose-algorithm` - Select algorithm based on table sizes
-- `query-join-use-any` - ANY JOIN when only one match needed
-- `query-join-filter-before` - Filter tables before joining
-- `query-join-consider-alternatives` - Dictionaries/denormalization vs JOIN
-- `query-join-null-handling` - join_use_nulls=0 for default values
-
-### Query Optimization - Indices (HIGH)
-
-- `query-index-skipping-indices` - Skipping indices for non-ORDER BY filters
-
-### Query Optimization - Materialized Views (HIGH)
-
-- `query-mv-incremental` - Incremental MVs for real-time aggregations
-- `query-mv-refreshable` - Refreshable MVs for complex joins
-
-### Insert Strategy - Batching (CRITICAL)
-
-- `insert-batch-size` - Batch 10K-100K rows per INSERT
-
-### Insert Strategy - Async (HIGH)
-
-- `insert-async-small-batches` - Async inserts for high-frequency small batches
-- `insert-format-native` - Native format for best performance
-
-### Insert Strategy - Mutations (CRITICAL)
-
-- `insert-mutation-avoid-update` - ReplacingMergeTree instead of ALTER UPDATE
-- `insert-mutation-avoid-delete` - Lightweight DELETE or DROP PARTITION
-
-### Insert Strategy - Optimization (HIGH)
-
-- `insert-optimize-avoid-final` - Let background merges work
-
----
-
-## When to Apply
-
-This skill activates when you encounter:
-
-- `CREATE TABLE` statements
-- `ALTER TABLE` modifications
-- `ORDER BY` or `PRIMARY KEY` discussions
-- Data type selection questions
-- Slow query troubleshooting
-- JOIN optimization requests
-- Data ingestion pipeline design
-- Update/delete strategy questions
-- ReplacingMergeTree or other specialized engine usage
-- Partitioning strategy decisions
-
----
-
-## Rule File Structure
-
-Each rule file in `rules/` contains:
-
-- **YAML frontmatter**: title, impact level, tags
-- **Brief explanation**: Why this rule matters
-- **Incorrect example**: Anti-pattern with explanation
-- **Correct example**: Best practice with explanation
-- **Additional context**: Trade-offs, when to apply, references
-
----
-
 ## Full Compiled Document
 
 For the complete guide with all rules expanded inline: `AGENTS.md`

skills/clickhousectl-cloud-deploy/SKILL.md

@@ -1,6 +1,6 @@
 ---
 name: clickhousectl-cloud-deploy
-description: Use when a user wants to deploy ClickHouse to the cloud, go to production, use ClickHouse Cloud, host a managed ClickHouse service, or migrate from a local ClickHouse setup to ClickHouse Cloud.
+description: "Guides deployment to ClickHouse Cloud using clickhousectl — covers CLI authentication (browser OAuth and API key), cloud service creation, schema migration from local development, deployment verification, and application connection configuration. Use when the user wants to deploy ClickHouse to the cloud, go to production, use ClickHouse Cloud, host a managed ClickHouse service, or migrate from a local ClickHouse setup to ClickHouse Cloud."
 license: Apache-2.0
 metadata:
   author: ClickHouse Inc
@@ -11,17 +11,6 @@ metadata:
 
 This skill walks through deploying to ClickHouse Cloud using `clickhousectl`. It covers account setup, CLI authentication, service creation, schema migration, and connecting your application. Follow these steps in order.
 
-## When to Apply
-
-Use this skill when the user wants to:
-- Deploy their ClickHouse application to production
-- Host ClickHouse as a managed cloud service
-- Migrate from a local ClickHouse setup to ClickHouse Cloud
-- Create a ClickHouse Cloud service
-- Set up ClickHouse Cloud for the first time
-
----
-
 ## Step 1: Sign up for ClickHouse Cloud
 
 Before using any cloud commands, the user needs a ClickHouse Cloud account.
@@ -30,11 +19,7 @@ Before using any cloud commands, the user needs a ClickHouse Cloud account.
 
 **If they do not have an account**, explain:
 
-> ClickHouse Cloud is a fully managed service that runs ClickHouse for you — no infrastructure to maintain, automatic scaling, backups, and upgrades included. There's a free trial so you can get started without a credit card.
->
-> To create an account, go to: **https://clickhouse.cloud**
->
-> Sign up with your email, Google, or GitHub account. Once you're in the console, let me know and we'll continue with the next step.
+> To create an account, go to **https://clickhouse.cloud** and sign up with email, Google, or GitHub. There's a free trial — no credit card needed.
 
 **Wait for the user to confirm** they have signed up or already have an account before proceeding.
 
@@ -82,13 +67,7 @@ clickhousectl cloud login --api-key <key> --api-secret <secret>
 
 If the user doesn't have API keys yet, guide them to create one:
 
-> In the ClickHouse Cloud console:
-> 1. Click the **gear icon** (Settings) in the left sidebar
-> 2. Go to **API Keys**
-> 3. Click **Create API Key**
-> 4. Give it a name (e.g., "cli") and select the **Admin** role
-> 5. Click **Generate API Key**
-> 6. **Copy both the Key ID and the Key Secret** — the secret is only shown once
+> In the ClickHouse Cloud console, go to **Settings → API Keys → Create API Key** (Admin role). Copy both the Key ID and Key Secret — the secret is only shown once.
 
 ---
 
@@ -181,9 +160,8 @@ Provide the user with the connection details:
 - **Password:** the password from step 3 (service creation)
 - **SSL/TLS:** required (always enabled on Cloud)
 
-**Example connection strings** (adapt to the user's language/framework):
+**Example connection** (adapt to the user's language/framework):
 
-**Python (clickhouse-connect):**
 ```python
 import clickhouse_connect
 
@@ -196,29 +174,6 @@ client = clickhouse_connect.get_client(
 )
 ```
 
-**Node.js (@clickhouse/client):**
-```javascript
-import { createClient } from '@clickhouse/client'
-
-const client = createClient({
-  url: 'https://<cloud-host>:8443',
-  username: 'default',
-  password: '<password>',
-})
-```
-
-**Go (clickhouse-go):**
-```go
-conn, err := clickhouse.Open(&clickhouse.Options{
-    Addr: []string{"<cloud-host>:9440"},
-    Auth: clickhouse.Auth{
-        Username: "default",
-        Password: "<password>",
-    },
-    TLS: &tls.Config{},
-})
-```
-
 Suggest the user store the password in an environment variable or secrets manager rather than hardcoding it.
 
 Suggest the user should not use the default user in production. A user should be created just for their app.