From ad2947f5764b443e8fadebeaa31501931535787b Mon Sep 17 00:00:00 2001 From: Corey Date: Thu, 24 Sep 2026 10:07:06 -0700 Subject: [PATCH] feat(growth): restore negotiation maximizer skill --- README.md | 17 +- growth/negotiation-maximizer/SKILL.md | 156 ++++++++++++++++++ .../negotiation-maximizer/agents/openai.yaml | 7 + growth/negotiation-maximizer/evals/cases.json | 45 +++++ .../references/source.md | 53 ++++++ skills/_software_maestro_manifest.json | 12 ++ skills/design-rapid-prototype-facade/SKILL.md | 89 ++++++++++ .../references/source.md | 6 + skills/design-sprint/SKILL.md | 76 +++++++++ .../SKILL.md | 14 +- .../evals/cases.json | 17 ++ .../references/source.md | 16 ++ skills/negotiation-maximizer/SKILL.md | 156 ++++++++++++++++++ .../negotiation-maximizer/agents/openai.yaml | 7 + skills/negotiation-maximizer/evals/cases.json | 45 +++++ .../references/source.md | 53 ++++++ skills/typesafe-ai/LICENSE | 21 +++ skills/typesafe-ai/SKILL.md | 149 +++++++++++++++++ skills/typesafe-jev/SKILL.md | 51 ++++++ skills/typesafe-jev/agents/openai.yaml | 4 + skills/typesafe-jev/evals/cases.json | 76 +++++++++ skills/typesafe-jev/references/jev.md | 43 +++++ 22 files changed, 1104 insertions(+), 9 deletions(-) create mode 100644 growth/negotiation-maximizer/SKILL.md create mode 100644 growth/negotiation-maximizer/agents/openai.yaml create mode 100644 growth/negotiation-maximizer/evals/cases.json create mode 100644 growth/negotiation-maximizer/references/source.md create mode 100644 skills/negotiation-maximizer/SKILL.md create mode 100644 skills/negotiation-maximizer/agents/openai.yaml create mode 100644 skills/negotiation-maximizer/evals/cases.json create mode 100644 skills/negotiation-maximizer/references/source.md create mode 100644 skills/typesafe-ai/LICENSE create mode 100644 skills/typesafe-ai/SKILL.md create mode 100644 skills/typesafe-jev/SKILL.md create mode 100644 skills/typesafe-jev/agents/openai.yaml create mode 100644 skills/typesafe-jev/evals/cases.json create mode 100644 skills/typesafe-jev/references/jev.md diff --git a/README.md b/README.md index 6349187..ad70525 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,6 @@ # software-maestro -A capability orchestra of 72 lean agent skills for AI IDE agents (Claude Code, Codex, Antigravity, Gemini, and Hermes Agent). +A capability orchestra of 74 lean agent skills for AI IDE agents (Claude Code, Codex, Antigravity, Gemini, and Hermes Agent). --- @@ -36,9 +36,9 @@ A capability orchestra of 72 lean agent skills for AI IDE agents (Claude Code, C
-engineering/ (10 skills across 3 subdomains) +engineering/ (11 skills across 3 subdomains) -* **`engineering/development-and-quality/`** (7 skills): Core engineering craft, code review checklists, Test-Driven Development (TDD), evaluation-driven AI development, ASD-STE100 technical writing standards, info-to-ink token compression, and open source contribution etiquette with Conventional Commits (`developer-development-rules`, `developer-code-review-rules`, `developer-test-driven-development`, `developer-eval-driven-development`, `technical-language-rules`, `info-to-ink`, `open-source-contributor`). +* **`engineering/development-and-quality/`** (8 skills): Core engineering craft, code review checklists, Test-Driven Development (TDD), evaluation-driven AI development, ASD-STE100 technical writing standards, info-to-ink token compression, open source contribution etiquette, and TypeSafe Jev workflow design (`developer-development-rules`, `developer-code-review-rules`, `developer-test-driven-development`, `developer-eval-driven-development`, `technical-language-rules`, `info-to-ink`, `open-source-contributor`, `typesafe-jev`). * **`engineering/architecture-and-resiliency/`** (2 skills): Clean/Hexagonal system architecture with circuit breakers and zero-key web search grounding (`system-architecture-rules`, `grounded-search-rules`). * **`engineering/cloud-and-infrastructure/`** (1 skill): Multi-cloud architecture, hybrid cloud networking, and reusable Terraform IaC modules (`multi-cloud-architecture`). @@ -92,7 +92,7 @@ A capability orchestra of 72 lean agent skills for AI IDE agents (Claude Code, C
-growth/ (6 skills) +growth/ (7 skills) * **`analytics-event-tracking`**: Behavioral telemetry taxonomies, retention modeling, and CRM lifecycle automation. * **`commerce-ux-rules`**: Shopping cart optimization, checkout friction reduction, and multi-step wizard forms. @@ -100,6 +100,7 @@ A capability orchestra of 72 lean agent skills for AI IDE agents (Claude Code, C * **`experimentation-hypothesis-engine`**: Statistical A/B testing, MDE sample sizing, SRM validation, and 3-way post-mortems. * **`conversion-copywriting`**: High-conversion landing page copy, value propositions, and concise prose. * **`american-dialect`**: Research-backed regional American English rewrites that preserve author voice and avoid caricature. +* **`negotiation-maximizer`**: Ethical, low-theater negotiation for compensation, purchases, renewals, contracts, and commercial terms.
@@ -143,8 +144,8 @@ A capability orchestra of 72 lean agent skills for AI IDE agents (Claude Code, C │ ├── interaction-and-layout/ # Responsive layouts, animation physics, IA wayfinding (3 skills) │ ├── experience-and-flows/ # Onboarding UX, Prototype Facade, Chart viz (3 skills) │ └── evaluation-and-quality/ # Usability heuristics & 5-Act interview testing (2 skills) -├── engineering/ # Code Quality, Architecture, Cloud & IaC (10 skills) -│ ├── development-and-quality/ # TDD, evals, code reviews, craft, STE prose, info-to-ink, OSS etiquette (7 skills) +├── engineering/ # Code Quality, Architecture, Cloud & IaC (11 skills) +│ ├── development-and-quality/ # TDD, evals, code reviews, craft, STE prose, info-to-ink, OSS etiquette, Jev (8 skills) │ ├── architecture-and-resiliency/ # System architecture, grounded search (2 skills) │ └── cloud-and-infrastructure/ # Multi-cloud architecture & Terraform IaC (1 skill) ├── finance/ # Settlement rails, tax nexus, chargebacks & treasury (1 skill) @@ -153,7 +154,7 @@ A capability orchestra of 72 lean agent skills for AI IDE agents (Claude Code, C ├── ops/ # SEV-0 incident command, crisis comms & blameless RCA (1 skill) ├── data-and-api/ # API guidelines, caching, causal inference, semantic layer, metric triage (5 skills) ├── security/ # Developer security & auth/identity rules (2 skills) -├── growth/ # Telemetry, commerce UX, SEO, experimentation, regional copy (6 skills) +├── growth/ # Telemetry, commerce UX, SEO, experimentation, regional copy, negotiation (7 skills) ├── quality/ # CI/CD, FinOps, dogfooding, a11y, observability, web perf, peekaboo (8 skills) └── productivity-maestro/ # Executive memos, meeting intelligence & 3-tier Scrum cadences (5 skills) ├── executive-and-async/ # Meeting intelligence & Smart Brevity async memos (2 skills) @@ -188,7 +189,7 @@ The export copies each skill package, including `references/`, `scripts/`, and ` ## 🔍 Local Skill Discovery CLI -Instantly search, discover, and inspect all 71 local skills from your terminal: +Instantly search, discover, and inspect all 74 local skills from your terminal: ```bash # Keyword search across names, triggers, and expert frameworks diff --git a/growth/negotiation-maximizer/SKILL.md b/growth/negotiation-maximizer/SKILL.md new file mode 100644 index 0000000..7461469 --- /dev/null +++ b/growth/negotiation-maximizer/SKILL.md @@ -0,0 +1,156 @@ +--- +name: negotiation-maximizer +description: Use when the user wants to negotiate a price, rate, compensation, contract, purchase, hotel, service, vendor deal, discount, upgrade, fee, renewal, refund, or other commercial terms; asks what to say to improve leverage or get a better deal; provides an offer/counteroffer and wants a response; or wants to maximize coupons, direct-booking value, bundles, competing configurations, or all-in economics. Do not trigger for ordinary writing, simple price lookup/comparison, arithmetic, generic persuasion, or conflict mediation without a concrete negotiated outcome. +compatibility: Claude Code, Gemini CLI, OpenAI Codex, and other Agent Skills-compatible runtimes +--- + +# Negotiation Maximizer + +## Objective + +Maximize expected deal value while preserving credibility, optionality, and relationship quality. + +Optimize for better economic and non-price terms, lower hidden cost, stronger fallback options, and minimum visible negotiation theater. + +**Core rule: maximize leverage; minimize visible tactics.** + +The final message should sound like a normal informed buyer or professional making a decision—not someone performing negotiation tactics. + +## Default voice + +- Casual, concise, direct. +- Maximize information-to-ink ratio. +- State exact requirements, dates, quantities, configurations, and decision criteria. +- Prefer one strong lever over a laundry list of tactics. +- Keep alternatives credible and usually implicit. +- Never reveal maximum willingness to pay unless strategically necessary. +- Make the counterparty move first when that improves information or price discovery. +- Ask for all-in / including-tax / including-fee pricing when relevant. +- Signal ability to close without sounding eager, desperate, or theatrical. +- Avoid corporate-sales phrasing, faux warmth, jargon, and obvious negotiation tricks. +- If a sentence can be removed without losing leverage or information, cut it. + +## Silent preparation + +Before drafting, determine: + +1. **Objective** — desired outcome. +2. **BATNA** — best credible alternative. +3. **Reservation point** — walk-away threshold, if known; keep private. +4. **Counterparty incentives** — revenue, occupancy, quota, retention, direct-channel economics, certainty, simplicity. +5. **Information gap** — what should they reveal before the user reveals more? +6. **Comparison basis** — normalize taxes, fees, configuration, quantity, cancellation, service level, and benefits. +7. **Leverage** — choose the strongest one or two points only. +8. **Sequencing** — decide whether to anchor, counter, or make them move first. +9. **Close path** — give them an easy way to say yes. + +Do not expose this analysis unless asked. + +## Strategy + +### Normalize the real economics +Compare total value, not headline price: taxes, mandatory fees, included benefits, cancellation, configuration, quantity, payment timing, quality, switching cost, and risk. + +### Protect BATNA and private information +Never invent competing offers. Do not disclose a weak BATNA. Reveal a strong alternative only when it improves the deal. Never anchor with the user's maximum. + +### Choose who anchors +Prefer making the counterparty move first when discount authority is unknown, unpublished pricing may exist, or the user lacks a strong market anchor. Anchor first when objective market data or a credible competing offer gives the user a strong reference point. + +### Use one calibrated problem-solving question +Prefer: +- “What can you do on the rate to make this the better option?” +- “What flexibility do you have on the total?” +- “How close can you get to the alternative?” +- “How can we structure this so it works at that level?” + +One good question beats several stacked questions. + +### Trade; do not donate +Make concessions conditional: “If you can do X, I can book/sign/move forward.” + +### Optimize configuration before squeezing price +Compare one large unit vs. two smaller units, room-only vs. package, monthly vs. annual, price vs. fee waiver/credit/upgrade, direct vs. marketplace, quantity tiers, or removal of low-value add-ons. + +### Use seller economics silently +Infer flexibility from occupancy/capacity, direct-channel commission savings, quotas, retention economics, marginal upgrade cost, perishable inventory, commitment size, and operational simplicity. Use this to choose the ask; do not lecture the seller about it. + +### Use loss aversion without theater +A credible preference is enough: “I’d rather book the two-bedroom if the numbers work.” Never invent deadlines or scarcity. + +## Default message architecture + +1. Brief context. +2. Exact desired outcome/configuration. +3. Strongest comparison or leverage. +4. One calibrated ask. +5. Easy path to close. + +## Deal-maximization layer + +Silently check whether value can come from direct booking, alternate configurations, quantity/length tiers, member or loyalty rates, public promos, targeted offers, card/portal benefits, package removal, fee waivers, credits, upgrades, refundable-vs-nonrefundable spread, price matching, retention offers, or timing. + +Do not dump these mechanisms into the message. Pick the strongest lever. + +Never recommend false eligibility, fabricated offers, account abuse, obvious pricing-error exploitation, or deception. + +## Response modes + +- **“What should I say?”** Return the finished message first; keep explanation minimal. +- **“What’s the strategy?”** Give strongest leverage, information to withhold, first ask, likely counter, and next move. +- **Offer/counteroffer provided:** normalize total economics, compare to BATNA, recommend accept/counter/wait. +- **Draft provided:** preserve voice and improve leverage with the fewest edits. +- **Maximize a deal:** identify alternate structures before assuming price is the only lever. + +## Ethics + +Never fabricate offers, eligibility, identities, authority, defects, scarcity, or deadlines. Never use coercion, harassment, impersonation, or reputational threats. Strong negotiation is compatible with truthfulness. + +## Make [Expert PhD Legend] Proud + +Before finalizing, pressure-test the strategy against the canonical thinkers who literally wrote the books practitioners still reference: + +- **Roger Fisher, William Ury, Bruce Patton — _Getting to Yes_**: BATNA protected; interests and objective criteria clear. +- **Howard Raiffa — _The Art and Science of Negotiation_**: alternatives, uncertainty, tradeoffs, and information value handled deliberately. +- **Max Bazerman — negotiation and behavioral decision research**: anchoring, bias, overconfidence, escalation, and judgment errors checked. +- **Deepak Malhotra — _Negotiation Genius_ / _Negotiating the Impossible_**: structural leverage, sequencing, hidden interests, and non-price value considered. +- **Robert Cialdini, PhD — _Influence_ / _Pre-Suasion_**: influence credible, ethical, and subtle—not manipulative. +- **Neil Rackham — _SPIN Selling_**: the seller-facing logic exposes the real problem and value gap rather than dumping arguments. +- **Chris Voss — _Never Split the Difference_**: a natural calibrated question or tactical-empathy move lets the counterparty solve the constraint without sounding scripted. +- **James K. Sebenius — 3-D Negotiation**: setup, sequencing, parties, scope, and deal design improved before bargaining harder. + +Final bar: +1. Fisher/Ury/Patton proud — BATNA protected. +2. Raiffa/Bazerman proud — economics normalized; biases checked. +3. Malhotra/Sebenius proud — structure optimized before squeezing price. +4. Cialdini proud — influence ethical and invisible. +5. Rackham proud — ask solves a real seller decision problem. +6. Voss proud — one calibrated question does more work than three arguments. +7. **User proud** — casual, concise, high information-to-ink ratio, no negotiation cosplay. + +If the draft impresses the experts but no longer sounds like the user, rewrite it. **Expert strategy stays invisible; user voice stays visible.** + +## Pre-Flight Deal Quality & Leverage Scorecard (DQLS) + +Evaluate candidate drafts silently against these gates before returning: + +### Hard Safety Gates (Binary: PASS / BLOCK) +- **Zero Ceiling Leakage**: Never discloses maximum willingness to pay, budget cap, or weak alternatives. +- **Zero Negotiation Cosplay**: Completely free of negotiation jargon, framework names, and theatrical bargaining speak (no "BATNA", "anchoring", "tactical empathy", or "hostage tactics"). +- **Strict Grounding & Credibility**: No fabricated counteroffers, fake deadlines, false authority, or fictitious competing quotes. + +### Strategic Quality Rubric (Target: >= 23 / 25) + +| Dimension | 1 - Critical Failure | 3 - Competent / Generic | 5 - Expert / Invisible Leverage | +| :--- | :--- | :--- | :--- | +| **Normalized Economics** | Fixates solely on headline rate. | Mentions taxes/fees but compares apples to oranges. | Compares true all-in total; tests alternate configurations (tiers, units, credits, direct-booking margin). | +| **Calibrated Burden** | Demands a flat discount or makes aggressive demands. | Asks multiple questions or vague favors. | Uses **exactly one** calibrated question that forces the counterparty to solve the pricing constraint. | +| **Conditional Trade** | Donates concessions for free or begs for price relief. | Mentions moving forward loosely without a firm condition. | Clean conditional trade: *"If you can do X, I can book/sign/close today."* | +| **Information-to-Ink** | Wordy, over-explains context, uses faux warmth/filler. | Standard professional email tone. | Maximum density: every word carries leverage; removes anything that doesn't advance the close. | +| **Frictionless Close** | Open-ended dead end with no clear next step. | Asks for a callback or generic follow-up. | One-click close path: gives the seller the simplest possible way to say "yes". | + +### Execution Gate +- **Score < 20 or any Hard Gate FAIL**: Rewrite immediately. +- **Score >= 23**: Deliver draft (message first; keep scorecard and strategy notes private unless requested). + +Read `references/source.md` when deeper rationale or attribution is needed. \ No newline at end of file diff --git a/growth/negotiation-maximizer/agents/openai.yaml b/growth/negotiation-maximizer/agents/openai.yaml new file mode 100644 index 0000000..1cba570 --- /dev/null +++ b/growth/negotiation-maximizer/agents/openai.yaml @@ -0,0 +1,7 @@ +interface: + display_name: "Negotiation Maximizer" + short_description: "Negotiate smarter while keeping tactics invisible" +default_prompt: "Use $negotiation-maximizer to improve this negotiation, identify leverage, and draft a concise message in my voice." + +policy: + allow_implicit_invocation: true diff --git a/growth/negotiation-maximizer/evals/cases.json b/growth/negotiation-maximizer/evals/cases.json new file mode 100644 index 0000000..a40db40 --- /dev/null +++ b/growth/negotiation-maximizer/evals/cases.json @@ -0,0 +1,45 @@ +{ + "skill": "negotiation-maximizer", + "version": 1, + "trigger_cases": [ + {"prompt": "Help me get this hotel to discount the two-bedroom suite.", "trigger": true}, + {"prompt": "Write a recruiter response; I want more comp without giving my floor.", "trigger": true}, + {"prompt": "They offered 15% off annual SaaS. What should I counter?", "trigger": true}, + {"prompt": "The 2BR costs more than two 1BRs. Help me use that.", "trigger": true}, + {"prompt": "Rewrite this so they name the discount first.", "trigger": true}, + {"prompt": "Convert 62,935 MXN to USD.", "trigger": false}, + {"prompt": "Summarize this hotel review.", "trigger": false}, + {"prompt": "Find current coupon codes for this store.", "trigger": false}, + {"prompt": "Rewrite this thank-you email.", "trigger": false}, + {"prompt": "What is the cancellation policy?", "trigger": false} + ], + "binary_cases": [ + { + "id": "protect-ceiling", + "prompt": "Draft a hotel negotiation email. My maximum is $650/night but I want them to quote first.", + "pass_rule": "The message asks the hotel to move first and does not disclose the user's $650 ceiling.", + "must": [["what can you do", "best rate", "flexibility"]], + "must_not": ["$650", "650/night", "my maximum"], + "pass_fixture": "What can you do on the all-in two-bedroom rate to make it the better option for the stay?", + "fail_fixture": "My maximum is $650 per night. Can you meet it?" + }, + { + "id": "all-in-economics", + "prompt": "Help me compare a direct hotel offer against another configuration with fees and taxes.", + "pass_rule": "The response normalizes the comparison to total/all-in economics before recommending a move.", + "must": [["all-in", "including tax", "total"], ["compare", "comparison"]], + "must_not": ["headline rate only"], + "pass_fixture": "Compare the two options on an all-in basis including tax and mandatory fees, then use the cheaper equivalent configuration as leverage.", + "fail_fixture": "Compare only the advertised nightly rates and ignore taxes and fees." + }, + { + "id": "invisible-tactics", + "prompt": "Make this negotiation email sound like me but stronger.", + "pass_rule": "The draft is concise and natural, uses at most one primary calibrated ask, and does not name negotiation frameworks or tactics.", + "must": [["what can you do", "what flexibility", "how close"]], + "must_not": ["BATNA", "anchoring strategy", "tactical empathy", "hostage negotiation"], + "pass_fixture": "What can you do on the two-bedroom rate, including tax, to make it the better value for the stay? If the numbers work, I'd prefer the two-bedroom and can book direct.", + "fail_fixture": "Using my BATNA and a calibrated-question hostage-negotiation tactic, I want to anchor you lower." + } + ] +} \ No newline at end of file diff --git a/growth/negotiation-maximizer/references/source.md b/growth/negotiation-maximizer/references/source.md new file mode 100644 index 0000000..6dcb953 --- /dev/null +++ b/growth/negotiation-maximizer/references/source.md @@ -0,0 +1,53 @@ +# Negotiation Maximizer — Evidence and Sources + +## Skill design and portability + +- OpenAI, Build skills / Skills guidance: skills use a `SKILL.md` manifest with concise descriptions and progressive disclosure; descriptions should clearly scope implicit invocation. + https://developers.openai.com/codex/build-skills + https://developers.openai.com/api/docs/guides/tools-skills + +- Agent Skills specification and description guidance: activation quality depends heavily on the skill description; test both positive and negative trigger prompts. + https://agentskills.io/ + https://agentskills.io/skill-creation/optimizing-descriptions + +- skills.sh: Agent Skills packages can be installed into multiple supported coding-agent environments from a shared skill source, including Claude Code, Codex, and Gemini CLI. + https://skills.sh/ + +## Negotiation foundations + +- Roger Fisher, William Ury, Bruce Patton — _Getting to Yes_: principled negotiation, BATNA, objective criteria, interests over positions. +- Harvard Program on Negotiation — BATNA and principled-negotiation guidance. + https://www.pon.harvard.edu/daily/batna/translate-your-batna-to-the-current-deal/ + https://www.pon.harvard.edu/daily/negotiation-skills-daily/principled-negotiation-focus-interests-create-value/ + +- Howard Raiffa — _The Art and Science of Negotiation_: analytical negotiation, decision theory, uncertainty, tradeoffs, and information. + +- Max H. Bazerman — negotiation and behavioral decision research: anchoring, judgment, bias, escalation, and decision quality. + +- Deepak Malhotra — _Negotiation Genius_ and _Negotiating the Impossible_: structural leverage, sequencing, hidden interests, and deal design. + +- James K. Sebenius — 3-D Negotiation: setup, sequencing, parties, scope, and architecture of the deal beyond table tactics. + +## Influence and selling + +- Robert Cialdini, PhD — _Influence_ and _Pre-Suasion_: evidence-based ethical influence principles. + +- Neil Rackham — _SPIN Selling_: research-derived consultative selling based on large-scale observation of sales calls; diagnose the buyer/seller decision problem instead of feature-dumping. + +- Chris Voss — _Never Split the Difference_: calibrated questions, labeling, tactical empathy, and inviting the counterpart to solve constraints. Use as practitioner communication mechanics, not as a claim that commercial bargaining is equivalent to hostage negotiation. + +## Deal / coupon maximization + +Treat price as one variable in a broader configuration problem: +- all-in cost +- taxes and fees +- bundles +- alternate room/product configurations +- direct-channel economics +- upgrades/credits +- cancellation terms +- commitment length +- quantity tiers +- retention economics + +Never recommend false eligibility, fabricated offers, coupon abuse, or deception. \ No newline at end of file diff --git a/skills/_software_maestro_manifest.json b/skills/_software_maestro_manifest.json index 5623e98..59882de 100644 --- a/skills/_software_maestro_manifest.json +++ b/skills/_software_maestro_manifest.json @@ -182,6 +182,10 @@ "slug": "multi-cloud-architecture", "source": "engineering/cloud-and-infrastructure/multi-cloud-architecture" }, + { + "slug": "negotiation-maximizer", + "source": "growth/negotiation-maximizer" + }, { "slug": "now-next-later-roadmaps", "source": "product/planning-and-cadence/now-next-later-roadmaps" @@ -278,6 +282,14 @@ "slug": "trust-safety-fraud-and-claims", "source": "trust/trust-safety-fraud-and-claims" }, + { + "slug": "typesafe-ai", + "source": ".agents/skills/typesafe-ai" + }, + { + "slug": "typesafe-jev", + "source": "engineering/development-and-quality/typesafe-jev" + }, { "slug": "user-story-mapping", "source": "product/planning-and-cadence/user-story-mapping" diff --git a/skills/design-rapid-prototype-facade/SKILL.md b/skills/design-rapid-prototype-facade/SKILL.md index 72ec4ae..34f68f5 100644 --- a/skills/design-rapid-prototype-facade/SKILL.md +++ b/skills/design-rapid-prototype-facade/SKILL.md @@ -16,6 +16,88 @@ description: "Build clickable wireframes and high-fidelity UI facades for user p > > **The Prime Directive**: *"Fake it, don't build it. Match fidelity to the uncertainty being tested."* +### Pencil.dev as a Disposable Exploration Facade + +Use Pencil.dev between Tuesday divergence and Wednesday decision when the team needs fast, visual Crazy 8s or a lightweight concept facade. The headless `pen` CLI creates and edits `.pen` files without requiring a GUI; the CLI can also export PNG, JPEG, WEBP, and PDF review artifacts. + +Before use, install the CLI with `npm install -g @pen.dev/cli`, confirm Node.js 22.19 or later, authenticate with `pen login` or `PEN_CLI_KEY`, and run `pen status`. Use `pen codex-login` when the CLI agent is Codex. For desktop/IDE work, open the target `.pen` file, enable Pencil MCP, reload Codex, and confirm `pencil` appears in the live MCP tool list. + +### Required Design-System Inputs + +Before generating or reconstructing any screen, load the product system into Pencil: + +1. Keep `DESIGN.md`, the working `.pen` file, the starter `.lib.pen` library, custom fonts, and referenced images in one workspace. +2. Read `DESIGN.md` and extract principles, aesthetic direction, accessibility, semantic colors, typography, spacing, radii, elevation, borders, motion, responsive rules, themes, and component constraints. +3. Import the `.lib.pen` file through Pencil’s Libraries panel before using the agent. Inspect variables, themes, components, slots, icons, and fonts. Do not assume a starter kit is active merely because it exists in the folder. +4. Attach `DESIGN.md` to CLI prompts with `--prompt-file DESIGN.md` and state that existing library instances and variables are mandatory. The CLI can edit the prepared `.pen` document; perform library setup in the desktop or IDE workflow when required. +5. Run a design-system audit before handoff. Check token usage, component reuse, accessibility, light/dark behavior, responsive behavior, motion, and distinctive visual character. Record every intentional deviation beside the affected frame. + +For this repository, the required system assets are [`design/pencil-systems/DESIGN.md`](../../pencil-systems/DESIGN.md), [`core-ios.lib.pen`](../../pencil-systems/core-ios.lib.pen), [`ios-reconstruction-template.pen`](../../pencil-systems/templates/ios-reconstruction-template.pen), and [`crazy-8s-ios-template.pen`](../../pencil-systems/templates/crazy-8s-ios-template.pen). They encode the preferred stack and visual language: SvelteKit/TypeScript, Vite, Bun, Biome, Vanilla CSS or Tailwind CSS, Bits UI/shadcn-svelte/Melt UI, Lucide/iconoir, Motion, SF Pro Text/Display for iOS, IBM Plex Mono for data, and semantic light/dark tokens. Use the library as a visual contract; do not add a production dependency merely to satisfy Pencil. + +The accessibility and usability gate is mandatory: 44pt targets, visible focus, WCAG AA contrast, non-color status communication, reduced-motion behavior, clear labels, progressive disclosure, authentic copy, and explicit idle/loading/empty/success/error/recovery states. Use icon nodes from the preferred library, not hand-drawn substitutes. Preserve instances and variable aliases; document every detach or deviation. + +### Use the new import and sync capabilities + +- **Existing web UI:** Keep the `.pen` file in the same workspace as the source. Use Code → Design to recreate the named component or page, then use the built-in browser to import the running page or selected element for visual comparison. JavaScript interactions do not transfer. +- **Existing iOS UI:** Attach Simulator screenshots and SwiftUI/UIKit source to the agent. Recreate editable layers from the screenshot and source. The screenshot remains authoritative for the existing app; `DESIGN.md` constrains proposed changes. +- **Existing Figma work:** Import the complete Figma file when available. Use SVG import for editable vectors and PNG/JPEG for locked reference evidence. +- **Design tokens:** Import CSS variables from `globals.css` or the project token file into Pencil. Export approved Pencil variable changes back to CSS only after reviewing conflicts and implementation impact. +- **Context:** Use repeatable `--prompt-file` attachments for `DESIGN.md`, source, screenshots, briefs, and critique. Use `--repo` to expose the relevant project workspace. Use `--preview-output` or `--enable-preview` when the review requires iterative visual evidence. +- **Icons:** Use the built-in Lucide, Phosphor, Feather, or Material Symbols library when appropriate; map production output to Lucide or iconoir according to the project preference. + +Use this prompt prefix for every Pencil task: + +```text +Read DESIGN.md before editing. +Use the imported Pencil library, its variables, themes, components, slots, +icons, and fonts. Reuse an existing primitive whenever it matches. +Do not invent competing tokens or patterns. List intentional deviations +from DESIGN.md with a reason. Keep all output editable. +``` + +Before a facade handoff, record the imported-library proof, variable/component reuse, simulator comparison, accessibility audit, state coverage, responsive and motion decisions, and unresolved risks. A Pencil reconstruction remains a visual artifact; SwiftUI/UIKit or the web facade remains the runtime source of truth. + +- **Crazy 8s**: Create eight labeled frames in one `.pen` file. Keep the target step, actor, content constraints, and viewport constant while varying composition, hierarchy, and interaction cues. +- **Prompt with a review question**: Include the sprint brief, authentic domain copy, required frame labels, the dimensions of variation, and the question the team will use for voting. +- **Refine selectively**: Pass critique notes as a prompt file and revise only the shortlisted frames. Preserve the original `.pen` board so the divergence evidence remains inspectable. +- **Export for decision**: Export a single board image for silent review, heatmap voting, and the Decider’s selection. Do not treat generated screens as proof of usability or implementation feasibility. +- **Handoff**: Carry the selected frame, target step, actor, decision rationale, and unresolved risks into the Wednesday storyboard. Use Pencil.dev for exploration; use this Thursday facade workflow only when the selected path needs clickable testing. + +Example: + +```bash +pen --out crazy-8s.pen --prompt-file DESIGN.md --prompt-file sprint-brief.md --prompt "Create eight materially different Crazy 8s concepts for the locked target step. Use the imported Pencil library and DESIGN.md as mandatory constraints. Label frames 1–8, keep the actor and content constraints constant, and vary layout, hierarchy, and interaction cues. Use authentic copy only." +pen --in crazy-8s.pen --out crazy-8s-shortlist.pen --prompt-file critique.md --prompt "Refine only the shortlisted frames identified in critique.md. Preserve frame labels and do not add backend behavior." +pen --in crazy-8s-shortlist.pen --export crazy-8s.png --export-scale 2 +``` + +### Recreating an Existing iOS App UI + +Pencil.dev cannot currently pull a running native iOS app into editable layers. Use this workflow when an existing app is the visual baseline: + +1. **Capture the runtime**: Run the app in Xcode Simulator and capture every storyboard state at the target device size. Use `xcrun simctl io booted screenshot app-state.png`, including home, loading, empty, menu, form, error, and selected states. +2. **Import the reference**: Import each PNG into Pencil.dev and lock it behind the reconstruction. PNG and JPEG files become image layers, so they preserve pixel accuracy but are not editable UI. +3. **Recreate the layers**: Keep the `.pen` file beside the Xcode project. Ask the AI agent to recreate the relevant SwiftUI or UIKit view as editable Pencil layers, using the screenshot as the visual source of truth and the source code as the structural source of truth. +4. **Build prototype states**: Duplicate the recreated screen into named states such as `Home`, `Home — menu open`, `Home — loading`, and `Home — error`. Wire transitions between those states in Pencil. +5. **Run visual QA**: Export the Pencil screen and compare it with the Simulator screenshot. Correct safe-area placement, typography, wrapping, spacing, colors, corner radii, shadows, icon sizes, and viewport dimensions before using the facade for testing. + +**Existing-app precedence:** The Simulator screenshot and observed runtime states are the reconstruction baseline. Use `DESIGN.md` and the imported library to identify the underlying system, preserve known tokens, and constrain proposed changes. Do not replace an existing screen with the starter system merely to make it look more consistent. Record any screenshot-versus-system mismatch as observed legacy, intentional product behavior, or proposed change. + +Use this prompt pattern: + +```text +Recreate the UI shown in app-home.png from +Sources/Features/Home/HomeView.swift. + +Match the screenshot exactly: +- Preserve the 393x852 viewport and safe-area placement +- Match typography, spacing, colors, corner radii, shadows, and icon sizes +- Use editable Pencil layers and reusable components +- Do not invent content +``` + +For exact native behavior, keep the prototype in SwiftUI or UIKit. Use Pencil for editable visual reconstruction, state exploration, and prototype transitions—not as a replacement for the iOS runtime. + --- ## When to use @@ -23,6 +105,7 @@ description: "Build clickable wireframes and high-fidelity UI facades for user p Use this skill on Thursday of a Design Sprint to build the testing prototype: - **Tier 1 (Basic Wireframe Facade)**: Building schematic, clickable wireframe prototypes in `tldraw Desktop` via `tldraw-offline` for early structural feedback. - **Tier 2 (High-Fidelity Web Facade)**: Generating pixel-perfect screens and HTML/CSS via **Google Stitch MCP** (`generate_screen_from_text`, `edit_screens`) and multi-screen baton assembly (`stitch-loop`). +- **Pre-Thursday (Disposable Visual Exploration)**: Generating and comparing Crazy 8s or selected concept frames in **Pencil.dev** with the headless `pen` CLI. Use this before storyboard lock, not as a substitute for the clickable Thursday facade. - Organizing the sprint team across **Maker**, **Stitcher**, **Writer**, and **Asset Collector** roles. - Enforcing **100% authentic copy and domain data** (strict zero *Lorem Ipsum* rule). - Conducting the mandatory **15:00 Trial Run QA audit**. @@ -37,6 +120,7 @@ Do not use this skill for: ## Trigger cues - Request mentions: `goldilocks prototype`, `prototype facade`, `realistic UI illusion`, `stitch prototype`, `tldraw wireframe prototype`, `tldraw clickable prototype`, `rapid interactive prototype`, `hollywood set facade`, `stitch rapid facade`, `basic wireframe facade`. +- Request mentions: `Pencil.dev`, `pencil prototype`, `pen CLI`, `Crazy 8s board`, or `disposable visual exploration`. ## Instructions @@ -56,5 +140,10 @@ Do not use this skill for: - [ ] Interactive facade covering 100% of storyboard scenes produced. - [ ] Zero *Lorem Ipsum* or generic placeholder text. +- [ ] `DESIGN.md` was read and attached to the agent task. +- [ ] The starter `.lib.pen` library was imported, and matching variables and components were reused. +- [ ] The actual imported library, themes, variables, slots, icons, fonts, and reusable components were inspected and recorded. +- [ ] Design-system audit passes for tokens, accessibility, themes, responsive behavior, motion, and distinctive character. +- [ ] Intentional deviations are documented beside the affected frame. - [ ] Sub-150ms interaction latency on golden path. - [ ] 15:00 Trial Run QA report logged with zero blocker bugs. diff --git a/skills/design-rapid-prototype-facade/references/source.md b/skills/design-rapid-prototype-facade/references/source.md index 0ea72ee..f5c2897 100644 --- a/skills/design-rapid-prototype-facade/references/source.md +++ b/skills/design-rapid-prototype-facade/references/source.md @@ -16,6 +16,8 @@ └───────────────────────────────┴────────────────────────────────┴──────────────────────────────────────────┘ ``` +Pencil.dev is the system-grounded exploration tier: use it for existing web or iOS UI reconstruction, Figma/web import, token synchronization, and controlled Crazy 8s before the selected concept becomes a clickable facade. + --- ## 2. Low-Fidelity Wireframing Workflow with `tldraw-offline` @@ -40,3 +42,7 @@ 3. **Multi-Screen Baton Assembly**: Use `stitch-loop` (`.stitch/next-prompt.md`) to assemble the full golden path in `site/public/`. 4. **Targeted Edits**: Use `edit_screens` for rapid micro-adjustments. 5. **Local Verification**: Verify with Chrome DevTools MCP (`npx serve site/public`) before the 15:00 Trial Run. + +## 4. System-grounded Pencil.dev workflow + +Read `design/pencil-systems/DESIGN.md`, import `core-ios.lib.pen`, and inspect variables, themes, components, slots, icons, and fonts. Use Code → Design for existing web components, browser import for running web pages, Figma import when a Figma source exists, and screenshot-plus-source reconstruction for iOS. Use `ios-reconstruction-template.pen` for a Simulator baseline and `crazy-8s-ios-template.pen` for eight controlled variations. Keep the screenshot, viewport, user, content, and target step constant; vary one hypothesis at a time. Synchronize CSS and Pencil variables only after conflict review. Export the selected frame, compare it with the source screenshot or running page, and attach the token, accessibility, state, responsive, motion, and deviation audit to the handoff. diff --git a/skills/design-sprint/SKILL.md b/skills/design-sprint/SKILL.md index 7445d1e..31896fd 100644 --- a/skills/design-sprint/SKILL.md +++ b/skills/design-sprint/SKILL.md @@ -13,6 +13,76 @@ description: "Facilitate 5-day design sprints, problem sketches, decision matric > > **The Prime Directive**: *"Start at the end, work alone together, make visual decisions, and choose the right fidelity for the problem."* +## Pencil.dev Prototyping for Crazy 8s + +Use Pencil.dev when the team needs eight fast visual variations that are easier to compare, refine, and export than hand-drawn sketches. Treat each `.pen` file as a disposable exploration artifact, not production UI. + +- **Create the board**: Use the headless `pen` CLI to create a `.pen` file with eight clearly labeled frames. Keep each frame focused on one solution variation or one target step. +- **Prompt the variations**: Use `pen --out crazy-8s.pen --prompt-file brief.md --prompt "Create eight materially different concepts for [target step]. Label each frame 1–8. Preserve the same user, goal, and content constraints while varying layout, hierarchy, and interaction."` +- **Work in batches**: Generate one board first, then use `pen --in crazy-8s.pen --out crazy-8s-refined.pen --prompt "Refine only frames 2, 5, and 7 using the critique notes in critique.md"` for targeted iteration. +- **Compare and share**: Export a review image with `pen --in crazy-8s-refined.pen --export crazy-8s.png --export-scale 2`. Use the exported board for silent review, heatmap voting, and Decider selection. +- **Keep the sprint boundary**: Do not add backend behavior, responsive completeness, polished copy, or design-system infrastructure. Carry only the selected concept and its evidence into the storyboard. + +### Current pen.dev workflow + +Use the current authenticated `pen` CLI or the pen.dev desktop/IDE MCP integration. Run `pen status` first; use `pen login` for local work, `PEN_CLI_KEY` for CI/CD, and `pen codex-login` for Codex agent mode. Keep the `.pen` file beside the project and pass `--repo` when the agent must inspect source, tokens, screenshots, or assets. + +- For an existing web app, use Code → Design: ask Pencil to recreate the relevant component or page from source, then compare it with the running UI. +- For an existing iOS app, attach Simulator screenshots and SwiftUI/UIKit source. Screenshots define visual truth; source defines structure. Pencil does not directly import a running native app. +- Synchronize CSS variables and Pencil variables when the system is approved. Resolve variable conflicts explicitly; do not create duplicate token families. +- Use desktop browser import for web pages or selected elements, Figma import for complete Figma files, SVG import for editable vectors, and PNG/JPEG import for flattened reference evidence. +- Use the current built-in icon libraries where appropriate: Lucide, Phosphor, Feather, or Material Symbols. Keep production code on the project’s preferred Lucide or iconoir path. +- In Codex, open the intended `.pen` file, enable the Pencil MCP integration, confirm `pencil` is in the live tool list, attach `DESIGN.md` and briefs, and include the full document path in the prompt. + +### Corey Pencil system contract + +Use [`design/pencil-systems/DESIGN.md`](../../pencil-systems/DESIGN.md) and import [`core-ios.lib.pen`](../../pencil-systems/core-ios.lib.pen) before generating. Start from [`crazy-8s-ios-template.pen`](../../pencil-systems/templates/crazy-8s-ios-template.pen) for iOS target steps, or [`ios-reconstruction-template.pen`](../../pencil-systems/templates/ios-reconstruction-template.pen) when a real screen is the baseline. These are required inputs, not optional starter kits. + +- Map Pencil variables and components to the project preferences: SvelteKit and TypeScript, Vite, Bun, Biome, Vanilla CSS or Tailwind CSS, Bits UI/shadcn-svelte/Melt UI, Lucide or iconoir, Motion, and Zod or Valibot where applicable. +- Use SF Pro Text and SF Pro Display for native iOS intent, IBM Plex Mono for data and audit notes, and declare any Pencil preview fallback. Do not silently substitute a competing type scale. +- Read `DESIGN.md`, run `read_skill()`, inspect `get_app_state()`, variables, themes, reusable components, slots, icons, and fonts, then verify the imported library before creating frames. +- Reuse instances and token aliases. Do not detach or invent colors, spacing, radii, type, icons, or motion without documenting the deviation. Use Code on Canvas only for repeated or parameterized structures. +- Apply the usability and accessibility gate: 44pt targets, visible focus, WCAG AA contrast, non-color state communication, reduced motion, authentic labels, and idle/loading/empty/success/error/recovery states. +- Keep the target step, actor, viewport, content, and brand constant across eight frames. Vary one meaningful hypothesis per frame and preserve the locked reference. + +### Pencil.dev Crazy 8s Prompt Contract + +Every Pencil.dev Crazy 8s prompt must state: + +1. The target step, actor, and user outcome. +2. The fixed content and domain constraints shared by all eight frames. +3. The dimensions of variation: layout, hierarchy, navigation, or interaction model. +4. The required labels, frame order, and output file. +5. The review question that will decide which concept advances. +6. The `DESIGN.md` path and imported starter library that the agent must use. +7. The allowed system deviations, with a reason for each deviation. +8. The design-system audit checks required before voting. + +### Importing an Existing iOS UI + +Pencil.dev does not import a running native iOS app as editable layers. Use the Simulator as the visual source of truth, then combine screenshots with the iOS source code. + +1. Capture each relevant app state at the target device size with `xcrun simctl io booted screenshot app-state.png`. +2. Import the PNG into Pencil.dev as a locked reference. PNG and JPEG imports remain flattened image layers. +3. Keep the `.pen` file beside the Xcode project and ask the agent to recreate the relevant SwiftUI or UIKit view as editable layers using the screenshot for visual accuracy and source code for structure. +4. Duplicate the recreated screen into interaction states, then use Pencil transitions for prototype behavior. + +For Crazy 8s, import one real screen as the baseline and vary only the locked target step. Keep the device viewport, brand system, typography, user, and real content constant. Vary layout hierarchy, primary-action placement, navigation, information density, and interaction reveal patterns. Treat the screenshot as exact visual evidence and the reconstruction as subject to visual QA. + +**Existing-app precedence:** During reconstruction, the Simulator screenshot and observed runtime states are the visual source of truth. `DESIGN.md` explains the tokens and principles behind the UI and constrains proposed changes; it must not silently overwrite an existing screen. Label screenshot-versus-system mismatches as observed legacy, intentional product behavior, or proposed change. + +### Mandatory Pencil.dev Design-System Setup + +Do not start Pencil exploration from a blank canvas when a product system exists. Treat `DESIGN.md` and the imported Pencil starter library as required design inputs. + +1. Read `DESIGN.md` and extract the product principles, aesthetic direction, accessibility stance, tokens, themes, responsive rules, motion rules, and component constraints. +2. Open or create the starter `.pen` file, then import its `.lib.pen` design library into the working document. Inspect variables, theme modes, components, slots, icons, fonts, and reusable patterns before generating screens. +3. Use existing variables and component instances for every matching element. Create a new component only when the concept requires a genuinely new pattern. +4. Include `DESIGN.md` in every agent prompt. Require the agent to cite intentional deviations and never invent a competing color, type scale, spacing scale, radius, or component pattern. +5. Audit each shortlisted concept against the `design-system-rules` checklist: principles, tokens, visual language, accessibility, component reuse, light/dark behavior, responsive behavior, motion, and distinctive character. + +If no starter library exists, create the minimum viable library before Crazy 8s: semantic color variables, typography variables, spacing, radii, elevation, focus treatment, light/dark themes, and the core Button, Text, Icon, Field, Feedback, Navigation, Overlay, and Layout components. + --- ## When to use @@ -28,10 +98,16 @@ Use this skill during the initial 3 days of a Design Sprint: - Use `tldraw-offline` (`/api/doc/:id/exec`) to programmatically render Customer Journey Maps with bound arrows (`helpers.createArrowBetweenShapes`) and arrange Crazy 8s / 10-15 panel storyboard frames directly on the user's live canvas. - **Google Stitch MCP for High-Fidelity Exploration**: - Prompt Stitch MCP (`generate_screen_from_text`) during Tuesday Lightning Demos to explore diverse UI atmospheres (*Bento Grid*, *Glassmorphism*, *Minimalist Monospace*) and synthesize `.stitch/DESIGN.md` tokens. +- **Pencil.dev for Crazy 8s and Disposable Visual Exploration**: + - Use the headless `pen` CLI to generate, revise, and export eight labeled `.pen` concepts. Use it for divergent visual exploration before the Decider locks one Target Step and Actor; use `design-rapid-prototype-facade` for the selected Thursday prototype. ## Completion gate - [ ] Clear phase artifacts generated (Map, Crazy 8s, or 10-15 panel Storyboard). - [ ] Exactly 1 Target Step and Actor locked by the Decider. - [ ] Storyboard laid out (on `tldraw` canvas, Stitch schema, or structured Markdown). +- [ ] Crazy 8s reviewed as eight labeled Pencil.dev concepts when visual divergence is part of the sprint. +- [ ] `DESIGN.md` and the Pencil starter library were loaded before generation. +- [ ] The actual Pencil library import, variables, themes, and reusable components were verified before generation. +- [ ] Shortlisted concepts pass the design-system audit, or document intentional deviations. - [ ] Storyboard ready for Thursday handoff to `design-rapid-prototype-facade`. diff --git a/skills/developer-eval-driven-development/SKILL.md b/skills/developer-eval-driven-development/SKILL.md index 634bda3..37098af 100644 --- a/skills/developer-eval-driven-development/SKILL.md +++ b/skills/developer-eval-driven-development/SKILL.md @@ -1,6 +1,6 @@ --- name: developer-eval-driven-development -description: "Build evaluation datasets, LLM-as-judge rubrics, and regression benchmarks for AI software." +description: "Build evaluation datasets, calibrated judges, typed-decision checks, and regression benchmarks for AI software." --- # EVAL-DRIVEN DEVELOPMENT (EDD) @@ -10,6 +10,7 @@ Treat evaluation as the development loop for behavior that cannot be proven with ## Operating boundary - Use EDD for probabilistic or semantic behavior: model responses, retrieval quality, tool selection, structured generation, safety, and multi-step agent outcomes. +- Use [TypeSafe Jev](../typesafe-jev/SKILL.md) when a workflow needs fast, typed probabilistic judgments; evaluate Jev questions, calibration, confidence gates, and the code-owned action policy as separate behaviors. - Use `developer-test-driven-development` for deterministic code paths, schemas, parsers, and business logic. - Use both when an AI feature contains deterministic infrastructure and probabilistic behavior. - Evaluate the application behavior users receive, not an isolated prompt in a playground. @@ -21,6 +22,7 @@ Treat evaluation as the development loop for behavior that cannot be proven with - Current prompt, model, tools, retrieval, and runtime configuration - Existing production examples, incidents, fixtures, or acceptance criteria - Cost, latency, safety, and release constraints +- For Jev: the primitive, question/criteria contract, pinned model version, confidence or probability threshold, fallback, and downstream action policy If credentials, required configuration, or the meaningful entry point are unavailable, report the exact blocker. Do not replace the system under evaluation with a fake and claim an end-to-end result. @@ -33,7 +35,9 @@ If credentials, required configuration, or the meaningful entry point are unavai - dataset slices - evaluator and pass rule for every criterion - release threshold, budget, and stop condition + - for Jev-backed criteria, separate typed-response validity, semantic correctness, calibration, and downstream policy correctness 3. Capture a baseline on the current implementation. Preserve per-example outputs, traces, scores, latency, cost, errors, and version identifiers. + - For Jev, also preserve the model version, question-set version, selected option or score, full probabilities where available, confidence for `Choice`/`Score`, and probability for `Noul`. 4. Prove the eval can fail: - run a known-bad fixture, historical failure, or deliberate behavioral mutation - confirm each evaluator detects the failure it claims to measure @@ -57,6 +61,9 @@ If credentials, required configuration, or the meaningful entry point are unavai - Run real-model calls in end-to-end evals. Mock them only in deterministic unit tests that are not presented as model-quality evidence. - Calibrate judge rubrics against labeled pass and fail examples before using them as gates. - Default classification evaluators to exact binary labels when the release decision is binary. +- For Jev, use `Choice` for unordered fixed options, `Score` for ordered rubrics, and `Noul` for a clear yes/no probability. Keep questions atomic; batch independent questions and compose their answers in code. +- Calibrate Jev thresholds on labeled development data for the actual risk. Do not transfer thresholds across primitives, datasets, or actions; `Noul` has probability but no separate confidence field. +- Treat a valid Jev response as schema evidence only. Test semantic correctness, uncertainty routing, and the deterministic action policy separately. - Use generic evaluator variables rather than embedding application-specific content in reusable judge prompts. - Keep development and holdout data separate. Do not tune against every case used for final acceptance. - Include happy paths, edge cases, adversarial cases, historical failures, and at least one uncertain failure-mode case. @@ -75,6 +82,8 @@ Use the least subjective evaluator that measures the criterion: Never use a judge to replace an executable assertion. Never use string equality to judge an open-ended answer. +For Jev-backed evaluation, keep arithmetic, date comparison, authorization, invariants, and side effects in code. Jev can judge bounded semantics, but it is not an agent or a prose generator. Evaluate the full workflow, including the code that routes, escalates, confirms, or acts on its typed answer. + ## Completion gate Before reporting success, verify: @@ -85,6 +94,8 @@ Before reporting success, verify: - every case has a result or a documented blocking error - critical slices and individual regressions were inspected - model, prompt, evaluator, dataset, application, and inference settings are recorded +- for Jev, model version, question-set version, primitive, calibration set, threshold, and fallback are recorded +- Jev thresholds were calibrated on labeled data, and low-confidence or ambiguous cases exercised the fallback - cost and latency stayed within their budgets - the holdout set was not used for iterative tuning - the final report distinguishes observed evidence from inference @@ -96,6 +107,7 @@ Do not claim improvement from raw scores without baseline comparison, per-exampl - **Decision:** ship, revise, or block, with the governing threshold - **Scorecard:** baseline versus candidate by criterion and dataset slice +- **Calibration:** Jev probability/confidence reliability, coverage, threshold behavior, and fallback rate when applicable - **Regressions:** failed examples, severity, evidence, and owner - **Validity:** evaluator calibration, dataset limits, variance, and contamination risks - **Actions:** prioritized application, evaluator, and dataset changes diff --git a/skills/developer-eval-driven-development/evals/cases.json b/skills/developer-eval-driven-development/evals/cases.json index e893346..b3325b6 100644 --- a/skills/developer-eval-driven-development/evals/cases.json +++ b/skills/developer-eval-driven-development/evals/cases.json @@ -7,6 +7,7 @@ {"prompt": "Compare these two agent prompts with reproducible evals.", "trigger": true}, {"prompt": "A model migration regressed tool selection. Add evals and fix it.", "trigger": true}, {"prompt": "Use EDD to improve this classifier without overfitting.", "trigger": true}, + {"prompt": "Evaluate a Jev confidence-gated classifier before enabling auto-routing.", "trigger": true}, {"prompt": "Write unit tests for this deterministic date parser.", "trigger": false}, {"prompt": "Review this pull request for security issues.", "trigger": false}, {"prompt": "Improve the visual hierarchy of this checkout page.", "trigger": false} @@ -92,6 +93,22 @@ "must_not": ["fix only"], "pass_fixture": "Minimize the failure, add it to the regression dataset, add an evaluator that catches it, fix the system, and monitor for recurrence.", "fail_fixture": "Apply a fix only and close the incident." + }, + { + "id": "jev-layered-evaluation", + "prompt": "Evaluate a Jev classifier that routes support requests automatically.", + "pass_rule": "Separates typed response validity, semantic correctness, calibration, and deterministic action policy, with versioned thresholds and a safe fallback.", + "must": [ + ["schema", "typed response", "validity"], + ["semantic", "labeled", "human"], + ["calibrat", "probability", "confidence"], + ["deterministic", "policy", "code"], + ["threshold", "fallback", "escalat"], + ["model version", "question-set version"] + ], + "must_not": ["valid schema proves correctness", "reuse one threshold for every action", "let Jev execute the side effect"], + "pass_fixture": "Evaluate four layers separately: typed response validity, semantic correctness against labeled examples or human review, probability and confidence calibration, and deterministic policy behavior in code. Record the model version, question-set version, threshold, and fallback; route uncertain cases to escalation and let code execute side effects.", + "fail_fixture": "The valid schema proves correctness, so reuse one threshold for every action and let Jev execute the side effect." } ] } diff --git a/skills/developer-eval-driven-development/references/source.md b/skills/developer-eval-driven-development/references/source.md index 28907b7..f88f774 100644 --- a/skills/developer-eval-driven-development/references/source.md +++ b/skills/developer-eval-driven-development/references/source.md @@ -45,6 +45,7 @@ Write the contract before changing the application. It should state: - dataset slices and provenance - criterion-to-evaluator mapping - pass rule for each criterion +- for typed-decision systems, separate response validity, semantic correctness, calibration, and downstream policy correctness - critical failures that cannot be averaged away - quality, safety, cost, and latency budgets - experiment budget and stopping rule @@ -146,6 +147,21 @@ An evaluator is another model or program that can fail. Before making it a gate: For high-stakes decisions, sample human review even after calibration. Investigate judge disagreement rather than averaging it away. +### Use Jev for bounded typed judgments + +Jev is useful in EDD when the criterion is a bounded semantic judgment and the application benefits from a typed answer, probability distribution, or confidence signal. Treat the Jev question contract as a versioned evaluator artifact, not as an informal prompt. + +- `Choice` fits unordered fixed labels such as intent or route. +- `Score` fits ordered levels with explicit anchors such as severity or completeness. +- `Noul` fits a clear yes/no condition when the probability itself is useful. +- Keep each question atomic. Batch independent questions in one request and combine them in code. Use a second request only when the first answer creates the next state or question. +- Evaluate four layers separately: typed response/schema validity, semantic agreement with labeled references or human review, probability/confidence calibration, and deterministic downstream policy behavior. +- Calibrate thresholds on labeled development data for each action and risk level. Keep holdout data untouched. Do not reuse thresholds across primitives or assume `Noul` probability equals `Choice`/`Score` confidence. +- Preserve the pinned Jev model version, question-set version, primitive, criteria, probabilities, confidence or probability, threshold, fallback, and action-policy result. +- Keep arithmetic, dates, authorization, structural invariants, and side effects in code. Jev is not an agent or a generation evaluator; test the complete workflow that consumes its answer. + +Typed output prevents parsing errors, not semantic errors. A Jev-based evaluator still needs known-good, known-bad, boundary, adversarial, and ambiguous cases, plus human calibration when the release decision is consequential. + ## 5. Experiment design Record: diff --git a/skills/negotiation-maximizer/SKILL.md b/skills/negotiation-maximizer/SKILL.md new file mode 100644 index 0000000..7461469 --- /dev/null +++ b/skills/negotiation-maximizer/SKILL.md @@ -0,0 +1,156 @@ +--- +name: negotiation-maximizer +description: Use when the user wants to negotiate a price, rate, compensation, contract, purchase, hotel, service, vendor deal, discount, upgrade, fee, renewal, refund, or other commercial terms; asks what to say to improve leverage or get a better deal; provides an offer/counteroffer and wants a response; or wants to maximize coupons, direct-booking value, bundles, competing configurations, or all-in economics. Do not trigger for ordinary writing, simple price lookup/comparison, arithmetic, generic persuasion, or conflict mediation without a concrete negotiated outcome. +compatibility: Claude Code, Gemini CLI, OpenAI Codex, and other Agent Skills-compatible runtimes +--- + +# Negotiation Maximizer + +## Objective + +Maximize expected deal value while preserving credibility, optionality, and relationship quality. + +Optimize for better economic and non-price terms, lower hidden cost, stronger fallback options, and minimum visible negotiation theater. + +**Core rule: maximize leverage; minimize visible tactics.** + +The final message should sound like a normal informed buyer or professional making a decision—not someone performing negotiation tactics. + +## Default voice + +- Casual, concise, direct. +- Maximize information-to-ink ratio. +- State exact requirements, dates, quantities, configurations, and decision criteria. +- Prefer one strong lever over a laundry list of tactics. +- Keep alternatives credible and usually implicit. +- Never reveal maximum willingness to pay unless strategically necessary. +- Make the counterparty move first when that improves information or price discovery. +- Ask for all-in / including-tax / including-fee pricing when relevant. +- Signal ability to close without sounding eager, desperate, or theatrical. +- Avoid corporate-sales phrasing, faux warmth, jargon, and obvious negotiation tricks. +- If a sentence can be removed without losing leverage or information, cut it. + +## Silent preparation + +Before drafting, determine: + +1. **Objective** — desired outcome. +2. **BATNA** — best credible alternative. +3. **Reservation point** — walk-away threshold, if known; keep private. +4. **Counterparty incentives** — revenue, occupancy, quota, retention, direct-channel economics, certainty, simplicity. +5. **Information gap** — what should they reveal before the user reveals more? +6. **Comparison basis** — normalize taxes, fees, configuration, quantity, cancellation, service level, and benefits. +7. **Leverage** — choose the strongest one or two points only. +8. **Sequencing** — decide whether to anchor, counter, or make them move first. +9. **Close path** — give them an easy way to say yes. + +Do not expose this analysis unless asked. + +## Strategy + +### Normalize the real economics +Compare total value, not headline price: taxes, mandatory fees, included benefits, cancellation, configuration, quantity, payment timing, quality, switching cost, and risk. + +### Protect BATNA and private information +Never invent competing offers. Do not disclose a weak BATNA. Reveal a strong alternative only when it improves the deal. Never anchor with the user's maximum. + +### Choose who anchors +Prefer making the counterparty move first when discount authority is unknown, unpublished pricing may exist, or the user lacks a strong market anchor. Anchor first when objective market data or a credible competing offer gives the user a strong reference point. + +### Use one calibrated problem-solving question +Prefer: +- “What can you do on the rate to make this the better option?” +- “What flexibility do you have on the total?” +- “How close can you get to the alternative?” +- “How can we structure this so it works at that level?” + +One good question beats several stacked questions. + +### Trade; do not donate +Make concessions conditional: “If you can do X, I can book/sign/move forward.” + +### Optimize configuration before squeezing price +Compare one large unit vs. two smaller units, room-only vs. package, monthly vs. annual, price vs. fee waiver/credit/upgrade, direct vs. marketplace, quantity tiers, or removal of low-value add-ons. + +### Use seller economics silently +Infer flexibility from occupancy/capacity, direct-channel commission savings, quotas, retention economics, marginal upgrade cost, perishable inventory, commitment size, and operational simplicity. Use this to choose the ask; do not lecture the seller about it. + +### Use loss aversion without theater +A credible preference is enough: “I’d rather book the two-bedroom if the numbers work.” Never invent deadlines or scarcity. + +## Default message architecture + +1. Brief context. +2. Exact desired outcome/configuration. +3. Strongest comparison or leverage. +4. One calibrated ask. +5. Easy path to close. + +## Deal-maximization layer + +Silently check whether value can come from direct booking, alternate configurations, quantity/length tiers, member or loyalty rates, public promos, targeted offers, card/portal benefits, package removal, fee waivers, credits, upgrades, refundable-vs-nonrefundable spread, price matching, retention offers, or timing. + +Do not dump these mechanisms into the message. Pick the strongest lever. + +Never recommend false eligibility, fabricated offers, account abuse, obvious pricing-error exploitation, or deception. + +## Response modes + +- **“What should I say?”** Return the finished message first; keep explanation minimal. +- **“What’s the strategy?”** Give strongest leverage, information to withhold, first ask, likely counter, and next move. +- **Offer/counteroffer provided:** normalize total economics, compare to BATNA, recommend accept/counter/wait. +- **Draft provided:** preserve voice and improve leverage with the fewest edits. +- **Maximize a deal:** identify alternate structures before assuming price is the only lever. + +## Ethics + +Never fabricate offers, eligibility, identities, authority, defects, scarcity, or deadlines. Never use coercion, harassment, impersonation, or reputational threats. Strong negotiation is compatible with truthfulness. + +## Make [Expert PhD Legend] Proud + +Before finalizing, pressure-test the strategy against the canonical thinkers who literally wrote the books practitioners still reference: + +- **Roger Fisher, William Ury, Bruce Patton — _Getting to Yes_**: BATNA protected; interests and objective criteria clear. +- **Howard Raiffa — _The Art and Science of Negotiation_**: alternatives, uncertainty, tradeoffs, and information value handled deliberately. +- **Max Bazerman — negotiation and behavioral decision research**: anchoring, bias, overconfidence, escalation, and judgment errors checked. +- **Deepak Malhotra — _Negotiation Genius_ / _Negotiating the Impossible_**: structural leverage, sequencing, hidden interests, and non-price value considered. +- **Robert Cialdini, PhD — _Influence_ / _Pre-Suasion_**: influence credible, ethical, and subtle—not manipulative. +- **Neil Rackham — _SPIN Selling_**: the seller-facing logic exposes the real problem and value gap rather than dumping arguments. +- **Chris Voss — _Never Split the Difference_**: a natural calibrated question or tactical-empathy move lets the counterparty solve the constraint without sounding scripted. +- **James K. Sebenius — 3-D Negotiation**: setup, sequencing, parties, scope, and deal design improved before bargaining harder. + +Final bar: +1. Fisher/Ury/Patton proud — BATNA protected. +2. Raiffa/Bazerman proud — economics normalized; biases checked. +3. Malhotra/Sebenius proud — structure optimized before squeezing price. +4. Cialdini proud — influence ethical and invisible. +5. Rackham proud — ask solves a real seller decision problem. +6. Voss proud — one calibrated question does more work than three arguments. +7. **User proud** — casual, concise, high information-to-ink ratio, no negotiation cosplay. + +If the draft impresses the experts but no longer sounds like the user, rewrite it. **Expert strategy stays invisible; user voice stays visible.** + +## Pre-Flight Deal Quality & Leverage Scorecard (DQLS) + +Evaluate candidate drafts silently against these gates before returning: + +### Hard Safety Gates (Binary: PASS / BLOCK) +- **Zero Ceiling Leakage**: Never discloses maximum willingness to pay, budget cap, or weak alternatives. +- **Zero Negotiation Cosplay**: Completely free of negotiation jargon, framework names, and theatrical bargaining speak (no "BATNA", "anchoring", "tactical empathy", or "hostage tactics"). +- **Strict Grounding & Credibility**: No fabricated counteroffers, fake deadlines, false authority, or fictitious competing quotes. + +### Strategic Quality Rubric (Target: >= 23 / 25) + +| Dimension | 1 - Critical Failure | 3 - Competent / Generic | 5 - Expert / Invisible Leverage | +| :--- | :--- | :--- | :--- | +| **Normalized Economics** | Fixates solely on headline rate. | Mentions taxes/fees but compares apples to oranges. | Compares true all-in total; tests alternate configurations (tiers, units, credits, direct-booking margin). | +| **Calibrated Burden** | Demands a flat discount or makes aggressive demands. | Asks multiple questions or vague favors. | Uses **exactly one** calibrated question that forces the counterparty to solve the pricing constraint. | +| **Conditional Trade** | Donates concessions for free or begs for price relief. | Mentions moving forward loosely without a firm condition. | Clean conditional trade: *"If you can do X, I can book/sign/close today."* | +| **Information-to-Ink** | Wordy, over-explains context, uses faux warmth/filler. | Standard professional email tone. | Maximum density: every word carries leverage; removes anything that doesn't advance the close. | +| **Frictionless Close** | Open-ended dead end with no clear next step. | Asks for a callback or generic follow-up. | One-click close path: gives the seller the simplest possible way to say "yes". | + +### Execution Gate +- **Score < 20 or any Hard Gate FAIL**: Rewrite immediately. +- **Score >= 23**: Deliver draft (message first; keep scorecard and strategy notes private unless requested). + +Read `references/source.md` when deeper rationale or attribution is needed. \ No newline at end of file diff --git a/skills/negotiation-maximizer/agents/openai.yaml b/skills/negotiation-maximizer/agents/openai.yaml new file mode 100644 index 0000000..1cba570 --- /dev/null +++ b/skills/negotiation-maximizer/agents/openai.yaml @@ -0,0 +1,7 @@ +interface: + display_name: "Negotiation Maximizer" + short_description: "Negotiate smarter while keeping tactics invisible" +default_prompt: "Use $negotiation-maximizer to improve this negotiation, identify leverage, and draft a concise message in my voice." + +policy: + allow_implicit_invocation: true diff --git a/skills/negotiation-maximizer/evals/cases.json b/skills/negotiation-maximizer/evals/cases.json new file mode 100644 index 0000000..a40db40 --- /dev/null +++ b/skills/negotiation-maximizer/evals/cases.json @@ -0,0 +1,45 @@ +{ + "skill": "negotiation-maximizer", + "version": 1, + "trigger_cases": [ + {"prompt": "Help me get this hotel to discount the two-bedroom suite.", "trigger": true}, + {"prompt": "Write a recruiter response; I want more comp without giving my floor.", "trigger": true}, + {"prompt": "They offered 15% off annual SaaS. What should I counter?", "trigger": true}, + {"prompt": "The 2BR costs more than two 1BRs. Help me use that.", "trigger": true}, + {"prompt": "Rewrite this so they name the discount first.", "trigger": true}, + {"prompt": "Convert 62,935 MXN to USD.", "trigger": false}, + {"prompt": "Summarize this hotel review.", "trigger": false}, + {"prompt": "Find current coupon codes for this store.", "trigger": false}, + {"prompt": "Rewrite this thank-you email.", "trigger": false}, + {"prompt": "What is the cancellation policy?", "trigger": false} + ], + "binary_cases": [ + { + "id": "protect-ceiling", + "prompt": "Draft a hotel negotiation email. My maximum is $650/night but I want them to quote first.", + "pass_rule": "The message asks the hotel to move first and does not disclose the user's $650 ceiling.", + "must": [["what can you do", "best rate", "flexibility"]], + "must_not": ["$650", "650/night", "my maximum"], + "pass_fixture": "What can you do on the all-in two-bedroom rate to make it the better option for the stay?", + "fail_fixture": "My maximum is $650 per night. Can you meet it?" + }, + { + "id": "all-in-economics", + "prompt": "Help me compare a direct hotel offer against another configuration with fees and taxes.", + "pass_rule": "The response normalizes the comparison to total/all-in economics before recommending a move.", + "must": [["all-in", "including tax", "total"], ["compare", "comparison"]], + "must_not": ["headline rate only"], + "pass_fixture": "Compare the two options on an all-in basis including tax and mandatory fees, then use the cheaper equivalent configuration as leverage.", + "fail_fixture": "Compare only the advertised nightly rates and ignore taxes and fees." + }, + { + "id": "invisible-tactics", + "prompt": "Make this negotiation email sound like me but stronger.", + "pass_rule": "The draft is concise and natural, uses at most one primary calibrated ask, and does not name negotiation frameworks or tactics.", + "must": [["what can you do", "what flexibility", "how close"]], + "must_not": ["BATNA", "anchoring strategy", "tactical empathy", "hostage negotiation"], + "pass_fixture": "What can you do on the two-bedroom rate, including tax, to make it the better value for the stay? If the numbers work, I'd prefer the two-bedroom and can book direct.", + "fail_fixture": "Using my BATNA and a calibrated-question hostage-negotiation tactic, I want to anchor you lower." + } + ] +} \ No newline at end of file diff --git a/skills/negotiation-maximizer/references/source.md b/skills/negotiation-maximizer/references/source.md new file mode 100644 index 0000000..6dcb953 --- /dev/null +++ b/skills/negotiation-maximizer/references/source.md @@ -0,0 +1,53 @@ +# Negotiation Maximizer — Evidence and Sources + +## Skill design and portability + +- OpenAI, Build skills / Skills guidance: skills use a `SKILL.md` manifest with concise descriptions and progressive disclosure; descriptions should clearly scope implicit invocation. + https://developers.openai.com/codex/build-skills + https://developers.openai.com/api/docs/guides/tools-skills + +- Agent Skills specification and description guidance: activation quality depends heavily on the skill description; test both positive and negative trigger prompts. + https://agentskills.io/ + https://agentskills.io/skill-creation/optimizing-descriptions + +- skills.sh: Agent Skills packages can be installed into multiple supported coding-agent environments from a shared skill source, including Claude Code, Codex, and Gemini CLI. + https://skills.sh/ + +## Negotiation foundations + +- Roger Fisher, William Ury, Bruce Patton — _Getting to Yes_: principled negotiation, BATNA, objective criteria, interests over positions. +- Harvard Program on Negotiation — BATNA and principled-negotiation guidance. + https://www.pon.harvard.edu/daily/batna/translate-your-batna-to-the-current-deal/ + https://www.pon.harvard.edu/daily/negotiation-skills-daily/principled-negotiation-focus-interests-create-value/ + +- Howard Raiffa — _The Art and Science of Negotiation_: analytical negotiation, decision theory, uncertainty, tradeoffs, and information. + +- Max H. Bazerman — negotiation and behavioral decision research: anchoring, judgment, bias, escalation, and decision quality. + +- Deepak Malhotra — _Negotiation Genius_ and _Negotiating the Impossible_: structural leverage, sequencing, hidden interests, and deal design. + +- James K. Sebenius — 3-D Negotiation: setup, sequencing, parties, scope, and architecture of the deal beyond table tactics. + +## Influence and selling + +- Robert Cialdini, PhD — _Influence_ and _Pre-Suasion_: evidence-based ethical influence principles. + +- Neil Rackham — _SPIN Selling_: research-derived consultative selling based on large-scale observation of sales calls; diagnose the buyer/seller decision problem instead of feature-dumping. + +- Chris Voss — _Never Split the Difference_: calibrated questions, labeling, tactical empathy, and inviting the counterpart to solve constraints. Use as practitioner communication mechanics, not as a claim that commercial bargaining is equivalent to hostage negotiation. + +## Deal / coupon maximization + +Treat price as one variable in a broader configuration problem: +- all-in cost +- taxes and fees +- bundles +- alternate room/product configurations +- direct-channel economics +- upgrades/credits +- cancellation terms +- commitment length +- quantity tiers +- retention economics + +Never recommend false eligibility, fabricated offers, coupon abuse, or deception. \ No newline at end of file diff --git a/skills/typesafe-ai/LICENSE b/skills/typesafe-ai/LICENSE new file mode 100644 index 0000000..8c73b41 --- /dev/null +++ b/skills/typesafe-ai/LICENSE @@ -0,0 +1,21 @@ +MIT License + +Copyright (c) 2026 TypeSafe AI + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. diff --git a/skills/typesafe-ai/SKILL.md b/skills/typesafe-ai/SKILL.md new file mode 100644 index 0000000..0109513 --- /dev/null +++ b/skills/typesafe-ai/SKILL.md @@ -0,0 +1,149 @@ +--- +name: typesafe-ai +license: MIT +description: > + Build AI-powered software with TypeSafe: small units of AI intelligence you + can use like programming primitives. Its System One models, including Jev, + turn natural language and application state into typed judgments and + probabilities that code can combine. Use when a feature needs programmable + common sense, when brainstorming what AI could make possible in an app, or + when an LLM prompt-and-parse step could become a structured decision. + Applications include routing, ranking, extraction, verification, and + interactive experiences; these are starting points, not the limits. + Read live docs and cookbooks to find useful patterns and discover new combinations. +--- + +# Build with TypeSafe + +TypeSafe makes units of AI intelligence usable like programming primitives: small +judgments you can compose into larger capabilities. Its **System One models** return +fast, focused judgments that software can consume directly. **Jev** is TypeSafe's +flagship and first System One model. It understands natural language and returns +typed answers and probabilities rather +than generating text or reasoning explanations. Code owns the workflow; the model +supplies programmable common sense where ordinary code needs semantic understanding. + +## Read the live docs + +**The live TypeSafe docs are the source of truth. Read them as part of the task.** +This skill gives direction; the docs carry current concepts, prompting guidance, +API contracts, SDK usage, models, limits, and worked examples. + +- Start with the [documentation index](https://docs.typesafe.ai/llms.txt) to discover + relevant pages and cookbooks. Use targeted reads rather than loading the entire site. +- Mintlify serves Markdown by appending `.md` to a page path, for example + [how to build with TypeSafe](https://docs.typesafe.ai/concepts/how-to-build-with-system-one.md). + Follow links from the index; convert extensionless documentation page links to + `.md` when useful. Resolve relative links against `https://docs.typesafe.ai`. +- Before writing an integration, read the current API or chosen SDK page and the + question guidance relevant to the design. For a new workflow, also inspect the + closest cookbook: it often shows a better decomposition than a generic classifier. +- If the index is unavailable, use the direct links below or the site's navigation. + If Markdown fetching fails, try the normal page. If live access is unavailable, + use available local docs or installed SDK types, state that limitation, and avoid + inventing version-dependent details. + +| Task | Start here; follow the relevant details | +| --- | --- | +| Understand the programming model | [System One](https://docs.typesafe.ai/concepts/system-one.md), [building guide](https://docs.typesafe.ai/concepts/how-to-build-with-system-one.md) | +| Explore what to build | [Use-case map](https://docs.typesafe.ai/concepts/use-case-map.md), then relevant cookbooks from the index | +| Prepare inputs and questions | [State](https://docs.typesafe.ai/concepts/state.md), [primitives](https://docs.typesafe.ai/primitives.md), then the chosen primitive's page | +| Decide how to handle uncertainty | [Confidence](https://docs.typesafe.ai/confidence.md) | +| Write API code | [HTTP API](https://docs.typesafe.ai/api.md), [Python SDK](https://docs.typesafe.ai/sdk/python.md), or [JavaScript SDK](https://docs.typesafe.ai/sdk/javascript.md) | +| Update an older integration | [Migration guide](https://docs.typesafe.ai/migrating-to-v1.md) and the installed SDK's current reference | + +## Find the useful shape + +Start from the behavior the user wants: what will the application show, select, +change, or hand off? Work backward to the judgments it needs. Keep known rules, +calculations, exact lookups, and execution in code. Preserve the user's chosen stack +and scope; add TypeSafe where semantic understanding helps. + +When brainstorming or choosing an architecture, consider more than classification. +The patterns below are starting points: combine primitives around the user's goal, +including ideas that do not fit an established recipe. + +- **Route and fill known arguments.** A request can select a handler and its typed + parameters. Ask useful branch-specific questions up front and consume only the + relevant answers. Explore [function calling](https://docs.typesafe.ai/cookbooks/function_calling.md) + and [speculative fan-out](https://docs.typesafe.ai/patterns/fan-out.md). +- **Select instead of generate.** Find candidate values or source spans in code, + use a judgment to select the intended one, then copy or normalize it. Code can + also assemble source text into a formatted document or reading guide. Explore + [value extraction](https://docs.typesafe.ai/cookbooks/pre_parsed_value_extraction_cookbook.md) + and [structure recovery](https://docs.typesafe.ai/cookbooks/autoformat.md). +- **Find and judge evidence.** Retrieve candidates, compare their relevance to a + query, and select useful context. Explore [reranking](https://docs.typesafe.ai/cookbooks/rerank_typesafe.md) + and [hierarchical classification](https://docs.typesafe.ai/cookbooks/hierarchical_classification.md). +- **Turn judgments into reusable data.** Score dimensions once, then let code or + user controls change weights, thresholds, rankings, and views. With labeled + outcomes, those signals can become classical ML features. Explore + [composite scoring](https://docs.typesafe.ai/patterns/composite-scoring.md) and + [feature discovery](https://docs.typesafe.ai/cookbooks/autoresearch_feature_discovery.md). +- **Verify and escalate.** Check specific claims or fields against their evidence; + send uncertain or failing cases to a person or reasoning model. Explore + [citation checks](https://docs.typesafe.ai/cookbooks/citation_check.md) and + [extraction cascades](https://docs.typesafe.ai/cookbooks/sde_cascade.md). +- **Respond to changing state.** Code can retain goals and observations while fresh + judgments guide the next bounded step. Keep inferred state distinct from observed + facts, and check freshness before applying a result to a changed situation. + +For open-ended requests, offer the few directions that best serve the user's goal +and recommend a starting point. For a concrete request, choose the relevant pattern +and build; a brainstorm is not a mandatory detour. + +## Design the judgments + +Choose by what the answer means, then read the relevant primitive page: + +| Need | Primitive | Important distinction | +| --- | --- | --- | +| One of a defined set | [Choice](https://docs.typesafe.ai/primitives/choice.md) | Picks one option; its distribution compares competing options | +| Whether a condition holds | [Noul](https://docs.typesafe.ai/primitives/noul.md) | Probability of yes; no separate confidence; use one per label when several may apply | +| Degree along a described dimension | [Score](https://docs.typesafe.ai/primitives/score.md) | Probability-weighted position on ordered levels; use comparable per-item Scores for graded ranking | + +Give each question enough relevant **state** to answer: source text, identities, +relationships, policies, and current facts. Prefer named JSON fields when context +has several parts. Put the judgment in **instructions** and define its possible +answers in **criteria**. Question IDs are for code and are not sent to the model; +include complete meaning in the question. Reference nested state with backticked +paths such as `ticket.messages[0].text`. + +Ask one narrow, coherent judgment per question. Split independently useful dimensions, +without destroying the relationship being judged. A bounded action selection or +contextual interpretation is valid; atomic does not mean literal fact extraction +or a one-sentence limit. Strings work for simple questions. Use structured objects +or arrays when definitions, contrasts, exclusions, or examples clarify instructions +or criteria. Score levels must describe concrete situations and stand on their own. + +Keep the needed answers available. Include a no-match outcome when nothing may fit; +use a separate presence judgment when it is independently useful. For source-value +selection, check candidate coverage: the model cannot choose an omitted value. + +## Compose and verify + +**Ask independent questions over the same state together**, including useful +speculative questions. They run in parallel and cannot see one another's answers. +State each speculative premise explicitly; code consumes the applicable answers. +A second request is warranted when an earlier answer is needed to fetch evidence, +construct new state, or determine the next options. Extra questions still use tokens; +measure actual request budgets, cost, and end-to-end latency. + +Use probabilities and confidence to guide behavior, with thresholds evaluated on +the user's data and consequences. Choice/Score confidence summarizes distribution +concentration, not overall workflow correctness or permission to act. A Noul near +0.5 means similar probability for yes and no, not medium intensity. Several +acceptable alternatives can also spread probability; low confidence need not +invalidate a harmless preference choice. Ignore uncertainty on unused branches. + +Keep policy explicit and raw judgments reusable. Weighted scores suit compensating +preferences; an “any serious violation” rule needs separate conditions. Changing a +weight or display filter need not rerun inference when evidence and question meanings +are unchanged. Typed output guarantees the interface, not truth. System One models +are trained for calibrated decisions; validate their performance in the target domain. + +Test representative cases and the resulting application behavior. For failures, +inspect the exact state, questions, candidates, answers, composition, and observed +outcome. Separate missing evidence, model errors, code errors, and service failures. +Treat cookbook thresholds and demo results as examples to evaluate, not universal +rules or permanent model limitations. Keep API credentials server-side in web apps. diff --git a/skills/typesafe-jev/SKILL.md b/skills/typesafe-jev/SKILL.md new file mode 100644 index 0000000..6ef7670 --- /dev/null +++ b/skills/typesafe-jev/SKILL.md @@ -0,0 +1,51 @@ +--- +name: typesafe-jev +description: "Design, review, and evaluate TypeSafe Jev workflows that turn text or JSON state into typed probabilistic decisions. Trigger: use for Jev primitives, routing, confidence gates, API integration, or Jev evals. Boundary: excludes prose generation, vision/audio/video, deterministic business rules, and unsupported claims about model correctness." +--- + +# TypeSafe Jev + +Use Jev as a fast, typed judgment component inside a code-owned workflow. Keep the product decision, authorization, arithmetic, dates, state changes, side effects, and user-facing prose outside Jev unless a separate system owns them. + +Read `references/jev.md` before implementation or when current model limits, pricing, SDK behavior, or privacy terms matter. + +## Start with the decision + +- State the actor, decision, allowed outputs, evidence, risk, fallback, and side effect. +- If the request is still an opportunity or feature idea, do not smuggle Jev into the solution. Reframe the problem as several HMWs across relevant lenses, then select exactly one primary HMW before defining the Jev contract. Use the `how-might-we` skill when available. +- Define what Jev may judge and what deterministic code must decide. High confidence is evidence, not permission. + +## Choose the smallest typed question + +- Use `Choice` for fixed classes, routes, or actions. Include `none`, `unknown`, or `other` when the domain allows them. +- Use `Score` for an ordered rubric with explicit anchors and boundaries. +- Use `Noul` for a single yes/no probability. It has probability but no confidence field. +- Split broad judgments into atomic questions. Batch independent questions in one request; use speculative fan-out when several candidate checks can run in parallel. Keep dependent questions in separate code-controlled steps. +- Ask literal, contrastive questions. Include only relevant state, criteria, examples, and boundary cases. Do not rely on Jev for counting, arithmetic, date comparison, code, structural invariants, or long irrelevant context. +- Preserve structure when it improves the judgment. `instructions` and `criteria` accept strings, objects, arrays, or `null`; use labeled JSON for multi-part questions, schemas, taxonomies, database rows, definitions, exclusions, and examples instead of flattening them into prose. +- Make boundaries explicit in structured `Choice` rubrics and structured `Noul` true/false criteria. State what each option covers and excludes. +- For deep taxonomies, walk one `Choice` level at a time in code. Pass the current node's children and enough subtree context, use probabilities to keep or beam-search close branches, and trim oversized subtrees before sending them. + +## Build the workflow + +1. Serialize only needed text or JSON state. Redact or omit personal, secret, and proprietary data unless the approved data path requires it. +2. Pin `jev-1.13.0` when thresholds or behavior are calibrated. Use aliases only for experiments; log the returned model version. +3. Call the official TypeSafe SDK or API from a server-side boundary. Keep the API key in the runtime secret store. +4. Validate the typed response, then apply deterministic rules in code. Route low-confidence or ambiguous results to review or a safe fallback. Require explicit authorization and confirmation for destructive or high-impact actions. +5. Add bounded timeouts, bounded exponential backoff for 429/529, `Retry-After` handling, and idempotency for retryable mutations. Never retry a side effect blindly. +6. Log model version, question-set version, outcome, probabilities/confidence, latency, request ID, and cost. Do not log raw sensitive state. + +## Evaluate before release + +- Preserve a baseline and compare Jev on the same versioned cases. +- Use labeled development, regression, and untouched holdout sets. Include normal, ambiguous, missing-context, adversarial, contradictory, multilingual, and context-rot cases. +- Measure per-question accuracy, calibration, coverage, false auto-action rate, false escalation rate, latency, and cost. Tune thresholds for the actual risk; do not transfer thresholds between primitives or datasets. +- Test semantic correctness separately from schema validity. A typed answer can still be wrong. +- Keep a generative model for replies, explanations, code, candidate creation, or other generation. Jev is not an agent and does not choose its next action. + +## Do not use Jev for + +- Generating prose, code, explanations, images, audio, or video. +- Authentication, authorization, payment/refund approval, policy enforcement, arithmetic, date/time logic, or irreversible state transitions. +- A single broad prompt where a small set of typed, independently testable questions would expose the decision boundary. +- Automatic action when uncertainty, missing context, adversarial input, or a critical regression lacks a safe fallback. diff --git a/skills/typesafe-jev/agents/openai.yaml b/skills/typesafe-jev/agents/openai.yaml new file mode 100644 index 0000000..3fe82df --- /dev/null +++ b/skills/typesafe-jev/agents/openai.yaml @@ -0,0 +1,4 @@ +interface: + display_name: "TypeSafe Jev" + short_description: "Design safe typed Jev workflows" + default_prompt: "Use $typesafe-jev to design or review this TypeSafe Jev workflow." diff --git a/skills/typesafe-jev/evals/cases.json b/skills/typesafe-jev/evals/cases.json new file mode 100644 index 0000000..afa3b36 --- /dev/null +++ b/skills/typesafe-jev/evals/cases.json @@ -0,0 +1,76 @@ +{ + "skill": "typesafe-jev", + "version": 1, + "trigger_cases": [ + {"prompt": "Design a Jev workflow to classify support intent and confidence-gate routing.", "trigger": true}, + {"prompt": "Evaluate Jev against our current model with a labeled holdout and calibration thresholds.", "trigger": true}, + {"prompt": "Add TypeSafe Jev to a backend for typed risk decisions.", "trigger": true}, + {"prompt": "Use structured Jev criteria to classify products through a deep taxonomy.", "trigger": true}, + {"prompt": "Generate a customer support reply in a warm tone.", "trigger": false}, + {"prompt": "Fix the CSS spacing on the settings page.", "trigger": false}, + {"prompt": "Compare two deterministic date-parser implementations.", "trigger": false} + ], + "binary_cases": [ + { + "id": "jev-decision-contract", + "prompt": "Design a safe Jev workflow for routing support requests.", + "pass_rule": "Defines the decision contract, chooses a typed primitive, keeps deterministic rules and side effects in code, and routes uncertainty safely.", + "must": [ + ["actor", "decision", "allowed outputs"], + ["Choice", "Score", "Noul"], + ["deterministic", "code"], + ["confidence", "probability"], + ["review", "fallback", "escalate"] + ], + "must_not": ["Jev generates the reply", "high confidence is permission", "Jev guarantees correctness"], + "pass_fixture": "Define the actor, decision, allowed outputs, risk, and fallback. Use Choice for fixed routes, keep authorization and side effects in deterministic code, and use probability or confidence to route uncertain cases to review. A separate generative model writes the reply.", + "fail_fixture": "Let Jev generate the reply and approve the route because high confidence is permission; Jev guarantees correctness." + }, + { + "id": "jev-evaluation-gate", + "prompt": "Decide whether a calibrated Jev classifier is ready for production auto-routing.", + "pass_rule": "Uses a baseline, versioned labeled slices, an untouched holdout, calibration, risk metrics, and a critical-regression gate.", + "must": [ + ["baseline", "comparison"], + ["labeled", "holdout"], + ["calibrat", "threshold"], + ["false", "auto-action"], + ["regression", "block"] + ], + "must_not": ["ship from aggregate score alone", "use every case for tuning", "schema validity proves semantic correctness"], + "pass_fixture": "Preserve a baseline and compare on versioned labeled normal, ambiguous, adversarial, and regression slices. Keep the holdout untouched, calibrate the threshold, measure false auto-actions, and block on a critical regression even if the aggregate improves. Schema validity does not prove semantic correctness.", + "fail_fixture": "Ship from aggregate score alone after tuning on every case; the schema is valid, so semantic correctness is proven." + }, + { + "id": "jev-boundary-and-reliability", + "prompt": "Integrate Jev into a server-side workflow with retries and privacy controls.", + "pass_rule": "Uses the official API boundary, secret storage, bounded retry behavior, version logging, and minimized sensitive state while excluding unsupported tasks.", + "must": [ + ["server-side", "official API"], + ["secret", "API key"], + ["429", "529", "backoff"], + ["model version", "question-set version"], + ["redact", "minimize"] + ], + "must_not": ["log raw sensitive state", "retry a side effect blindly", "use Jev for arithmetic"], + "pass_fixture": "Call the official API server-side with the API key in secret storage. Handle 429 and 529 with bounded backoff, log model version and question-set version, minimize and redact sensitive state, and keep arithmetic and side effects in code.", + "fail_fixture": "Put the API key in client code, log raw sensitive state, retry a side effect blindly, and use Jev for arithmetic." + }, + { + "id": "jev-structured-hierarchy", + "prompt": "Classify a product through a deep taxonomy with Jev.", + "pass_rule": "Uses structured instructions and criteria, explicit option boundaries, and a code-owned hierarchical walk that uses probabilities for close branches.", + "must": [ + ["structured", "JSON", "object"], + ["instructions", "criteria"], + ["cover", "exclude", "boundary"], + ["taxonomy", "hierarch", "tree"], + ["probabil", "beam", "multiple branches"], + ["code", "each level", "current node"] + ], + "must_not": ["flatten the taxonomy into one prose string", "let Jev walk the tree autonomously", "ignore close probabilities"], + "pass_fixture": "Keep instructions and criteria as structured JSON objects. Define each option's coverage, exclusions, and examples. Walk the taxonomy in code one level at a time from the current node, pass child subtrees, and retain multiple branches when probabilities are close. Trim oversized branches before inference.", + "fail_fixture": "Flatten the taxonomy into one prose string and let Jev walk the tree autonomously; ignore close probabilities and send the entire tree every time." + } + ] +} diff --git a/skills/typesafe-jev/references/jev.md b/skills/typesafe-jev/references/jev.md new file mode 100644 index 0000000..e0b16bb --- /dev/null +++ b/skills/typesafe-jev/references/jev.md @@ -0,0 +1,43 @@ +# TypeSafe Jev reference + +Checked 2026-09-18. Re-read the official pages before implementation because aliases, limits, pricing, SDKs, and legal terms can change. + +## Official model facts + +- [Introduction](https://docs.typesafe.ai/introduction): Jev maps text state plus typed questions to structured answers and probabilities; it does not generate text. +- [System One](https://docs.typesafe.ai/concepts/system-one): text-only input; calibrated probabilities are group-level behavior, not an individual correctness guarantee. +- [State](https://docs.typesafe.ai/concepts/state): state may be a string, JSON object, or array of text values. English is strongest; other languages and CJK require validation. +- [Primitives](https://docs.typesafe.ai/primitives): `Choice`, `Score`, and `Noul`; questions are independent and can be requested together. +- [Advanced primitive structure](https://docs.typesafe.ai/primitives/advanced): `instructions` and `criteria` accept structured JSON; structured rubrics clarify boundaries and support taxonomy traversal. +- [Choice](https://docs.typesafe.ai/primitives/choice): up to 255 options; use a second staged question for larger candidate sets. +- [Confidence](https://docs.typesafe.ai/confidence): confidence is derived from the distribution for `Choice` and `Score`; `Noul` exposes probability, not confidence. +- [Patterns](https://docs.typesafe.ai/patterns): speculative fan-out, confidence-gated routing, composite scoring, and intent routing. +- [How to build](https://docs.typesafe.ai/concepts/how-to-build-with-system-one): code owns workflow, rules, side effects, and next actions; Jev supplies comparable typed judgments. + +## Current API and operations + +- [API](https://docs.typesafe.ai/api): `POST https://api.typesafe.ai/v1/systemone` with bearer authentication, `state`, `model`, and `questions`. +- [Models](https://docs.typesafe.ai/models): current documented alias is `jev-latest` → `jev-1.13.0`; the page documents a 64k request limit, 32k state plus longest-question limit, text-only input, and current pricing/rate limits. Pin versions when calibrated. +- [Jev 1.13 limitations](https://docs.typesafe.ai/model-jaggedness/jev-1.13): weak areas include indirection, literal reading, math/counting, dates, irrelevant context, adversarial content, contradictory criteria, structural invariants, and generation. +- [JavaScript SDK](https://github.com/typesafe-ai/typesafe-sdk-js): official package is `@typesafe-ai/sdk`; Node 20+. +- [Python SDK](https://github.com/typesafe-ai/typesafe-sdk-python): official Python client. + +## Advanced primitive structure + +Every question field is an `EntryType`. These fields accept a string, object, array, or `null`: + +- `instructions` for `Choice`, `Score`, and `Noul` +- `Choice` option descriptions +- `Score` level descriptions +- `Noul` `criteria.true` and `criteria.false` + +Use structure when it makes the judgment clearer or carries supporting data. Pass a schema, taxonomy, database row, comparison list, or labeled definition as JSON instead of serializing it into a string template. A useful instruction object can separate the question, fields to compare, and focus. + +For `Choice`, structured option descriptions can include coverage, exclusions, and examples to sharpen boundaries. For subtle `Noul` decisions, structured true/false criteria can define both sides. + +For deep taxonomies, walk the tree in code: ask one `Choice` for the current level, provide its child options plus enough subtree context, then continue with the selected child. If probabilities are close, keep multiple candidate paths or beam-search them. Trim oversized branches to direct children and a representative leaf sample. + +## Vendor evals and policy + +- [Workflow evals](https://evals.typesafe.ai/): vendor-published comparisons across security incidents, agent trace observability, invoice processing, and customer service. Reference labels use an average of GPT-6 Astra and Claude Fable 5.1; treat results as directional, not independent ground truth. +- [Legal](https://docs.typesafe.ai/legal) and [privacy policy](https://typesafe.ai/legal/privacy-policy): TypeSafe states that customer requests and responses are not used for training; verify retention, DPA, regional hosting, and enterprise zero-data-retention terms for the deployment.