Skip to content

Implement 2-pass summarization with JSON extraction and formatting - #14

Merged
ivgtr merged 2 commits into
mainfrom
claude/stabilize-llm-output-qRz1w
Mar 10, 2026
Merged

ivgtr merged 2 commits into
mainfrom
claude/stabilize-llm-output-qRz1w

Conversation

@ivgtr

@ivgtr ivgtr commented Mar 10, 2026

Copy link
Copy Markdown
Owner

Summary

Implement a 2-pass summarization pipeline for TDnet disclosure documents that separates information extraction (pass 1) from output formatting (pass 2). This improves output quality and consistency by using specialized prompts and lower temperature settings for formatting.

Key Changes

New Files

  • src/lib/summary-schema.ts: Defines TypeScript interfaces and JSON schema strings for each document type (earnings, earnings revision, dividend, M&A, share repurchase, other). Includes document-type-specific extraction rules and rating criteria for earnings reports.
  • src/lib/format-prompts.ts: Provides pass 2 (formatting) prompts that convert extracted JSON data into structured text output using document-type-specific templates.

Modified Files

  • src/background/index.ts:

    • Added twoPassMode setting (defaults to true)
    • Implemented summarizeTwoPass() function that executes pass 1 (extraction) and pass 2 (formatting) sequentially
    • Implemented summarizeOnePass() function for backward compatibility
    • Added tryParseJson() helper to robustly extract JSON from LLM responses with fallback handling
    • Routes summarization through 2-pass or 1-pass pipeline based on settings
  • src/lib/prompts.ts:

    • Added getExtractionPrompt() function for pass 1 (information extraction)
    • Defined extraction role, rules, and document-type-specific guidance
    • Integrated schema definitions and rating rules from summary-schema.ts
  • src/lib/llm-client.ts:

    • Added optional temperature parameter to LLMConfig interface for fine-grained control over generation behavior
  • src/options/Options.tsx:

    • Added UI toggle for 2-pass vs 1-pass summarization mode
    • Persists twoPassMode setting to Chrome storage
    • Includes explanatory text for each mode

Implementation Details

  • Pass 1 (Extraction): Uses specialized prompts to extract structured information into JSON format, with strict rules against fabrication and emphasis on numerical accuracy
  • Pass 2 (Formatting): Converts JSON to formatted text using document-type-specific templates with lower temperature (0.3) for consistency
  • Fallback Handling: If JSON parsing fails in pass 2, the system gracefully falls back to using pass 1 output directly
  • Earnings-Specific Logic: Handles quarterly vs full-year contexts with different evaluation criteria and progress metrics
  • Topics Extraction: Emphasizes extracting segment-level insights, forward indicators, and one-time factors with specific numerical data

https://claude.ai/code/session_01KrEGwXcD4TS32dd9LvEA67

claude added 2 commits March 8, 2026 09:53
情報抽出(JSON)→フォーマット整形の2段階でLLM出力の安定性を向上。
性能の低いモデルでも一貫したレイアウトの要約結果を得られる。

- パス1: 文書タイプ別JSONスキーマで構造化情報を抽出
- パス2: 低temperature(0.3)で固定テンプレートに沿って整形
- JSON解析失敗時は1パスフォールバック
- 設定UIで2パス/1パスの切替が可能(デフォルトON)
- トピックス抽出でセグメント別業績・受注増減等の詳細情報を重視

https://claude.ai/code/session_01KrEGwXcD4TS32dd9LvEA67
@ivgtr
ivgtr merged commit 4d711eb into main Mar 10, 2026
@ivgtr
ivgtr deleted the claude/stabilize-llm-output-qRz1w branch March 10, 2026 00:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants