Skip to content

Refine earnings extraction rules for null handling and range forecasts - #16

Merged
ivgtr merged 3 commits into
mainfrom
claude/improve-prompt-nulls-jLWYv
Mar 12, 2026
Merged

ivgtr merged 3 commits into
mainfrom
claude/improve-prompt-nulls-jLWYv

Conversation

@ivgtr

@ivgtr ivgtr commented Mar 12, 2026

Copy link
Copy Markdown
Owner

Summary

This PR clarifies and strengthens the rules for handling missing data, null values, and range-format forecasts in earnings report extraction and formatting. The changes ensure consistent behavior when information is absent from source documents and improve handling of forecast ranges.

Key Changes

Null/Missing Data Handling

  • Explicit null handling: Updated rules to clarify that missing items should be omitted entirely rather than marked with placeholder text like "null", "本文に記載なし", or "該当なし"
  • JSON null values: Specified that JSON extraction should use proper null values (not string "null") for missing change rates and dividend fields
  • Format rules: Added dedicated section in FORMAT_RULES for null/missing value processing with clear guidance on when to omit sections, rows, or just parenthetical content

Range Forecast Handling

  • Progress rate calculation: Added explicit instructions for handling range-format forecasts (e.g., 3,706〜4,097百万円) by calculating progress rates for both lower and upper bounds in "XX.X%〜YY.Y%" format
  • Rating distinction: Clarified that range forecasts are NOT considered "未定" (undefined) and should be rated using the midpoint value
  • Extraction notes: Enhanced guidance to ensure range forecasts are always calculated when numeric values exist

Improved Clarity

  • Forecast status: Changed "未定・ゼロ・赤字" to specifically "未定と明記" (explicitly stated as undefined) to avoid false positives
  • Comparison rates: Added guidance for handling missing year-over-year/period-over-period rates (e.g., for newly consolidated entities or IPOs) with "★—(前年同期比なし)" rating
  • Progress section: Clarified that progress section should be omitted entirely when forecast is undefined, with optional omission of prior-year comparison if unavailable
  • Dividend fields: Specified that dividend items should only appear if explicitly stated in source document

Schema Documentation

  • Updated JSON schema comments to reflect null handling for:
    • progress object (null when forecast is "未定")
    • change fields in performance and forecast items (null when rate not in source)
    • All dividend fields (null when not mentioned)

Implementation Details

  • Changes maintain backward compatibility with existing extraction logic
  • Formatting rules now have explicit priority guidance for null value handling
  • All modifications are in prompt/rule definitions; no core extraction logic changes

https://claude.ai/code/session_01HZQy3vHFiwbjuiHvhBoiwR

claude added 3 commits March 12, 2026 06:41
- FORMAT_RULES: null値の項目レベル省略ルールを追加(セクション省略だけでなく行単位の省略を明示)
- テンプレート: items内のchangeがnullの場合に括弧ごと省略する注釈を追加
- 修正・配当セクション: 配当項目が本文にない場合は行ごと省略する指示を追加
- COMMON_RULES: 増減率が本文にない場合は括弧部分ごと省略する指示を追加
- EXTRACTION_RULES: null設定のガイダンスをより具体的に(JSON null vs 文字列"null"の区別)
- スキーマ: 各フィールドにnull条件の説明コメントを追加

https://claude.ai/code/session_01HZQy3vHFiwbjuiHvhBoiwR
- レンジ予想(例: 3,706〜4,097百万円)を「未定」と誤判定する問題を修正
  - 進捗率: レンジ予想の場合は下限・上限で算出し「XX.X%〜YY.Y%」形式に
  - 評価ルール: レンジ予想は中央値で判定、「未定」扱いしない指示を追加
- 前年同期比なし(初連結化等)の場合の評価を「★—(前年同期比なし)」に明確化
- 1パスモード・2パスモード両方の判定ルールを統一的に修正

https://claude.ai/code/session_01HZQy3vHFiwbjuiHvhBoiwR
@ivgtr
ivgtr merged commit d79f15f into main Mar 12, 2026
@ivgtr
ivgtr deleted the claude/improve-prompt-nulls-jLWYv branch March 12, 2026 07:07

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: bcb6c31bb1

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/lib/summary-schema.ts
const progressOrLanding = isQuarterly
? `■ 進捗: 経常利益の進捗率 − 標準進捗率(${standardRate}%)の差で判定
★5: +10pt以上 / ★4: +5〜+10pt / ★3: ±5pt / ★2: △5〜△10pt / ★1: △10pt超 / 通期予想が未定・ゼロ・赤字→「★—」`
★5: +10pt以上 / ★4: +5〜+10pt / ★3: ±5pt / ★2: △5〜△10pt / ★1: △10pt超 / 通期予想が「未定」と明記→「★—」

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Treat zero/negative forecasts as non-ratable progress cases

This rule now reserves ★— only for forecasts explicitly marked as 未定, but the same earnings prompts still instruct 算出不可 when the full-year forecast is zero or negative (where progress-rate scoring is undefined). For quarterly disclosures with zero/赤字通期予想, the model receives conflicting instructions and can emit a normal star rating instead of an unratable result, which directly distorts the user-facing evaluation output.

Useful? React with 👍 / 👎.

Comment thread src/lib/format-prompts.ts

## 修正・配当
- 業績予想の修正: {{revision}}
※以下のdividendフィールドがnullの項目は行ごと省略すること:

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Remove internal dividend-null note from output template

This newly added plain-text instruction is inside the emitted template body, so the formatter is likely to reproduce it in end-user summaries as literal text. Because it is an internal prompting directive rather than disclosure content, it degrades output quality and readability whenever the earnings template is used.

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants