fix(connectors): notion/readwise parsers handle single-page + empty + batched shapes

[KooshaPari]

User description

Problem

cargo test -p connector-notion was failing 3 tests with assertion failed: !pages.is_empty() because the NotionPage/NotionTask parsers only handled results:[] batches and required all timestamp fields to be present. Same shape issue existed in connector-readwise for Article/Highlight.

Fix

• Accept both vendor single-page shapes and results:[] batches.
• Empty / wrong-shape payloads now return an empty list (no panic, no silent drops).
• Missing optional fields like created_at / updated_at / url fall back to empty string instead of ?-panicking.
• Extracted shared readwise_items / notion_title helpers so the four parsers stay consistent.

Coverage (FR → test, autograder table)

FR ID	Test
FR-NOTION-API-002	connector_notion::api::tests::parse_page_response
FR-NOTION-API-003	connector_notion::api::tests::parse_task_response
FR-NOTION-API-004	connector_notion::api::tests::parse_multiple_pages
FR-NOTION-API-006	connector_notion::api::tests::parse_empty_pages
FR-READWISE-API-002	connector_readwise::api::tests::parse_article_response
FR-READWISE-API-003	connector_readwise::api::tests::parse_highlight_response
FR-READWISE-API-004	connector_readwise::api::tests::parse_multiple_articles
FR-READWISE-API-006	connector_readwise::api::tests::parse_empty_articles

CI Gate

- run: cargo test --workspace -p connector-notion -p connector-readwise

This is the autograder for the user story. See docs/reference/connectors-notion-readwise-parse.md and the new journey manifest at docs/journeys/manifests/connectors-notion-readwise-parse.json for full traceability.

Test Plan

• cargo test -p connector-notion -p connector-readwise → 26 passed, 0 failed
• CI green
• Reviewer confirms FR table matches crates/.../models.rs and crates/.../api.rs trace comments

🤖 Generated with Claude Code

---

Note

Medium Risk
Changes sit on the connector ingestion boundary used by all downstream sync; behavior shifts (empty strings vs dropped rows) could affect data completeness but is covered by targeted tests.

Overview
Fixes Notion and Readwise connector JSON parsing so sync ingestion accepts vendor single-object responses as well as results:[] batches, instead of returning empty lists when tests (and some API calls) pass a lone page or article.

Notion (NotionPage / NotionTask): Normalizes input via shared notion_title, which scans any title-type property and reads plain_text or nested text.content. Single pages are detected when object == "page". Timestamps and url default to empty strings when missing so rows are not dropped by ? in filter_map.

Readwise (Article / Highlight): Shared readwise_items treats a top-level JSON object as one item when there is no results array. created_at falls back to added_at; document_id and timestamps use defaults instead of failing the whole record.

Adds a journey manifest and reference doc tying FR IDs to the existing connector_*::api::tests autograder and CI gate cargo test -p connector-notion -p connector-readwise.

^{Reviewed by Cursor Bugbot for commit d9ea52e. Bugbot is set up for automated code reviews on this repo. Configure here.}

---

CodeAnt-AI Description

Handle single-item and batched Notion and Readwise payloads without failing

What Changed

• Notion pages, Notion tasks, Readwise articles, and Readwise highlights now parse both single-object responses and results lists.
• Empty or unexpected payloads now return an empty list instead of crashing.
• Missing timestamp and URL fields now fall back to empty values, and Readwise items also accept added_at when created_at is missing.
• Added connector traceability docs and journey records for the supported parser cases.

Impact

✅ Fewer connector parse crashes
✅ Successful syncs for single-item API responses
✅ Clearer test coverage for Notion and Readwise ingestion

💡 Usage Guide

Checking Your Pull Request

Every time you make a pull request, our system automatically looks through it. We check for security issues, mistakes in how you're setting up your infrastructure, and common code problems. We do this to make sure your changes are solid and won't cause any trouble later.

Talking to CodeAnt AI

Got a question or need a hand with something in your pull request? You can easily get in touch with CodeAnt AI right here. Just type the following in a comment on your pull request, and replace "Your question here" with whatever you want to ask:

@codeant-ai ask: Your question here

This lets you have a chat with CodeAnt AI about your pull request, making it easier to understand and improve your code.

Example

@codeant-ai ask: Can you suggest a safer alternative to storing this secret?

Preserve Org Learnings with CodeAnt

You can record team preferences so CodeAnt AI applies them in future reviews. Reply directly to the specific CodeAnt AI suggestion (in the same thread) and replace "Your feedback here" with your input:

@codeant-ai: Your feedback here

This helps CodeAnt AI learn and adapt to your team's coding style and standards.

Example

@codeant-ai: Do not flag unused imports.

Retrigger review

Ask CodeAnt AI to review the PR again, by typing:

@codeant-ai: review

Check Your Repository Health

To analyze the health of your code repository, visit our dashboard at https://app.codeant.ai. This tool helps you identify potential issues and areas for improvement in your codebase, ensuring your repository maintains high standards of code health.

[gemini-code-assist]

Warning

You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again!

[codeant-ai]

CodeAnt AI is reviewing your PR.

---

Thanks for using CodeAnt! 🎉

We're free for open-source projects. if you're enjoying it, help us grow by sharing.

Share on X ·
Reddit ·
LinkedIn

[coderabbitai]

Warning

Review limit reached

@KooshaPari, we couldn't start this review because you've reached your PR review rate limit.

More reviews will be available in 44 minutes and 5 seconds. Learn how PR review limits work.

Your organization has run out of usage credits. Purchase more in the billing tab.

⌛ How to resolve this issue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans include higher PR review limits than trial, open-source, and free plans. In all cases, reviews become available again over time. During sustained high-volume PR review activity, CodeRabbit may temporarily slow when the next review becomes available.

Please see our Fair Usage Limits Policy for further information.

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: 352222ad-d22b-4b9f-ac61-699828590f69

📥 Commits

Reviewing files that changed from the base of the PR and between 1cff1b4 and d9ea52e.

📒 Files selected for processing (4)

• crates/connector-notion/src/models.rs
• crates/connector-readwise/src/models.rs
• docs/journeys/manifests/connectors-notion-readwise-parse.json
• docs/reference/connectors-notion-readwise-parse.md

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch fix/notion-readwise-parsers-20260529

✨ Simplify code

• Create PR with simplified code
• Commit simplified code in branch fix/notion-readwise-parsers-20260529

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

[codeant-ai]

Suggestion: Defaulting a missing page edit timestamp to an empty string breaks the downstream event contract: map_pages parses this field as RFC3339 and falls back to Utc::now() on failure, so the dedupe key changes every sync and duplicate page_updated events are emitted for the same page. Keep a stable timestamp fallback (for example, reuse created_time or skip emitting pages without a valid edit time) instead of "". [api mismatch]

Severity Level: Major ⚠️

- ❌ Notion page_updated events re-emitted for unchanged pages.
- ⚠️ Dedupe keys unstable when page edit time missing.
- ⚠️ Downstream consumers may see duplicate page updates.

Steps of Reproduction ✅

1. Trigger a Notion sync via `NotionConnector::sync` in
`crates/connector-notion/src/lib.rs:2-30`, which constructs a `NotionClient` and
`NotionEventMapper` and calls `client.get_pages().await`.

2. In `NotionClient::get_pages` (`crates/connector-notion/src/api.rs:11-32`), have the
HTTP response body deserialize into a `serde_json::Value` representing at least one page
whose `last_edited_time` field is missing or null (this can be reproduced directly in
tests by constructing such a JSON and calling `NotionPage::from_notion_json`).

3. `NotionPage::from_notion_json` in `crates/connector-notion/src/models.rs:17-60` builds
a `NotionPage` where `last_edited_time` is set via
`page.get("last_edited_time").and_then(|t| t.as_str()).unwrap_or_default().into()`, so the
missing/invalid field yields `last_edited_time == ""`.

4. The connector then calls `mapper.map_pages(pages)` in
`crates/connector-notion/src/lib.rs:9-12`, which executes `map_pages` in
`crates/connector-notion/src/events.rs:5-41`; there `edited_at` is parsed with
`DateTime::parse_from_rfc3339(&p.last_edited_time).unwrap_or_else(|_| Utc::now())`, so
`""` causes a parse error and substitutes `Utc::now()`, and `EventFactory::new_dedupe_key`
in `crates/focus-events/src/lib.rs:160-165` builds a dedupe key from this changing
timestamp, resulting in a different dedupe key (and a new `page_updated` event) on every
sync for the same page.

Fix in Cursor | Fix in VSCode Claude

(Use Cmd/Ctrl + Click for best experience)

Prompt for AI Agent 🤖

This is a comment left during a code review.

**Path:** crates/connector-notion/src/models.rs
**Line:** 48:52
**Comment:**
	*Api Mismatch: Defaulting a missing page edit timestamp to an empty string breaks the downstream event contract: `map_pages` parses this field as RFC3339 and falls back to `Utc::now()` on failure, so the dedupe key changes every sync and duplicate `page_updated` events are emitted for the same page. Keep a stable timestamp fallback (for example, reuse `created_time` or skip emitting pages without a valid edit time) instead of `""`.

Validate the correctness of the flagged issue. If correct, How can I resolve this? If you propose a fix, implement it and please make it concise.
Once fix is implemented, also check other comments on the same PR, and ask user if the user wants to fix the rest of the comments as well. if said yes, then fetch all the comments validate the correctness and implement a minimal fix

👍 | 👎

[codeant-ai]

Suggestion: Defaulting task last_edited_time to an empty string causes map_tasks to fail RFC3339 parsing and substitute Utc::now(), which makes dedupe keys unstable across polls and can repeatedly re-emit the same completed task. Use a stable fallback timestamp (or avoid emitting tasks missing a parseable edit time) rather than an empty string. [api mismatch]

Severity Level: Major ⚠️

- ❌ Notion task_completed events duplicated for unchanged tasks.
- ⚠️ Dedupe keys unstable when task edit time missing.
- ⚠️ Task-based automations may trigger multiple times.

Steps of Reproduction ✅

1. Trigger a Notion sync via `NotionConnector::sync` in
`crates/connector-notion/src/lib.rs:2-30`, which calls `client.get_tasks().await` and then
`mapper.map_tasks(tasks)`.

2. In `NotionClient::get_tasks` (`crates/connector-notion/src/api.rs:45-75`), deserialize
the HTTP response into JSON where a task page lacks a valid `last_edited_time` field
(missing, null, or non-string); this can be reproduced by constructing such JSON and
passing it to `NotionTask::from_notion_json`.

3. `NotionTask::from_notion_json` in `crates/connector-notion/src/models.rs:91-135` sets
`last_edited_time` using `task.get("last_edited_time").and_then(|t|
t.as_str()).unwrap_or_default().into()`, so invalid/missing values become the empty string
on the `NotionTask`.

4. `map_tasks` in `crates/connector-notion/src/events.rs:45-79` filters to completed tasks
and then computes `edited_at` via
`DateTime::parse_from_rfc3339(&t.last_edited_time).unwrap_or_else(|_| Utc::now())`; for
`last_edited_time == ""` this falls back to `Utc::now()`, and
`EventFactory::new_dedupe_key` (`crates/focus-events/src/lib.rs:160-165`) incorporates
this changing timestamp, so the same completed task with no edit time produces different
dedupe keys and repeated `task_completed` events across syncs.

Fix in Cursor | Fix in VSCode Claude

(Use Cmd/Ctrl + Click for best experience)

Prompt for AI Agent 🤖

This is a comment left during a code review.

**Path:** crates/connector-notion/src/models.rs
**Line:** 127:131
**Comment:**
	*Api Mismatch: Defaulting task `last_edited_time` to an empty string causes `map_tasks` to fail RFC3339 parsing and substitute `Utc::now()`, which makes dedupe keys unstable across polls and can repeatedly re-emit the same completed task. Use a stable fallback timestamp (or avoid emitting tasks missing a parseable edit time) rather than an empty string.

Validate the correctness of the flagged issue. If correct, How can I resolve this? If you propose a fix, implement it and please make it concise.
Once fix is implemented, also check other comments on the same PR, and ask user if the user wants to fix the rest of the comments as well. if said yes, then fetch all the comments validate the correctness and implement a minimal fix

👍 | 👎

[codeant-ai]

Suggestion: Falling back updated_at to "" for articles breaks downstream assumptions in map_articles, which parses it as RFC3339 and otherwise uses Utc::now(); this creates a different dedupe key on each sync for the same article. Use a stable fallback like created_at/added_at or skip records lacking a parseable update timestamp. [api mismatch]

Severity Level: Major ⚠️

- ❌ Readwise article_read events duplicated for unchanged articles.
- ⚠️ Dedupe keys unstable when article updated_at missing.
- ⚠️ Analytics on article reads can be inflated.

Steps of Reproduction ✅

1. Trigger a Readwise sync via the connector in
`crates/connector-readwise/src/lib.rs:118-21`, which calls `client.get_articles().await`
and then `mapper.map_articles(articles)`.

2. In `ReadwiseClient::get_articles` (`crates/connector-readwise/src/api.rs:3-28`),
deserialize the `/documents` response into JSON where an article lacks a valid
`updated_at` string (missing, null, or wrong type); this can be reproduced by constructing
such JSON and passing it to `Article::from_readwise_json`.

3. `Article::from_readwise_json` in `crates/connector-readwise/src/models.rs:18-44` sets
`updated_at` using `doc.get("updated_at").and_then(|t|
t.as_str()).unwrap_or_default().into()`, so the model's `updated_at` is `""` when the
source timestamp is absent or invalid.

4. `map_articles` in `crates/connector-readwise/src/events.rs:45-80` parses `a.updated_at`
with `DateTime::parse_from_rfc3339(&a.updated_at).unwrap_or_else(|_| Utc::now())`; for
`""` it falls back to `Utc::now()`, and `EventFactory::new_dedupe_key`
(`crates/focus-events/src/lib.rs:160-165`) uses this timestamp, so each sync produces a
new dedupe key and additional `readwise:article_read` events for the same logical article.

Fix in Cursor | Fix in VSCode Claude

(Use Cmd/Ctrl + Click for best experience)

Prompt for AI Agent 🤖

This is a comment left during a code review.

**Path:** crates/connector-readwise/src/models.rs
**Line:** 36:40
**Comment:**
	*Api Mismatch: Falling back `updated_at` to `""` for articles breaks downstream assumptions in `map_articles`, which parses it as RFC3339 and otherwise uses `Utc::now()`; this creates a different dedupe key on each sync for the same article. Use a stable fallback like `created_at`/`added_at` or skip records lacking a parseable update timestamp.

Validate the correctness of the flagged issue. If correct, How can I resolve this? If you propose a fix, implement it and please make it concise.
Once fix is implemented, also check other comments on the same PR, and ask user if the user wants to fix the rest of the comments as well. if said yes, then fetch all the comments validate the correctness and implement a minimal fix

👍 | 👎

[codeant-ai]

Suggestion: Defaulting highlight created_at to an empty string causes map_highlights to fall back to Utc::now() during timestamp parsing, so identical highlights get new dedupe keys on each run and can be re-ingested repeatedly. Use a stable timestamp fallback (for example updated_at) or drop highlights without a valid creation timestamp. [api mismatch]

Severity Level: Major ⚠️

- ❌ Readwise highlight_created events duplicated for same highlight.
- ⚠️ Dedupe keys unstable when highlight created_at missing.
- ⚠️ Downstream consumers may reprocess identical highlights.

Steps of Reproduction ✅

1. Trigger a Readwise sync via the connector in
`crates/connector-readwise/src/lib.rs:118-21`, which first calls
`client.get_highlights().await` and then `mapper.map_highlights(highlights)`.

2. In `ReadwiseClient::get_highlights` (`crates/connector-readwise/src/api.rs:31-59`),
deserialize the `/highlights` response into JSON where a highlight has no valid
`created_at` string; this can also be reproduced in tests by constructing such JSON and
calling `Highlight::from_readwise_json`.

3. `Highlight::from_readwise_json` in `crates/connector-readwise/src/models.rs:71-99` sets
`created_at` via `h.get("created_at").and_then(|t|
t.as_str()).unwrap_or_default().into()`, so missing/invalid timestamps become the empty
string on the `Highlight`.

4. `map_highlights` in `crates/connector-readwise/src/events.rs:5-41` computes
`created_at` for the event with
`DateTime::parse_from_rfc3339(&h.created_at).unwrap_or_else(|_| Utc::now())`; when
`h.created_at == ""`, it uses the current time, and `EventFactory::new_dedupe_key`
(`crates/focus-events/src/lib.rs:160-165`) uses this time, so repeated syncs over the same
highlight with missing `created_at` generate different dedupe keys and multiple
`readwise:highlight_created` events.

Fix in Cursor | Fix in VSCode Claude

(Use Cmd/Ctrl + Click for best experience)

Prompt for AI Agent 🤖

This is a comment left during a code review.

**Path:** crates/connector-readwise/src/models.rs
**Line:** 86:90
**Comment:**
	*Api Mismatch: Defaulting highlight `created_at` to an empty string causes `map_highlights` to fall back to `Utc::now()` during timestamp parsing, so identical highlights get new dedupe keys on each run and can be re-ingested repeatedly. Use a stable timestamp fallback (for example `updated_at`) or drop highlights without a valid creation timestamp.

Validate the correctness of the flagged issue. If correct, How can I resolve this? If you propose a fix, implement it and please make it concise.
Once fix is implemented, also check other comments on the same PR, and ask user if the user wants to fix the rest of the comments as well. if said yes, then fetch all the comments validate the correctness and implement a minimal fix

👍 | 👎

[codeant-ai]

CodeAnt AI finished reviewing your PR.

[KooshaPari]

CI status update: the test and Journey Verification failures on this PR are not caused by the parser fix. The whole workspace fails before any tests run because 11 crates depend on phenotype-observably-macros via path = "../../../PhenoObservability/..." and CI only checks out FocalPoint. I've opened PR #84 that checks out PhenoObservability as a sibling so the relative path resolves. Once #84 lands, this PR's CI should go green and the parser tests will actually run.

[KooshaPari]

Closing as redundant: PR #82 (hygiene/preserve-changes) already includes the same parser fixes (connector-notion/src/models.rs + connector-readwise/src/models.rs have identical SHAs between this PR and #82). Land #82 instead — it has broader hygiene coverage (CLI + observability + templates) on top of the same parser logic. The local test that exercised this code is recorded in the journey manifest under #82 if applicable.

[kilo-code-bot]

Kilo Code Review could not run — your account is out of credits.

Add credits or switch to a free model to enable reviews on this change.