Skip to content

Conversation

@luandro
Copy link
Contributor

@luandro luandro commented Dec 10, 2025

Closes #115

Fix to detect and handle expiring S3 URLs in fresh content fetched from Notion.

Changes

  1. URL Expiration Check (imageReplacer.ts)

    • Added isUrlExpiringSoon(url, thresholdSeconds) utility
    • Checks S3 URLs for X-Amz-Expires + X-Amz-Date or Expires
    • Returns true if expiration is within threshold (default 5 mins)
  2. Validation & Retry in Cache Loaders (cacheLoaders.ts)

    • Modified loadWithCache to accept a validateResult function
    • loadBlocksForPage: checking blocks JSON for expiring URLs
    • loadMarkdownForPage: checking markdown string for expiring URLs
    • If content has expiring URLs, it retries the fetch with linear backoff (up to 3 times)
  3. Unit Tests

    • Created scripts/notion-fetch/__tests__/urlExpiration.test.ts to verify expiration logic
    • 6 tests for URL expiration detection
    • 47 tests for markdown processing with retry

Verification Results

  • ✅ All 116 tests passing
  • ✅ TypeScript type checking passes
  • ✅ Pre-commit hook passes (eslint --max-warnings=0)
  • ✅ All CI checks passing

@digidem digidem deleted a comment from chatgpt-codex-connector bot Dec 14, 2025
- Add isUrlExpiringSoon utility to imageReplacer.ts
- Add validation and retry logic to cacheLoaders.ts for blocks and markdown
- Add recursive URL traversal for efficient validation
- Add unit tests for expiration logic and traversal
Simplifies wrangler.toml by removing env.production and env.staging
sections that referenced non-existent projects. The deployment uses
a single project (comapeo-docs) with branch-based environments via
--branch flag in GitHub workflows.

Also updates compatibility_date to current date.
@github-actions
Copy link
Contributor

github-actions bot commented Dec 14, 2025

🚀 Preview Deployment

Your documentation preview is ready!

Preview URL: https://pr-116.comapeo-docs.pages.dev

🔄 Content: Regenerated 5 pages from Notion (script changes detected)

💡 Tip: Add label fetch-all-pages to test with full content, or fetch-10-pages for broader coverage.

This preview will update automatically when you push new commits to this PR.


Built with commit 7faaf9b

@digidem digidem deleted a comment from chatgpt-codex-connector bot Dec 14, 2025
@luandro luandro merged commit a0e7868 into main Dec 14, 2025
4 checks passed
@luandro luandro deleted the issue-115 branch December 14, 2025 22:28
@github-actions
Copy link
Contributor

🧹 Preview Deployment Cleanup

The preview deployment for this PR has been cleaned up.

Preview URL was: https://pr-116.comapeo-docs.pages.dev


Note: Cloudflare Pages deployments follow automatic retention policies. Old previews are cleaned up automatically.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

S3 URL expiration detection only checks cached content

2 participants