Skip to content

fix(asin): ASIN extraction systems + rate-limit hardening + CI/CD pipeline - #20

Merged
jenilrupapara001 merged 1 commit into
mainfrom
arena/019fc604-retailops
Aug 3, 2026
Merged

fix(asin): ASIN extraction systems + rate-limit hardening + CI/CD pipeline#20
jenilrupapara001 merged 1 commit into
mainfrom
arena/019fc604-retailops

Conversation

@jenilrupapara001

Copy link
Copy Markdown
Collaborator

Summary

Both data-extraction systems (Amazon Creators API + Octoparse) made production-ready, plus CI/CD fixed.

Creators API

  • Never-exhaust / never-skip rate limiter (1 TPS + TPD per credential, 429 failover, pauses at daily cap instead of skipping)
  • Scheduled sync actually runs (syncSeller routing fix + env fallback)
  • Field ownership — API never nulls Octoparse rating/A+ data; %-based price dispute
  • Honest per-seller sync status; catch-up pass; batch history inserts

Octoparse

  • 429 backoff on status polling, UUID/group caching, index-friendly queries (migration 007)
  • Cluster-safe Redis locks; no swallowed cleanup errors; recovery-sync overlap protection

CI/CD

  • Backend lockfile regenerated (compression + cron-parser were missing -> npm ci was failing)
  • Backend CI now runs unit tests (110 passing) + full-tree syntax check + critical-module import check
  • CD applies DB migrations on deploy + uses PM2 ecosystem cluster config (8GB heap, expose-gc)
  • ecosystem.config.js paths made deploy-dir agnostic

Verification

  • 110/110 backend unit tests green
  • Frontend build passes (Vite)
  • 7 migrations (001-007) tracked via SchemaMigrations

Deploy note

After merge, CD auto-deploys to the VPS and applies migrations. Requires secrets: SSH_PRIVATE_KEY, DEPLOY_HOST, DEPLOY_USER.

@jenilrupapara001
jenilrupapara001 merged commit b1065a1 into main Aug 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants