fix(parser): fail loudly when the OssInsight ranking is unavailable

[sigmanor]

Problem

OssInsight paused its star-based rankings. The trends endpoint still answers with HTTP 200, but with an empty data.rows and a data_quality block spelling out why:

$ curl -s 'https://api.ossinsight.io/v1/trends/repos/?period=past_24_hours&language=All'
{"data":{"rows":[],...},"data_quality":{"status":"unavailable",
 "metric":"github_event_derived_ranking","unavailable_since":"2026-03-01",
 "reason":"...our capture of those events fell to roughly 0.3% of baseline..."}}

GetTrendingReposFromOssInsight read only data.rows, so the chain was:

empty rows → allRepos == nil → FilterExistingRepos(nil) → [] → AutoGenerate responds status: "ok" with empty added/dont_added.

A silent failure: a collect run reported success while fetching nothing, with no error anywhere in the logs.

Change

Parse data_quality and reject the response before iterating rows:

• status present and not ok → error naming the metric, the date and the upstream reason;
• no rows at all → error naming the requested period and language.

The check reads data.rows rather than the filtered slice, so a page where every description fails isEnglishText remains a legitimate empty result rather than an error.

ossinsight stays in validResources in auto_generate.go — a direct API call now gets a meaningful 500 with the real cause logged, which is better than a 400 claiming the resource is invalid.

Follow-up from review

The empty-payload guard was initially unconditional, which caught more than the outage it was meant for. language has no allow-list in auto_generate.go (unlike period), so a narrow filter against a healthy OssInsight can legitimately match nothing — and GetTrendingRepos returns an empty slice in exactly that situation for the GitHub source (parser/parser.go:294). The two sources would have disagreed on the same class of input, one answering 200 with an empty list and the other a 500.

The guard is now limited to an unfiltered query (language empty or All), where no rows can only mean the ranking is broken. It still covers a response that has lost its data_quality block.

Verification

• go build ./... and go vet ./... clean.
• Live call against the real endpoint now returns:
  OssInsight metric "github_event_derived_ranking" is unavailable since 2026-03-01: This ranking is ordered by recent star/PR/issue event counts, and our capture of those events fell to roughly 0.3% of baseline...
• The same request with "resource": "github" is unaffected.

Related

• fix(generate): disable OssInsight while its rankings are paused content-sentinel#17 disables the option in the UI
• fix(store): keep the scheduled collect off the paused OssInsight source content-maestro#4 moves the stored cron setting back to github

[sigmanor]

🎉 This PR is included in version 3.9.1 🎉

The release is available on GitHub release

Your semantic-release bot 📦🚀