Skip to content

data(cpu): recover 210 CPUs the earlier batches skipped on naming mismatches - #137

Merged
Seungpyo1007 merged 2 commits into
mainfrom
data/cpu-specs-batch6
Jul 30, 2026
Merged

data(cpu): recover 210 CPUs the earlier batches skipped on naming mismatches#137
Seungpyo1007 merged 2 commits into
mainfrom
data/cpu-specs-batch6

Conversation

@Seungpyo1007

@Seungpyo1007 Seungpyo1007 commented Jul 30, 2026

Copy link
Copy Markdown
Member

What

210 records / 240 fields (process_node 205, l3_cache_mb 35). CPU process_node coverage 56% → 61%, l3_cache_mb 54% → 55%.

Why these were missed, not missing

After #132#135 merged, 1,737 CPUs still had no process_node — and several whole families sat at 0% filled (A-Series 213/213, Ryzen 125/125, Core 2 124/124, Opteron 103/103, Atom 78/78). That looked like absent data. It wasn't; it was three naming mismatches:

  1. Model codes are not always digit-first. The table parser required a leading digit, so Core 2's E6600/Q6600 and Atom's N270/Z530 were discarded before any matching happened.
  2. The Atom page has no wikitext tables at all — 244 {{cpulist}} templates — and had only ever been run through the table parser.
  3. The Opteron page has a bare model-number column and no branding column, so nothing matched a family regex. The family name now comes from the page itself (that page lists only Opterons), with identity still carried by the cross-check.

Correctness

1,610 rows parsed → 1,053 matched a record → 885 passed the cross-check → 210 still had a gap to fill. Same rules as the merged batches: base clock within 0.05 GHz, TDP within 1 W, L3 must be a real cache capacity and must not equal the row's own TDP.

Spot-checked against known parts, all correct:

record filled correct for
i5-680, i5-660, i3-530 32 nm Clarkdale
i7-4770R, i5-4670R 22 nm Haswell
i7-5775R 14 nm Broadwell
Atom 330, D410, D425 45 nm Diamondville/Pineview
Atom D2500, D2550 32 nm Cedarview
Opteron 1385, 2382, 8378 L3 6 MB Shanghai

Older families were left empty on purpose. Core 2 E-series, Atom N270 and early Opteron rows did not pass the cross-check, so nothing was written for them rather than guessed — which is why no 65 nm or 130 nm values appear here at all.

All 4 source pages HTTP-verified; every record cites the page it came from. python -m app.validateData validation passed. Dump refreshed for the 210 changed pages only.

Closes #1

…kipped

Recovers records that the merged batches (#132-#135) left untouched because of
three separate mismatches in how the pages name things, not because the data
was unavailable:

- model codes are not always digit-first (Core 2 uses E6600, Atom uses N270),
  and the table parser required a leading digit;
- the Atom page is built entirely from {{cpulist}} templates and had never been
  run through that parser;
- the Opteron page lists a bare model-number column with no branding column, so
  the family name now comes from the page itself.

Same confirm-before-write rule as before (base clock within 0.05 GHz, TDP within
1 W, L3 must be a real capacity and must not equal the row's TDP): 1,610 rows
parsed, 1,053 matched a record, 885 passed, 210 still had a gap to fill.

Verified against known parts: Clarkdale i5-6xx/i3-5xx 32 nm, Haswell i7-4770R
22 nm, Broadwell i7-5775R 14 nm, Atom 330/D410 45 nm, Atom D2500 32 nm,
Opteron Shanghai 6 MB L3. Older families whose rows failed the cross-check
(Core 2 E-series, Atom N270, early Opteron) were left empty rather than guessed.

Refs #1
Regenerates site/public/v1/cpus for the 210 records changed in the previous
commit, with the engine at the pinned submodule commit. Only pages whose content
actually differs are committed; 7,745 timestamp-only pages are left alone.

Refs #1
@TechEngineBot

Copy link
Copy Markdown
Member

🔎 Data verification — Tier 0 (offline existence/trust)

Scored by app.verify; posted by TechEngineBot. Informational only — the structural gate (app.validate) is separate and authoritative for merge.

Changed records in this PR

210 record(s) scored.

%%{init: {"theme":"base","themeVariables":{"pie1":"#3fb950","pie2":"#d29922","pie3":"#f85149","pieStrokeWidth":"0px","pieOpacity":"1"}}}%%
pie showData
    title Verification bands — all records
    "Green" : 210
    "Yellow" : 0
    "Red" : 0
Loading
Category Distribution Total 🟢 🟡 🔴 🟢 %
cpu 🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩 210 210 0 0 100.0%
All 🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩 210 210 0 0 100.0%

Full-dataset baseline

105742 record(s) scored.

%%{init: {"theme":"base","themeVariables":{"pie1":"#3fb950","pie2":"#d29922","pie3":"#f85149","pieStrokeWidth":"0px","pieOpacity":"1"}}}%%
pie showData
    title Verification bands — all records
    "Green" : 14778
    "Yellow" : 89175
    "Red" : 1789
Loading
Category Distribution Total 🟢 🟡 🔴 🟢 %
brand 🟩🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨 207 24 183 0 11.6%
soc 🟩🟨🟨🟨🟨🟨🟥🟥🟥🟥🟥🟥 2104 123 804 1177 5.8%
smartphone 🟩🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟥 93396 11602 81206 588 12.4%
tablet 🟩🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟥 3455 174 3266 15 5.0%
watch 🟩🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟥 433 11 415 7 2.5%
pda 🟩🟩🟨🟨🟨🟨🟨🟨🟨🟨🟨🟥 140 27 112 1 19.3%
gpu 🟩🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨 2030 245 1785 0 12.1%
cpu 🟩🟩🟩🟩🟩🟩🟩🟨🟨🟨🟨🟥 3977 2572 1404 1 64.7%
All 🟩🟩🟨🟨🟨🟨🟨🟨🟨🟨🟨🟥 105742 14778 89175 1789 14.0%

Hard violations (forced red):

Count Check
1 !boost_ge_base

green = authoritative source + complete + consistent · yellow = plausible, needs confirmation · red = sparse/weak source or a hard contradiction. Promotion to verified runs in the scheduled verify-network workflow.

@Seungpyo1007
Seungpyo1007 merged commit 84442f8 into main Jul 30, 2026
5 checks passed
@github-project-automation github-project-automation Bot moved this from In Progress to Done in TechAPI-Project Jul 30, 2026
@Seungpyo1007
Seungpyo1007 deleted the data/cpu-specs-batch6 branch July 30, 2026 02:23
@TechEngineBot

Copy link
Copy Markdown
Member

TechEngine change review: PASS

Check Result
python -m app.validate PASS
python integrity_check.py TechAPI/data --strict PASS

Changed data

Category Added Modified Deleted Added verified Added unverified Added Kaggle-sourced
brand 0 0 0 0 0 0
soc 0 0 0 0 0 0
smartphone 0 0 0 0 0 0
tablet 0 0 0 0 0 0
watch 0 0 0 0 0 0
pda 0 0 0 0 0 0
gpu 0 0 0 0 0 0
cpu 0 0 0 0 0 0

Changed record examples

  • No data file changes detected.

Heuristic review

  • Heuristic warnings: none found.

@TechEngineBot

Copy link
Copy Markdown
Member

TechEngine validation stats: PASS

Data summary

Category Total Verified Unverified Missing verified Tracked Verified % of tracked
brand 207 10 197 0 207 4.8%
soc 2104 123 1981 0 2104 5.8%
smartphone 93396 8453 84943 0 93396 9.1%
tablet 3455 174 3281 0 3455 5.0%
watch 433 11 422 0 433 2.5%
pda 140 27 113 0 140 19.3%
gpu 2030 244 1786 0 2030 12.0%
cpu 3977 986 2991 0 3977 24.8%
all 105742 10028 95714 0 105742 9.5%

Warning

Tracked verified coverage is below 50% for watch 2.5% (11/433), brand 4.8% (10/207), tablet 5.0% (174/3455), soc 5.8% (123/2104), smartphone 9.1% (8453/93396), all 9.5% (10028/105742), gpu 12.0% (244/2030), pda 19.3% (27/140), and 1 more.
Tracked coverage excludes records missing the verified field; see the Missing verified column for those records.
This does not fail validation. Keep imported records verified: false until manual audit, but treat this as follow-up verification work before relying on the affected categories as curated data.

Validation notes

  • Full advisory outlier listings are suppressed on successful runs because they are dataset-wide and mostly stable between PRs.
  • Failure runs still include a detailed log excerpt for debugging.

Key output:

## app.validate
## integrity_check.py --strict
loaded CPU=3977 GPU=2030
✅ integrity gate: no hard anomalies.
Integrity section Flagged lines
structural 0
CPU name/tier consistency (desktop mainstream only) 0
CPU single>multi (cinebench/geekbench — should be multi>=single) 0
CPU era-vs-score outliers 8
CPU cross-source ratio outliers (possible wrong-variant) 152
GPU cross-source ratio outliers + sanity 18

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

data Dataset changes enhancement New feature or request

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

Massive dataset rebuild: CPU + brand + GPU + smartphone + SoC (1989-2026)

2 participants