Skip to content

data(cpu): fill process node and L3 cache for 84 AMD CPUs - #135

Merged
Seungpyo1007 merged 2 commits into
mainfrom
data/cpu-amd-specs
Jul 29, 2026
Merged

data(cpu): fill process node and L3 cache for 84 AMD CPUs#135
Seungpyo1007 merged 2 commits into
mainfrom
data/cpu-amd-specs

Conversation

@Seungpyo1007

@Seungpyo1007 Seungpyo1007 commented Jul 29, 2026

Copy link
Copy Markdown
Member

What

84 AMD records gain 136 previously-null fieldsprocess_node 84, l3_cache_mb 52 — from the Wikipedia Athlon / Athlon 64 / Phenom / FX / Sempron / Turion list pages.

Running total across #132#134 and this PR: CPU process_node 24% → 55%, l3_cache_mb 19% → 54%.

This batch is small on purpose

769 rows parsed, 332 matched a record, 176 passed the cross-check, and only 84 records had a gap left to fill (the rest were already covered by the earlier batches). AMD's list pages are far less uniform than Intel's, and two problems had to be solved before writing anything:

1. Mixed clock units. Some pages state the clock in MHz, others in GHz. Sempron rows read 1500 against our 1.5, so every one of them failed the cross-check. Normalising the unit turned that page from 1 trusted row into 30.

2. A page that files TDP as cache. The Athlon II page proposed:

record proposed L3 its actual TDP
Athlon II X4 638 65 MB 65 W
Athlon II X4 641 100 MB 100 W

…plus a 4.0 GHz turbo for parts that have no turbo. Its columns are misaligned, so every row-derived value from that page is discarded; only process_node, which comes from the section heading rather than the row, is kept.

Guards added

  • L3 must be a real cache capacity (0.5–64 MB from the known set) and must not equal the row's own TDP — that equality is the signature of a bled column.
  • A turbo clock must exceed the base clock and stay within a sane delta.
  • The standing rule still applies: base clock within 0.05 GHz and TDP within 1 W, or the row is dropped.

After the guards the proposed values are 2/4/6/8 MB caches and 130/90/65/45/32 nm nodes — all real for these parts. All 7 source pages were HTTP-verified; every written record cites the specific page.

python -m app.validateData validation passed. Dump refreshed for the 84 changed pages only.

Closes #1

84 AMD records gain 136 previously-null fields (process_node 84,
l3_cache_mb 52) from the Wikipedia Athlon/Athlon 64/Phenom/FX/Sempron/
Turion list pages.

Two source quirks had to be handled before anything was written:

- These tables state the clock in MHz on some pages and GHz on others, so
  the value is normalised before it is compared (1500 -> 1.5). Without that
  the cross-check rejected every Sempron row.
- One page (Athlon II) was caught filing a TDP figure in the L3 column --
  it proposed "L3 = 65 MB" for a 65 W part and "100 MB" for a 100 W part,
  and a 4.0 GHz turbo for chips that have no turbo at all. Its row-derived
  values are therefore discarded entirely; only the node, which comes from
  the section heading rather than the row, is kept from that page.

Values are additionally checked against the set of real cache capacities,
and a turbo clock must exceed the base clock by a sane margin. The usual
confirm-before-write rule still applies (base clock within 0.05 GHz, TDP
within 1 W): 332 rows matched a record, 176 passed.

Refs #1
Regenerates site/public/v1/cpus for the 84 records changed in the previous
commit, with the engine at the pinned submodule commit. Only the pages whose
content actually differs are committed; 7,871 timestamp-only pages are not.

Refs #1
@TechEngineBot

Copy link
Copy Markdown
Member

🔎 Data verification — Tier 0 (offline existence/trust)

Scored by app.verify; posted by TechEngineBot. Informational only — the structural gate (app.validate) is separate and authoritative for merge.

Changed records in this PR

84 record(s) scored.

%%{init: {"theme":"base","themeVariables":{"pie1":"#3fb950","pie2":"#d29922","pie3":"#f85149","pieStrokeWidth":"0px","pieOpacity":"1"}}}%%
pie showData
    title Verification bands — all records
    "Green" : 82
    "Yellow" : 2
    "Red" : 0
Loading
Category Distribution Total 🟢 🟡 🔴 🟢 %
cpu 🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟨 84 82 2 0 97.6%
All 🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟨 84 82 2 0 97.6%

Full-dataset baseline

105742 record(s) scored.

%%{init: {"theme":"base","themeVariables":{"pie1":"#3fb950","pie2":"#d29922","pie3":"#f85149","pieStrokeWidth":"0px","pieOpacity":"1"}}}%%
pie showData
    title Verification bands — all records
    "Green" : 14702
    "Yellow" : 89127
    "Red" : 1913
Loading
Category Distribution Total 🟢 🟡 🔴 🟢 %
brand 🟩🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨 207 24 183 0 11.6%
soc 🟩🟨🟨🟨🟨🟥🟥🟥🟥🟥🟥🟥 2104 123 680 1301 5.8%
smartphone 🟩🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟥 93396 11602 81206 588 12.4%
tablet 🟩🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟥 3455 174 3266 15 5.0%
watch 🟩🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟥 433 11 415 7 2.5%
pda 🟩🟩🟨🟨🟨🟨🟨🟨🟨🟨🟨🟥 140 27 112 1 19.3%
gpu 🟩🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨 2030 245 1785 0 12.1%
cpu 🟩🟩🟩🟩🟩🟩🟩🟨🟨🟨🟨🟥 3977 2496 1480 1 62.8%
All 🟩🟩🟨🟨🟨🟨🟨🟨🟨🟨🟨🟥 105742 14702 89127 1913 13.9%

Hard violations (forced red):

Count Check
1 !boost_ge_base

green = authoritative source + complete + consistent · yellow = plausible, needs confirmation · red = sparse/weak source or a hard contradiction. Promotion to verified runs in the scheduled verify-network workflow.

@Seungpyo1007
Seungpyo1007 merged commit df1f762 into main Jul 29, 2026
5 checks passed
@github-project-automation github-project-automation Bot moved this from In Progress to Done in TechAPI-Project Jul 29, 2026
@Seungpyo1007
Seungpyo1007 deleted the data/cpu-amd-specs branch July 29, 2026 08:27
@TechEngineBot

Copy link
Copy Markdown
Member

TechEngine change review: PASS

Check Result
python -m app.validate PASS
python integrity_check.py TechAPI/data --strict PASS

Changed data

Category Added Modified Deleted Added verified Added unverified Added Kaggle-sourced
brand 0 0 0 0 0 0
soc 0 0 0 0 0 0
smartphone 0 0 0 0 0 0
tablet 0 0 0 0 0 0
watch 0 0 0 0 0 0
pda 0 0 0 0 0 0
gpu 0 0 0 0 0 0
cpu 0 0 0 0 0 0

Changed record examples

  • No data file changes detected.

Heuristic review

  • Heuristic warnings: none found.

@TechEngineBot

Copy link
Copy Markdown
Member

TechEngine validation stats: PASS

Data summary

Category Total Verified Unverified Missing verified Tracked Verified % of tracked
brand 207 10 197 0 207 4.8%
soc 2104 123 1981 0 2104 5.8%
smartphone 93396 8453 84943 0 93396 9.1%
tablet 3455 174 3281 0 3455 5.0%
watch 433 11 422 0 433 2.5%
pda 140 27 113 0 140 19.3%
gpu 2030 244 1786 0 2030 12.0%
cpu 3977 986 2991 0 3977 24.8%
all 105742 10028 95714 0 105742 9.5%

Warning

Tracked verified coverage is below 50% for watch 2.5% (11/433), brand 4.8% (10/207), tablet 5.0% (174/3455), soc 5.8% (123/2104), smartphone 9.1% (8453/93396), all 9.5% (10028/105742), gpu 12.0% (244/2030), pda 19.3% (27/140), and 1 more.
Tracked coverage excludes records missing the verified field; see the Missing verified column for those records.
This does not fail validation. Keep imported records verified: false until manual audit, but treat this as follow-up verification work before relying on the affected categories as curated data.

Validation notes

  • Full advisory outlier listings are suppressed on successful runs because they are dataset-wide and mostly stable between PRs.
  • Failure runs still include a detailed log excerpt for debugging.

Key output:

## app.validate
## integrity_check.py --strict
loaded CPU=3977 GPU=2030
✅ integrity gate: no hard anomalies.
Integrity section Flagged lines
structural 0
CPU name/tier consistency (desktop mainstream only) 0
CPU single>multi (cinebench/geekbench — should be multi>=single) 0
CPU era-vs-score outliers 8
CPU cross-source ratio outliers (possible wrong-variant) 152
GPU cross-source ratio outliers + sanity 18

Seungpyo1007 added a commit that referenced this pull request Jul 30, 2026
…kipped

Recovers records that the merged batches (#132-#135) left untouched because of
three separate mismatches in how the pages name things, not because the data
was unavailable:

- model codes are not always digit-first (Core 2 uses E6600, Atom uses N270),
  and the table parser required a leading digit;
- the Atom page is built entirely from {{cpulist}} templates and had never been
  run through that parser;
- the Opteron page lists a bare model-number column with no branding column, so
  the family name now comes from the page itself.

Same confirm-before-write rule as before (base clock within 0.05 GHz, TDP within
1 W, L3 must be a real capacity and must not equal the row's TDP): 1,610 rows
parsed, 1,053 matched a record, 885 passed, 210 still had a gap to fill.

Verified against known parts: Clarkdale i5-6xx/i3-5xx 32 nm, Haswell i7-4770R
22 nm, Broadwell i7-5775R 14 nm, Atom 330/D410 45 nm, Atom D2500 32 nm,
Opteron Shanghai 6 MB L3. Older families whose rows failed the cross-check
(Core 2 E-series, Atom N270, early Opteron) were left empty rather than guessed.

Refs #1
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

data Dataset changes enhancement New feature or request

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

Massive dataset rebuild: CPU + brand + GPU + smartphone + SoC (1989-2026)

2 participants