Skip to content

data(soc): fill gpu_cores for 248 SoCs the earlier pass missed - #136

Merged
Seungpyo1007 merged 2 commits into
mainfrom
data/soc-gpu-cores-mp-suffix
Jul 29, 2026
Merged

data(soc): fill gpu_cores for 248 SoCs the earlier pass missed#136
Seungpyo1007 merged 2 commits into
mainfrom
data/soc-gpu-cores-mp-suffix

Conversation

@Seungpyo1007

@Seungpyo1007 Seungpyo1007 commented Jul 29, 2026

Copy link
Copy Markdown
Member

What

248 SoC records gain gpu_cores — coverage 9% → 21% — read from the vendor's own core-count notation in the gpu_name the record already carries.

Why there is more to do here after #51

#51 did this derivation in June and filled 139 SoCs, after which the work was recorded as finished. It wasn't: its pattern required a word boundary before MP, so it matched PowerVR SGX543MP3 but silently skipped every name where the suffix follows a digit — Mali-G76MP12, Mali-T880MP4, ARM Mali-G52MP2. Those 248 records have been sitting empty since.

Correctness

  • Cross-checked against the 138 records that already hold gpu_cores: all 138 agree with what their name encodes, zero conflicts. That is the rule validating itself on a held-out set.
  • Several names carry a second, independent statement of the same fact — PowerVR SGX543MP4 (quad-core graphics) — and the two always agree.
  • Names that spell the count out instead of using MP<n> are also read (PowerVR GXA6850 (octa-core graphics), Apple GPU (7-core graphics)) — 6 records.
  • Records that already hold a value are skipped, never overwritten; only counts in 1–128 are accepted.

Nothing external is consulted, so no new source is claimed: the value comes from a field the record already has, backed by its existing source_urls.

Deliberately not done

cpu_config is not derived from raw_chipset. Vendor marketing strings misdescribe the topology — Tegra 3's "penta-core" is a 4+1 arrangement — so a bulk derivation would encode a structure that is simply wrong. That field needs a real per-SoC source.

python -m app.validateData validation passed. Dump refreshed for the 248 changed pages only (3,961 timestamp-only pages left alone).

Closes #1

248 SoC records gain gpu_cores (9% -> 21% coverage). The count is taken from
the vendor's own notation in gpu_name -- 'PowerVR SGX543MP3', 'Mali-G76MP12',
'MP<n>' meaning n cores -- and from names that spell it out, such as
'PowerVR GXA6850 (octa-core graphics)'.

Nothing external is consulted, so no new source is claimed: the value is read
from a field the record already carries and whose existing source_urls back it.

The rule was checked against the 138 records that already hold gpu_cores:
all 138 agree with what the name encodes, none conflict. Records already
holding a value are skipped rather than overwritten, and only counts in the
1-128 range are accepted.

Note: cpu_config is deliberately NOT derived this way. Marketing strings like
Tegra 3's "penta-core" describe a 4+1 arrangement, so deriving a core split
from raw_chipset would encode a misleading structure.

Refs #1
Regenerates site/public/v1/socs for the 248 records changed in the previous
commit, with the engine at the pinned submodule commit so scoring output is
unchanged. Only pages whose content actually differs are committed; the 3,961
pages that differed solely by created_at/updated_at are left alone.

Refs #1
@TechEngineBot

Copy link
Copy Markdown
Member

🔎 Data verification — Tier 0 (offline existence/trust)

Scored by app.verify; posted by TechEngineBot. Informational only — the structural gate (app.validate) is separate and authoritative for merge.

Changed records in this PR

332 record(s) scored.

%%{init: {"theme":"base","themeVariables":{"pie1":"#3fb950","pie2":"#d29922","pie3":"#f85149","pieStrokeWidth":"0px","pieOpacity":"1"}}}%%
pie showData
    title Verification bands — all records
    "Green" : 2
    "Yellow" : 330
    "Red" : 0
Loading
Category Distribution Total 🟢 🟡 🔴 🟢 %
soc 🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨 248 0 248 0 0.0%
cpu 🟩🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨 84 2 82 0 2.4%
All 🟩🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨 332 2 330 0 0.6%

Full-dataset baseline

105742 record(s) scored.

%%{init: {"theme":"base","themeVariables":{"pie1":"#3fb950","pie2":"#d29922","pie3":"#f85149","pieStrokeWidth":"0px","pieOpacity":"1"}}}%%
pie showData
    title Verification bands — all records
    "Green" : 14622
    "Yellow" : 89331
    "Red" : 1789
Loading
Category Distribution Total 🟢 🟡 🔴 🟢 %
brand 🟩🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨 207 24 183 0 11.6%
soc 🟩🟨🟨🟨🟨🟨🟥🟥🟥🟥🟥🟥 2104 123 804 1177 5.8%
smartphone 🟩🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟥 93396 11602 81206 588 12.4%
tablet 🟩🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟥 3455 174 3266 15 5.0%
watch 🟩🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟥 433 11 415 7 2.5%
pda 🟩🟩🟨🟨🟨🟨🟨🟨🟨🟨🟨🟥 140 27 112 1 19.3%
gpu 🟩🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨 2030 245 1785 0 12.1%
cpu 🟩🟩🟩🟩🟩🟩🟨🟨🟨🟨🟨🟥 3977 2416 1560 1 60.7%
All 🟩🟩🟨🟨🟨🟨🟨🟨🟨🟨🟨🟥 105742 14622 89331 1789 13.8%

Hard violations (forced red):

Count Check
1 !boost_ge_base

green = authoritative source + complete + consistent · yellow = plausible, needs confirmation · red = sparse/weak source or a hard contradiction. Promotion to verified runs in the scheduled verify-network workflow.

@Seungpyo1007
Seungpyo1007 merged commit f3f2a6d into main Jul 29, 2026
5 checks passed
@github-project-automation github-project-automation Bot moved this from In Progress to Done in TechAPI-Project Jul 29, 2026
@Seungpyo1007
Seungpyo1007 deleted the data/soc-gpu-cores-mp-suffix branch July 29, 2026 11:59
@TechEngineBot

Copy link
Copy Markdown
Member

TechEngine change review: PASS

Check Result
python -m app.validate PASS
python integrity_check.py TechAPI/data --strict PASS

Changed data

Category Added Modified Deleted Added verified Added unverified Added Kaggle-sourced
brand 0 0 0 0 0 0
soc 0 0 0 0 0 0
smartphone 0 0 0 0 0 0
tablet 0 0 0 0 0 0
watch 0 0 0 0 0 0
pda 0 0 0 0 0 0
gpu 0 0 0 0 0 0
cpu 0 84 0 0 0 0

Changed record examples

cpu modified

  • cpu/amd/2008/consumer/amd-athlon-64-3700-plus.json - AMD Athlon 64 3700+
  • cpu/amd/2008/consumer/amd-athlon-64-x2-dual-core-3600-plus.json - AMD Athlon 64 X2 Dual Core 3600+
  • cpu/amd/2008/consumer/amd-athlon-64-x2-dual-core-4000-plus.json - AMD Athlon 64 X2 Dual Core 4000+
  • cpu/amd/2008/consumer/amd-athlon-64-x2-dual-core-4400-plus.json - AMD Athlon 64 X2 Dual Core 4400+
  • cpu/amd/2008/consumer/amd-athlon-64-x2-dual-core-4600-plus.json - AMD Athlon 64 X2 Dual Core 4600+
  • cpu/amd/2008/consumer/amd-athlon-64-x2-dual-core-4800-plus.json - AMD Athlon 64 X2 Dual Core 4800+
  • cpu/amd/2008/consumer/amd-athlon-64-x2-dual-core-5200-plus.json - AMD Athlon 64 X2 Dual Core 5200+
  • cpu/amd/2008/consumer/amd-sempron-2400-plus.json - AMD Sempron 2400+
  • cpu/amd/2008/consumer/amd-sempron-2800-plus.json - AMD Sempron 2800+
  • cpu/amd/2008/consumer/amd-sempron-3800-plus.json - AMD Sempron 3800+
  • cpu/amd/2009/consumer/amd-athlon-64-fx-74.json - AMD Athlon 64 FX-74
  • cpu/amd/2009/consumer/amd-athlon-64-x2-dual-core-5800-plus.json - AMD Athlon 64 X2 Dual Core 5800+
  • cpu/amd/2009/consumer/amd-phenom-ii-x2-545.json - AMD Phenom II X2 545
  • cpu/amd/2009/consumer/amd-phenom-ii-x2-550.json - AMD Phenom II X2 550
  • cpu/amd/2009/consumer/amd-phenom-ii-x2-b53.json - AMD Phenom II X2 B53
  • ... 69 more

Heuristic review

  • Heuristic warnings: none found.

@TechEngineBot

Copy link
Copy Markdown
Member

TechEngine validation stats: PASS

Data summary

Category Total Verified Unverified Missing verified Tracked Verified % of tracked
brand 207 10 197 0 207 4.8%
soc 2104 123 1981 0 2104 5.8%
smartphone 93396 8453 84943 0 93396 9.1%
tablet 3455 174 3281 0 3455 5.0%
watch 433 11 422 0 433 2.5%
pda 140 27 113 0 140 19.3%
gpu 2030 244 1786 0 2030 12.0%
cpu 3977 986 2991 0 3977 24.8%
all 105742 10028 95714 0 105742 9.5%

Warning

Tracked verified coverage is below 50% for watch 2.5% (11/433), brand 4.8% (10/207), tablet 5.0% (174/3455), soc 5.8% (123/2104), smartphone 9.1% (8453/93396), all 9.5% (10028/105742), gpu 12.0% (244/2030), pda 19.3% (27/140), and 1 more.
Tracked coverage excludes records missing the verified field; see the Missing verified column for those records.
This does not fail validation. Keep imported records verified: false until manual audit, but treat this as follow-up verification work before relying on the affected categories as curated data.

Validation notes

  • Full advisory outlier listings are suppressed on successful runs because they are dataset-wide and mostly stable between PRs.
  • Failure runs still include a detailed log excerpt for debugging.

Key output:

## app.validate
## integrity_check.py --strict
loaded CPU=3977 GPU=2030
✅ integrity gate: no hard anomalies.
Integrity section Flagged lines
structural 0
CPU name/tier consistency (desktop mainstream only) 0
CPU single>multi (cinebench/geekbench — should be multi>=single) 0
CPU era-vs-score outliers 8
CPU cross-source ratio outliers (possible wrong-variant) 152
GPU cross-source ratio outliers + sanity 18

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

data Dataset changes enhancement New feature or request

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

Massive dataset rebuild: CPU + brand + GPU + smartphone + SoC (1989-2026)

2 participants