Repository navigation
Expand file tree
/
Copy pathcontribute.html
More file actions
133 lines (124 loc) · 6.89 KB
/
Copy pathcontribute.html
File metadata and controls
133 lines (124 loc) · 6.89 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1, viewport-fit=cover" />
<title>Contribute — Bindwidth</title>
<meta name="description" content="Why Bindwidth needs measured runs, and how to contribute a model, machine, or benchmark by editing one JSON file." />
<meta name="author" content="Juxhin Radhima" />
<meta name="color-scheme" content="light" />
<link rel="preconnect" href="https://fonts.googleapis.com" />
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin />
<link href="https://fonts.googleapis.com/css2?family=IBM+Plex+Mono:wght@400;500;600&family=IBM+Plex+Sans:wght@400;500;600&family=Space+Grotesk:wght@500;600;700&display=swap" rel="stylesheet" />
<link rel="stylesheet" href="./assets/styles.css?v=13" />
<link rel="icon" href="data:image/svg+xml,<svg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 32 32'><rect width='32' height='32' rx='6' fill='%2315171c'/><rect x='6' y='8' width='9' height='3.5' rx='1.75' fill='%23565d6b'/><rect x='6' y='14.25' width='20' height='3.5' rx='1.75' fill='%23d4870f'/><rect x='6' y='20.5' width='13' height='3.5' rx='1.75' fill='%23565d6b'/></svg>" />
</head>
<body>
<header class="masthead masthead-solo">
<a class="wordmark" href="./" style="color:#fff;text-decoration:none">
Bindwidth <span class="dim">size your way to private local AI</span>
</a>
<div class="spacer"></div>
<div class="masthead-tools">
<div class="masthead-actions">
<a class="btn ghost-dark sm" href="./">Back to calculator</a>
<a class="btn ghost-dark sm" href="https://github.com/juxhinr/bindwidth" target="_blank" rel="noopener">GitHub</a>
</div>
</div>
</header>
<main class="page">
<a class="page-back" href="./">← Calculator</a>
<h1>Contribute</h1>
<p class="lede">
Bindwidth is a frontend with no server and no telemetry. The maths is open.
What it gets wrong — or gets right — lives in the catalogues: models, hardware,
and benchmarks that people have actually run.
</p>
<h2>Why this matters</h2>
<p>
Organisations buy machines against the wrong constraint when the numbers underneath
are guessed. Optimistic decode efficiency makes a configuration look cheap until
someone tries to run it. A KV figure copied from a blog post quietly doubles the
node count.
</p>
<p>
The calculator cannot invent measured truth. It can only surface the best data it
has — and label how trustworthy each figure is. A pull request that adds one honest
run is worth more than a dozen opinions about the UI.
</p>
<h2>You do not need to touch the application</h2>
<p>
Adding a model, a machine, or a benchmark means editing one JSON file under
<code>data/</code>. The interface loads those files at startup. If you can open a
pull request, you can improve the tool for everyone who sizes against it.
</p>
<ul>
<li><code>data/models.json</code> — architecture profiles that drive memory and speed</li>
<li><code>data/hardware.json</code> — owned nodes and rental options</li>
<li><code>data/benchmarks.json</code> — historical measured runs (not used in maths until reviewed)</li>
</ul>
<h2>The ground rule</h2>
<p>
<strong>A number without its configuration cannot be reproduced.</strong>
Every entry carries a <code>confidence</code> field. It appears in the exported
report. Six months from now, knowing which figures were measured and which were
derived will matter more than the figures themselves.
</p>
<div class="levels-wrap">
<table class="levels">
<thead>
<tr><th>Level</th><th>Use it when</th></tr>
</thead>
<tbody>
<tr><td><code>measured</code></td><td>You ran it, on a pinned build, and can show the counters</td></tr>
<tr><td><code>published</code></td><td>There is a public recipe someone else can follow</td></tr>
<tr><td><code>community</code></td><td>Someone reported it and it has not been verified</td></tr>
<tr><td><code>list-price</code></td><td>Straight from the vendor’s price page</td></tr>
<tr><td><code>aggregator</code></td><td>From a price tracker, not the vendor</td></tr>
<tr><td><code>estimated</code></td><td>Derived from bandwidth and memory arithmetic</td></tr>
<tr><td><code>derived / kv-derived</code></td><td>Back-derived from another measurement; include the derivation and range</td></tr>
<tr><td><code>template</code></td><td>A starting point, expected to be replaced</td></tr>
</tbody>
</table>
</div>
<p>Never promote a level without evidence. Downgrading is always allowed.</p>
<h2>What belongs in a good submission</h2>
<ul>
<li>The exact engine build (a commit hash, not “latest”)</li>
<li>Flags, concurrency, and whether traffic was synthetic or realistic</li>
<li>Failure modes — a run that documents what broke is more useful than a happy path alone</li>
<li>Open contradictions left visible, not quietly averaged away — or split into separate catalogue rows when the configurations differ (e.g. NVFP4 KV vs FP8 KV)</li>
</ul>
<h2>First-party measured runs</h2>
<p>
The catalogue can accept your own hardware soaks via the existing benchmark issue
template. Prefer first-party when you operated the box: pinned build, full flags,
concurrency and context, soak duration, and failure modes (even “none observed”).
Reviewed values move into <code>profiles.json</code> only after that review — the
calculator never invents a measured number.
</p>
<p>
Open a
<a href="https://github.com/juxhinr/bindwidth/issues/new?template=benchmark.yml" target="_blank" rel="noopener">benchmark issue</a>
or edit <code>data/benchmarks.json</code> directly with confidence <code>measured</code>.
</p>
<p>
Full field-by-field guidance lives in
<a href="https://github.com/juxhinr/bindwidth/blob/main/CONTRIBUTING.md" target="_blank" rel="noopener">CONTRIBUTING.md</a>
on GitHub.
</p>
<h2>Reporting a modelling error</h2>
<p>
If the arithmetic itself is wrong, open an issue with the inputs, what the tool
produced, and what you believe it should produce. Errors in the model matter more
than errors in the data: a formula pointing at the wrong cell still returns a
number, and it looks fine until someone checks.
</p>
<div class="cta-row">
<a class="btn primary" href="https://github.com/juxhinr/bindwidth" target="_blank" rel="noopener">Open the repository</a>
<a class="btn" href="https://github.com/juxhinr/bindwidth/blob/main/CONTRIBUTING.md" target="_blank" rel="noopener">Read CONTRIBUTING.md</a>
<a class="btn" href="./">Return to the calculator</a>
</div>
</main>
</body>
</html>