Repository navigation
fix(engine): a full link chart skips what it can't draw, and a self-link is one end (#1947) #2223
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -1390,13 +1390,17 @@ fn link_chart(s: &Session, rel: Option<&str>) -> Result<(String, Option<String>) | |
| } | ||
| // The features a key relates to are counted once, however many features share it. | ||
| let mut counted: HashSet<String> = HashSet::new(); | ||
| // Where a key's destinations already in the chart sit among its destinations, found when | ||
| // the chart has no room for another one and kept while it stays that size: a full chart | ||
| // then looks at these only, not at every destination of every feature. | ||
| let mut in_chart: HashMap<String, (usize, Vec<usize>)> = HashMap::new(); | ||
| for orow in 0..o.dataset.features.len() { | ||
| let Some(key) = key_of(&o.dataset, orow, ok) else { continue }; | ||
| let Some(drows) = by_key.get(&key) else { continue }; | ||
| related.insert((om, o.id, orow)); | ||
| // Counted in 64 bits: relations can pass `usize` in the browser (32 bits). | ||
| relations += drows.len() as u64; | ||
| if counted.insert(key) { | ||
| if counted.insert(key.clone()) { | ||
| related.extend(drows.iter().map(|&drow| (dm, d.id, drow))); | ||
| } | ||
| if links.len() >= MAX_LINKS { | ||
|
|
@@ -1407,29 +1411,49 @@ fn link_chart(s: &Session, rel: Option<&str>) -> Result<(String, Option<String>) | |
| // out; one already in the chart may still be linked. | ||
| // Up to 200 links drawn from it: destinations the chart has no room for don't count | ||
| // toward them, so a later one it can draw still is. | ||
| let origin = (om, o.id, orow); | ||
| let mut a = index.get(&origin).copied(); | ||
| let mut a = index.get(&(om, o.id, orow)).copied(); | ||
| // A feature the chart has no room for can't be drawn, so none of its links can be. | ||
| if a.is_none() && nodes.len() >= MAX_NODES { | ||
| continue; | ||
| } | ||
| let mut drawn_here = 0; | ||
| for &drow in drows { | ||
| let mut i = 0; | ||
| while i < drows.len() { | ||
| if drawn_here >= MAX_FEATURE_LINKS || links.len() >= MAX_LINKS { | ||
| break; | ||
| } | ||
| let destination = (dm, d.id, drow); | ||
| let new = usize::from(a.is_none()) + usize::from(destination != origin && !index.contains_key(&destination)); | ||
| let drow = drows[i]; | ||
| // A feature related to itself is one end, not two. | ||
| let origin_new = a.is_none(); | ||
| let dest_new = !index.contains_key(&(dm, d.id, drow)) && !(origin_new && (om, o.id, orow) == (dm, d.id, drow)); | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Possible bug in the PR description: the self-link "fix" doesn't change behaviour. On main the count was That means the new The full-chart skip/jump logic (the
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yes: against today's main it isn't a fix. #2201 started counting a feature related to itself as one end (the #2201 also skips a feature the chart has no room for, which was the other half of this PR. The one part main doesn't have is this: once a destination has no room, a feature already in the chart goes straight to the next destination the chart holds (or to itself), not through each one in turn. I've corrected the description to say so. I'm also timing that case against main and will add the numbers there. |
||
| let new = usize::from(origin_new) + usize::from(dest_new); | ||
| if nodes.len() + new > MAX_NODES { | ||
| // No room for a destination the chart doesn't hold: on to the next one it | ||
| // does, or to the feature itself. | ||
| let (at, rows) = in_chart.entry(key.clone()).or_insert((usize::MAX, Vec::new())); | ||
| if *at != nodes.len() { | ||
| *rows = (0..drows.len()).filter(|&j| index.contains_key(&(dm, d.id, drows[j]))).collect(); | ||
| *at = nodes.len(); | ||
| } | ||
| let next_held = rows.get(rows.partition_point(|&j| j <= i)).copied(); | ||
| let next_self = | ||
| if origin_new && (om, o.id) == (dm, d.id) { drows.binary_search(&orow).ok().filter(|&j| j > i) } else { None }; | ||
| match next_held.into_iter().chain(next_self).min() { | ||
| Some(j) => i = j, | ||
| None => break, | ||
| } | ||
| continue; | ||
| } | ||
| let (Some(from), Some(to)) = | ||
| (a.or_else(|| node(&mut index, &mut nodes, om, o, orow, gi)), node(&mut index, &mut nodes, dm, d, drow, gi)) | ||
| else { | ||
| i += 1; | ||
| continue; | ||
| }; | ||
| a = Some(from); | ||
| links.push(json!({"source": from, "target": to, "relationship": r.name})); | ||
| drawn_here += 1; | ||
| i += 1; | ||
| } | ||
| } | ||
| } | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -12819,6 +12819,27 @@ fn a_link_chart_says_what_its_limits_leave_out() { | |
| t.run("share.linkChart", &json!({})).unwrap(); | ||
| // First draws 1 link (2 features), Fill 1,499 (2,997), and Last its one link to x (1 more). | ||
| assert_eq!((drawn("Relationships").0, drawn("Relationships").1), (1501, 3000)); | ||
| // A feature related to itself is one end, not two: with the chart at 2,999, its link still | ||
| // fits. On a session of its own, with First and Fill as above. | ||
| let mut u = world(); | ||
| u.run("share.dataStore", &json!({"name": "out", "path": dir.to_str().unwrap()})).unwrap(); | ||
| add(&mut u, "self_first_a.csv", &["x".to_string()]); | ||
| add(&mut u, "self_first_b.csv", &["x".to_string()]); | ||
| add(&mut u, "past_fill_one.csv", &fill_one); | ||
| add(&mut u, "past_fill_many.csv", &fill_many); | ||
| add(&mut u, "past_self.csv", &["z".to_string()]); | ||
| for (name, origin, destination) in | ||
| [("First", "self_first_a", "self_first_b"), ("Fill", "past_fill_one", "past_fill_many"), ("Self", "past_self", "past_self")] | ||
| { | ||
| u.run( | ||
| "data.relate", | ||
| &json!({"name": name, "origin": origin, "originKey": "key", "destination": destination, "destinationKey": "key"}), | ||
| ) | ||
| .unwrap(); | ||
| } | ||
| u.run("share.linkChart", &json!({})).unwrap(); | ||
| // First draws 1 link (2 features), Fill 1,499 (2,997), and Self its feature's link to itself (1 more). | ||
| assert_eq!((drawn("Relationships").0, drawn("Relationships").1), (1501, 3000)); | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. AGENTS.md asks for both sides of a limit to be tested: lossy input gives the notice, clean input gives none. This new block checks the drawn counts only, not the notice. At 1,501 links and 3,000 features, |
||
| // A feature isn't drawn alone: one feature related to 2 (3 features), then pairs. After 1,498 | ||
| // pairs the chart holds 2,999, and a pair needs 2 more: the rest are left out whole, rather | ||
| // than an origin drawn without its link. | ||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Grouped minor nits on the new full-chart path (no change needed for correctness; I traced the skip, jump and self-link branches and they hold):
(usize::MAX, Vec::new())sentinel inin_chartcould be anOption<(usize, Vec<usize>)>or a small struct, so the cache's "not yet computed" state is explicit rather than relying on no chart ever reachingusize::MAXnodes.rows.partition_pointanddrows.binary_searchboth rely ondrowsbeing sorted ascending. That holds becauseby_keyis filled by iterating rows in order, but nothing says so. A one-line comment at theby_keypush would keep it true under later edits.drows[i](line ~1415) anddrows[j](line ~1425) index rather than useget(). Both are guarded by the loop and range, so this is style, not a crash path, and it can wait for the release quality pass.next_self, i.e. a new origin atMAX_NODES - 1whose earlier destinations don't fit and whose own row comes later indrows. TheLastcase covers the held-destination jump, and theSelfcase covers a self-link that fits directly.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for tracing the branches. All four points are left for the release quality pass:
usize::MAXsentinel: the chart is capped at 3,000 nodes, so it can't be confused with a real count. AnOptionor a struct would make that explicit, but nothing behaves differently.drowssorted: this holds becauseby_keyis filled in row order. A comment there would only document it.get(): both indexes are bounded by the loop and the range, as you say.next_self: I traced it the same way. With a new origin at 2,999 nodes, earlier new destinations need two places and are skipped. The jump lands on the feature's own row, which needs one, and the link is drawn. Once that origin is in the chart,next_selfisNoneand the held-row list is rebuilt for 3,000. An extra test for code that works can wait for the pass.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Claude autofix (declined): Left for the release quality pass. The comment says no change is needed for correctness, and the points are style or extra coverage: the sentinel's form, a comment on the sorted drows invariant, indexing vs get(), and a test for next_self. The drows ordering does hold, since by_key is filled in row order.