Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
36 changes: 30 additions & 6 deletions crates/engine/src/cmd/sharing.rs
Original file line number Diff line number Diff line change
Expand Up @@ -1390,13 +1390,17 @@ fn link_chart(s: &Session, rel: Option<&str>) -> Result<(String, Option<String>)
}
// The features a key relates to are counted once, however many features share it.
let mut counted: HashSet<String> = HashSet::new();
// Where a key's destinations already in the chart sit among its destinations, found when
// the chart has no room for another one and kept while it stays that size: a full chart
// then looks at these only, not at every destination of every feature.
let mut in_chart: HashMap<String, (usize, Vec<usize>)> = HashMap::new();

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Grouped minor nits on the new full-chart path (no change needed for correctness; I traced the skip, jump and self-link branches and they hold):

  • The (usize::MAX, Vec::new()) sentinel in in_chart could be an Option<(usize, Vec<usize>)> or a small struct, so the cache's "not yet computed" state is explicit rather than relying on no chart ever reaching usize::MAX nodes.
  • rows.partition_point and drows.binary_search both rely on drows being sorted ascending. That holds because by_key is filled by iterating rows in order, but nothing says so. A one-line comment at the by_key push would keep it true under later edits.
  • drows[i] (line ~1415) and drows[j] (line ~1425) index rather than use get(). Both are guarded by the loop and range, so this is style, not a crash path, and it can wait for the release quality pass.
  • Coverage gap: no committed test reaches next_self, i.e. a new origin at MAX_NODES - 1 whose earlier destinations don't fit and whose own row comes later in drows. The Last case covers the held-destination jump, and the Self case covers a self-link that fits directly.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for tracing the branches. All four points are left for the release quality pass:

  • The usize::MAX sentinel: the chart is capped at 3,000 nodes, so it can't be confused with a real count. An Option or a struct would make that explicit, but nothing behaves differently.
  • drows sorted: this holds because by_key is filled in row order. A comment there would only document it.
  • Indexing rather than get(): both indexes are bounded by the loop and the range, as you say.
  • A test reaching next_self: I traced it the same way. With a new origin at 2,999 nodes, earlier new destinations need two places and are skipped. The jump lands on the feature's own row, which needs one, and the link is drawn. Once that origin is in the chart, next_self is None and the held-row list is rebuilt for 3,000. An extra test for code that works can wait for the pass.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude autofix (declined): Left for the release quality pass. The comment says no change is needed for correctness, and the points are style or extra coverage: the sentinel's form, a comment on the sorted drows invariant, indexing vs get(), and a test for next_self. The drows ordering does hold, since by_key is filled in row order.

for orow in 0..o.dataset.features.len() {
let Some(key) = key_of(&o.dataset, orow, ok) else { continue };
let Some(drows) = by_key.get(&key) else { continue };
related.insert((om, o.id, orow));
// Counted in 64 bits: relations can pass `usize` in the browser (32 bits).
relations += drows.len() as u64;
if counted.insert(key) {
if counted.insert(key.clone()) {
related.extend(drows.iter().map(|&drow| (dm, d.id, drow)));
}
if links.len() >= MAX_LINKS {
Expand All @@ -1407,29 +1411,49 @@ fn link_chart(s: &Session, rel: Option<&str>) -> Result<(String, Option<String>)
// out; one already in the chart may still be linked.
// Up to 200 links drawn from it: destinations the chart has no room for don't count
// toward them, so a later one it can draw still is.
let origin = (om, o.id, orow);
let mut a = index.get(&origin).copied();
let mut a = index.get(&(om, o.id, orow)).copied();
// A feature the chart has no room for can't be drawn, so none of its links can be.
if a.is_none() && nodes.len() >= MAX_NODES {
continue;
}
let mut drawn_here = 0;
for &drow in drows {
let mut i = 0;
while i < drows.len() {
if drawn_here >= MAX_FEATURE_LINKS || links.len() >= MAX_LINKS {
break;
}
let destination = (dm, d.id, drow);
let new = usize::from(a.is_none()) + usize::from(destination != origin && !index.contains_key(&destination));
let drow = drows[i];
// A feature related to itself is one end, not two.
let origin_new = a.is_none();
let dest_new = !index.contains_key(&(dm, d.id, drow)) && !(origin_new && (om, o.id, orow) == (dm, d.id, drow));

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Possible bug in the PR description: the self-link "fix" doesn't change behaviour. On main the count was usize::from(a.is_none()) + usize::from(destination != origin && !index.contains_key(&destination)). For a self-related feature, destination == origin, so the destination term was already 0 and the feature counted as one end. The new dest_new is logically identical in every case: when origin == destination, a is None exactly when the origin is not in index, so the new expression is also false. I could not find a case where main counts a self-link as two.

That means the new Self case in a_link_chart_says_what_its_limits_leave_out should pass on main too, which conflicts with the "Reproduced first … left: (1500, 2999)" output in the PR description. Could you run that test against main and confirm it fails there? If it passes, the self-link part is not a fix. The description and test should then say so, or a different reproduction should be found. Confidence: about 80%, from reading the diff; I have not run it.

The full-chart skip/jump logic (the in_chart cache and next_self) does look equivalent to the old behaviour to me, with no change in what's drawn.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes: against today's main it isn't a fix. #2201 started counting a feature related to itself as one end (the destination != origin you quote) after this PR was opened. The (1500, 2999) in the description came from this branch's first base, 803a014, which is before #2201. With main merged in, dest_new gives the same result as main's line and the Self block passes on main too. It stays only as a check of that case at 2,999 features.

#2201 also skips a feature the chart has no room for, which was the other half of this PR. The one part main doesn't have is this: once a destination has no room, a feature already in the chart goes straight to the next destination the chart holds (or to itself), not through each one in turn. I've corrected the description to say so. I'm also timing that case against main and will add the numbers there.

let new = usize::from(origin_new) + usize::from(dest_new);
if nodes.len() + new > MAX_NODES {
// No room for a destination the chart doesn't hold: on to the next one it
// does, or to the feature itself.
let (at, rows) = in_chart.entry(key.clone()).or_insert((usize::MAX, Vec::new()));
if *at != nodes.len() {
*rows = (0..drows.len()).filter(|&j| index.contains_key(&(dm, d.id, drows[j]))).collect();
*at = nodes.len();
}
let next_held = rows.get(rows.partition_point(|&j| j <= i)).copied();
let next_self =
if origin_new && (om, o.id) == (dm, d.id) { drows.binary_search(&orow).ok().filter(|&j| j > i) } else { None };
match next_held.into_iter().chain(next_self).min() {
Some(j) => i = j,
None => break,
}
continue;
}
let (Some(from), Some(to)) =
(a.or_else(|| node(&mut index, &mut nodes, om, o, orow, gi)), node(&mut index, &mut nodes, dm, d, drow, gi))
else {
i += 1;
continue;
};
a = Some(from);
links.push(json!({"source": from, "target": to, "relationship": r.name}));
drawn_here += 1;
i += 1;
}
}
}
Expand Down
21 changes: 21 additions & 0 deletions crates/engine/src/tests.rs
Original file line number Diff line number Diff line change
Expand Up @@ -12819,6 +12819,27 @@ fn a_link_chart_says_what_its_limits_leave_out() {
t.run("share.linkChart", &json!({})).unwrap();
// First draws 1 link (2 features), Fill 1,499 (2,997), and Last its one link to x (1 more).
assert_eq!((drawn("Relationships").0, drawn("Relationships").1), (1501, 3000));
// A feature related to itself is one end, not two: with the chart at 2,999, its link still
// fits. On a session of its own, with First and Fill as above.
let mut u = world();
u.run("share.dataStore", &json!({"name": "out", "path": dir.to_str().unwrap()})).unwrap();
add(&mut u, "self_first_a.csv", &["x".to_string()]);
add(&mut u, "self_first_b.csv", &["x".to_string()]);
add(&mut u, "past_fill_one.csv", &fill_one);
add(&mut u, "past_fill_many.csv", &fill_many);
add(&mut u, "past_self.csv", &["z".to_string()]);
for (name, origin, destination) in
[("First", "self_first_a", "self_first_b"), ("Fill", "past_fill_one", "past_fill_many"), ("Self", "past_self", "past_self")]
{
u.run(
"data.relate",
&json!({"name": name, "origin": origin, "originKey": "key", "destination": destination, "destinationKey": "key"}),
)
.unwrap();
}
u.run("share.linkChart", &json!({})).unwrap();
// First draws 1 link (2 features), Fill 1,499 (2,997), and Self its feature's link to itself (1 more).
assert_eq!((drawn("Relationships").0, drawn("Relationships").1), (1501, 3000));

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AGENTS.md asks for both sides of a limit to be tested: lossy input gives the notice, clean input gives none. This new block checks the drawn counts only, not the notice. At 1,501 links and 3,000 features, left should be None, so please assert that share.linkChart's notice for this session is empty (the notice_mark / said pattern used earlier in this test works here). Confidence: high that the assertion is missing; the rule applies because this is a clean case.

// A feature isn't drawn alone: one feature related to 2 (3 features), then pairs. After 1,498
// pairs the chart holds 2,999, and a pair needs 2 more: the rest are left out whole, rather
// than an origin drawn without its link.
Expand Down
Loading