Dynamically rendering classification background & boundaries on an infinite, zoomable canvas

[Weiyu-Kong]

Description

Background

When the “Show Background” option in the right sidebar is checked, we expect to display a classification background behind the embedding points, so that users can visually understand class distributions and decision boundaries in the 2D space.

Previous approach (legacy solution)

In the original (finite canvas) implementation, we used the following pipeline:

1. During each epoch of the dimensionality reduction process, we determine the visible 2D coordinate range:

   • Top-left corner: [x_min, y_min]
   • Bottom-right corner: [x_max, y_max]

2. Given a chosen resolution, we generate 2D coordinates for every pixel within this range.

3. Using a trained visualization model, we project these 2D coordinates back into the high-dimensional space (via a specific encoder–decoder architecture).

4. We then feed these high-dimensional points into the original model to compute class prediction scores and confidences.

5. For each pixel, we assign a color according to the predicted class and confidence, and generate a background image (.png).

6. This image is then rendered as the background of the canvas, providing a visual classification background and decision boundaries.

In this workflow, dimensionality reduction and background image generation are tightly coupled and done synchronously.

Current issues

In the new interaction design, the embedding canvas has the following properties:

• The canvas is effectively infinite, and users can pan and zoom freely.

• Panning and zooming happen frequently during exploration.

• The background generation is strongly coupled to the visualization algorithm and model architecture (e.g., a specific encoder–decoder):

  • When we use our custom encoder–decoder, we can generate the background as before.
  • But when we use a generic dimensionality reduction method (e.g., UMAP), we can no longer rely on the old “inverse projection + PNG” approach.

As a result, the previous strategy of “offline PNG generation + fixed canvas extent” no longer fits the new “infinite canvas + multiple DR algorithms” scenario.

Question / Requirements

We are looking for a frontend-friendly approach that can:

1. Dynamically render a classification background or decision boundaries on the canvas based on the classes and density of points in the current viewport;
2. Make the background responsive to panning and zooming, updating in real time (instead of using a static PNG);
3. Work without relying on a specific encoder–decoder visualization model (i.e., it should also work when using UMAP or other generic DR methods);
4. Preserve all point interactions (selection, hover, tooltips, dragging, etc.) without noticeable performance degradation.

We would appreciate any suggestions on:

• Possible algorithms (e.g., kernel density estimation, Voronoi / Delaunay-based regions, grid-based interpolation, etc.);
• Practical frontend rendering techniques (Canvas, WebGL, offscreen rendering, layering strategies) that integrate well with an infinite, zoomable canvas while keeping interactions smooth.