[Feature] Test-Time Adaptation Visualization for Local TimeVis Refinement

[Weiyu-Kong]

Summary

Implement a lightweight test-time adaptation visualization feature for TimeVis.

After the initial embedding is generated, the user can select a point and click a new Refine button. The system should then re-visualize the selected point and its local neighborhood in high-dimensional space, and use the refined result to replace the current epoch's projection for that local region.

The main goal is to make the selected point's 2D neighbors more consistent with its high-dimensional neighbors. For example, the top-10 high-dimensional neighbors can be treated as ground truth, and the top-10 2D neighbors as prediction, with the refinement aiming to improve local precision and recall.

Background

This repository is a deep learning training-process visualization tool, and the visualization algorithms are mainly implemented under the tool directory.

TimeVis is the current target algorithm in this issue. In short, it learns a 2D embedding for training representations so that nearby samples in the original feature space remain close in the visualization. In this task, we only want to reuse TimeVis in a simple local refinement setting, rather than redesign the algorithm itself.

Scope

Please implement the simplest workable version first:

• only refine points in the current epoch;
• only for TimeVis;
• only for one selected point;
• only one refinement pass per button click;
• local refinement can simply rerun TimeVis on the selected point and its nearby samples;
• the refined result should overwrite the current epoch's existing 2D coordinates for the affected points.

Requirements

• Add a Refine button in the frontend.
• Use the currently selected point as the refinement center.
• If no point is selected, disable the button or show a clear message.
• The backend should find a local neighborhood for the selected point in the high-dimensional representation space.
• The backend should rerun TimeVis on that local subset.
• The backend should return the updated projection result for the current epoch.
• The frontend should replace the current epoch's projection in memory and re-render immediately.

Expected Behavior

• The user selects a point in the current epoch view.
• The user clicks Refine.
• The backend performs local re-embedding for that point and its local neighborhood.
• The current epoch view updates in place.
• The selected point's local 2D neighborhood should better match its high-dimensional neighborhood after refinement.

Acceptance Criteria

• A user can trigger refinement from the frontend for a selected point.
• The frontend updates the current epoch projection without reloading all epochs.
• The updated local result is visible immediately after refinement.
• [Hard] After refinement, the selected point should show improved neighborhood retention, measured by higher local precision and recall between its high-dimensional neighbors and 2D neighbors.

Related Code Files

This list is not exhaustive, but these files are likely relevant.

Frontend

• web/src/component/function-panel.tsx
  • likely place for the Refine button
• web/src/component/chart.tsx
  • point selection and current visualization interaction
• web/src/views/plotView.tsx
  • current epoch projection loading and frontend state update
• web/src/state/state.unified.ts
  • stores selected point, epoch, and projection data
• web/src/communication/backend.ts
  • frontend request wrapper for the new refinement API

Backend API

• tool/server/server.py
  • add a new endpoint for local refinement
• tool/server/server_utils.py
  • useful for loading projection data, representations, and neighbors

Visualization Algorithm

• tool/visualize/strategy/timevis_strategy.py
  • main TimeVis logic to reuse for local refinement
• tool/visualize/strategy/projector.py
  • may be involved in projection-related logic
• tool/visualize/strategy/trainer.py
  • may be involved if local refinement reuses training flow
• tool/visualize/visualize_model.py
  • TimeVis model definition
• tool/visualize/data_provider.py
  • access to per-epoch representation data

Deliverable

Create a first working version of test-time adaptation visualization for TimeVis local refinement, where the user selects a point, clicks Refine, and immediately sees an updated local projection for the current epoch.