Skip to content

Add API surface image on top of documentation page, with links#1266

Open
nevencaplar wants to merge 12 commits intomainfrom
API_surface_links
Open

Add API surface image on top of documentation page, with links#1266
nevencaplar wants to merge 12 commits intomainfrom
API_surface_links

Conversation

@nevencaplar
Copy link
Copy Markdown
Member

@nevencaplar nevencaplar commented Feb 28, 2026

As discussed at LINCCup, replaces non-informative HATS partitioning of GAIA DR3 with API surface graph, with links embedded.
Codex helped me create a JavaScript file that handles rescaling of the coordinates of the boxes, as well as visual debug overlay (activated b appending ?mapdebug=1 to the url of the webpage containing the graph, e.g., https://lsdb--1266.org.readthedocs.build/en/1266/?mapdebug=1).

The source plot is an Excalidraw file (docs/tools/data/API_Surface_Feb_12.excalidraw), and a python script (docs/tools/update_api_surface_coords_from_excalidraw.py) that should regenerate the boxes from the source file. docs/tools/validate_api_surface_map.py should validate the result. This procedure is also described in docs/tools/README.md.

@codecov
Copy link
Copy Markdown

codecov Bot commented Feb 28, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 96.82%. Comparing base (4d75377) to head (f6014ce).
⚠️ Report is 18 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1266      +/-   ##
==========================================
+ Coverage   96.65%   96.82%   +0.16%     
==========================================
  Files          48       48              
  Lines        2960     3052      +92     
==========================================
+ Hits         2861     2955      +94     
+ Misses         99       97       -2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@github-actions
Copy link
Copy Markdown

github-actions Bot commented Feb 28, 2026

Before [baf81cb] After [16a4957] Ratio Benchmark (Parameter)
162±2ms 167±1ms 1.03 benchmarks.time_open_many_columns_list
356±1ms 361±1ms 1.01 benchmarks.time_open_many_columns_default
20.1±0.01s 20.4±0.03s 1.01 benchmarks.time_save_big_catalog
27.5±0.1ms 27.4±0.2ms 1 benchmarks.time_box_filter_on_partition
101±0.9ms 100±0.9ms 1 benchmarks.time_kdtree_crossmatch
7.94±0.01s 7.92±0.02s 1 benchmarks.time_lazy_crossmatch_many_columns_all_suffixes
3.66±0.02s 3.66±0.01s 1 benchmarks.time_open_many_columns_all
8.03±0.02s 7.93±0s 0.99 benchmarks.time_lazy_crossmatch_many_columns_overlapping_suffixes
1.14±0.01s 1.12±0s 0.98 benchmarks.time_create_midsize_catalog
45.9±0.4ms 45.0±0.4ms 0.98 benchmarks.time_polygon_search

Click here to view all benchmarks.

The previous script put raw Excalidraw canvas coordinates into <area coords>
while the JS was treating them as 1441×822-authored-space values, causing
visible misalignment.

Changes:
- Compute the exact Excalidraw-export transform (pad=9, scale≈3.0075) so
  every text-element bbox is expressed in 3719×2164 PNG-pixel coordinates.
- Set data-original-map-width/height equal to data-map-width/height (3719×2164)
  so the JS authored→native step is a no-op; only rendered scaling remains.
- Update validator defaults to 3719×2164.
- All 65 hotspot <area coords> rewritten with correct positions.
…ens, remove debug labels

- updater: compute visual_h_excal = line_h_excal * min(1, 1.2/lineHeight) so
  text elements with large lineHeight (write_catalog 4.29x, MarginCatalog
  4.57x) produce correctly-sized boxes instead of bloated slots
- updater: skip strokeColor '#1e1e1e' black elements from TOKEN candidates so
  section headers ('Crossmatch', 'Join/Merge/Nest') don't shadow the real
  purple function-list entries for crossmatch and join areas
- updater: require 60% length overlap for both partial_token and line match to
  prevent short substrings ('catalog') matching long names ('margincatalog')
- api_surface_map.js: remove function-name label from debug overlay boxes
- validate_api_surface_map.py: update default bounds to 3719x2164
@nevencaplar nevencaplar requested a review from Copilot March 19, 2026 00:07
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds an interactive “API surface” image map to the documentation landing page, along with supporting tooling to regenerate/validate hotspot coordinates from an Excalidraw source.

Changes:

  • Replace the existing hero figure on docs/index.rst with a raw-HTML <img usemap> + <map><area> hotspot definition.
  • Add client-side JS/CSS to scale image-map coordinates to the rendered image size (plus optional debug overlay).
  • Add maintainer scripts + Excalidraw source data to update and validate the hotspot coordinates; update several reference autosummary lists.

Reviewed changes

Copilot reviewed 11 out of 13 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
docs/index.rst Adds the API surface image + image-map <area> links on the docs home page.
docs/conf.py Loads the new api_surface_map.js as a global Sphinx static JS file.
docs/_static/custom.css Adds styling for the API surface wrapper/image and debug overlay.
docs/_static/api_surface_map.js Implements responsive scaling for <area coords> and optional debug overlay.
docs/tools/update_api_surface_coords_from_excalidraw.py Regenerates <area coords> from Excalidraw geometry/text.
docs/tools/validate_api_surface_map.py Validates <area> rectangles and link targets in docs/index.rst.
docs/tools/README.md Documents how to run the update/validation tools and build a local preview.
docs/tools/data/API_Surface_Feb_12.excalidraw Adds the Excalidraw “source of truth” used for hotspot regeneration.
docs/reference/io.rst Adds show_versions to the autosummary list.
docs/reference/catalog_inspection.rst Adds Catalog.estimate_size to the autosummary list.
docs/reference/catalog_filtering.rst Adds Catalog.rename to the autosummary list.
docs/reference/catalog_dataframe.rst Adds Catalog.to_delayed to the autosummary list.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

You can also share your feedback on Copilot code review. Take the survey.

Comment thread docs/index.rst Outdated
Comment thread docs/tools/update_api_surface_coords_from_excalidraw.py Outdated
Comment thread docs/tools/update_api_surface_coords_from_excalidraw.py
Comment thread docs/tools/validate_api_surface_map.py Outdated
nevencaplar and others added 3 commits March 18, 2026 17:38
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
@nevencaplar nevencaplar requested review from dougbrn and hombit March 19, 2026 15:57
@nevencaplar nevencaplar marked this pull request as ready for review March 19, 2026 15:57
Copy link
Copy Markdown
Contributor

@hombit hombit left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

  • It is not clear that these methods are clickable. Maybe add a caption?
  • Image is really large, almost a megabyte. The load time would be significant on a bad connection. I propose to make it x2 resolution and convert to webp.
  • On Safari, my pointer is trying to select a word from the image, so it is not clickable. Please ask your imaginary friend ChatGPT for help (I guess the image should become a "background" or something like that).
Screen.Recording.2026-03-19.at.12.52.28.mov
  • Font is small. If it is hard to change, let's open a new issue to make the central part of the page wider, so the plot gets larger at least on larger monitors. (Note the large unused white area on the right)
Image

Copy link
Copy Markdown
Contributor

@dougbrn dougbrn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is really cool, just following on from Kostya's points. I think a callout box above the plot saying things are interactive would be good here, and when uploading excalidraw drawings for nested-pandas I found that svg was better format for filesize compared to png, so would encourage the same switch here (or exporting to something else if you want).

@hombit
Copy link
Copy Markdown
Contributor

hombit commented Mar 19, 2026

@dougbrn SVG actually sounds really good. It is small, scalable, and scripting should be more straightforward with it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants