Skip to content

[Feature]: Core image annotation — architecture, Annotate tab and delivery proposal #1717

Open

Description

@robmarkcole

Before you start

What problem does this solve?

I annotate georeferenced imagery for computer-vision training, usually one TIFF at a time. I want precise rotated boxes, circles, polygons and points, with editable classes/colours, individual-object selection and deletion, undo, and convenient navigation between annotations.

Each TIFF should have one companion GeoJSON containing its annotations, image reference and category palette. OBB exchange should expose four ordered original-image pixel corners for downstream conversion using tools such as Roboflow Supervision. My primary downstream target is RF-DETR; this proposal does not assume native RF-DETR OBB training support.

#1544 demonstrates the workflow, with screenshots, interaction examples, metadata documentation and tests. Following the maintainer review, this issue seeks architectural agreement before further implementation.

Proposed solution

Core implementation. I propose keeping the complete workflow in core, as demonstrated by #1544, using RustGIS commands, editable layers, selection and undo. Representative commands include annotation.addBox, annotation.updateBox, annotation.addShape, annotation.setClass, annotation.openImage, annotation.saveImage and annotation.obb. The proposed tool.set additions also need agreement. Is core the right architecture? A plugin, or a plugin using reusable core editing tools, remains an alternative for discussion.

Dedicated Annotate tab — preferred. Annotation is a repeated workflow: choose an image and target layer, choose a class, draw/select/refine objects, navigate, and save annotations. A dedicated tab makes that sequence discoverable and keeps the active image, layer and class explicit. It also provides a coherent home as annotation shapes, class management and review tools expand, without spreading the workflow across several tabs or crowding Edit. Cartographic text labeling remains under Labeling.

I propose Image, Annotation Layers, Shapes and Classes groups, with deeper settings in an annotation pane also reachable through View. Existing navigation, undo/redo and export destinations should be reused rather than duplicated. The tab would extend the current ribbon purposes and requires explicit lead approval; Edit plus an annotation pane is the alternative. The split UI PR would document routes and check the layout at 1366 px.

Metadata. Keep a single GeoJSON with geographic features, stable IDs, versioned construction records, original-pixel OBB corners, and collection metadata describing the image grid and category IDs/names/colours, including unused categories. Exact circle centre/radius and box construction survive polygon export. No XML or palette sidecars.

The PR proposes a rustgis_annotation foreign member and optional Dataset.annotation field. Should RustGIS use that dedicated field, a general metadata mechanism, or another representation? The agreed design must define preservation through projects, packages, item files and plugin bridges. This is a proposed RustGIS convention, not a native Label Studio or COCO import format.

Compatibility. Replace the PR’s unconditional format-2 writing with conditional versioning: ordinary documents retain their existing format; only documents carrying otherwise lossy annotation metadata require a newer version. Reader support should ship at least one release before newer-format writing, following the saved-file policy.

Delivery. After agreement, split the prototype into independently reviewable PRs:

  1. GeoTIFF affine and original-image-grid reading.
  2. Rotated-box editing and commands on ordinary editable layers, without image binding.
  3. Circle, polygon and point tools.
  4. Approved metadata representation and reader support.
  5. Image binding, palettes and GeoJSON/OBB exchange, with newer-format writing following the agreed release sequence.

The prototype touches data, engine, ui-egui, and metadata propagation through map, plugin and persistence paths. It adds no dependencies or crates. The unrelated Clippy rewrites will move to a separate PR.

Responsiveness and validation. Input-sized reads, saves and OBB extraction must run off the desktop UI thread with progress and Cancel, using existing background paths. Each split PR will include focused regressions, cargo xtask ci, WebAssembly validation and actual UI evidence where applicable.

The review’s other findings will be addressed in the relevant stages: counted export-time approximation/loss notices, explicit handling of attribute collisions, bounded CRS parsing, consistent selection/Alt-overlap gestures, and clear unsupported-version errors. Compatibility and exchange tests will cover ordinary/annotated documents, affine transforms, palettes, construction records and OBB corners.

Estimated scope

Wishlist: a larger feature delivered through smaller, agreed stages.

Alternatives considered

A plugin-based workflow, Edit-tab tools with a pane, or external annotation software. My preferred proposal is the integrated core workflow and dedicated Annotate tab demonstrated in #1544, subject to architectural agreement.

Requested decision: @giswqs, is the proposed core workflow and dedicated Annotate tab appropriate, and which metadata representation and compatibility sequence should the implementation follow?

AI assistance: drafted with OpenAI Codex.

Activity

  1. pick2510 commented on Nov 10, 2026

    @pick2510
    Contributor

    Thanks for the clear proposal and the staged plan.

    My view: split it.

    Core: stages 1–3. GeoTIFF affine and original-grid reading, rotated boxes, and circle, polygon and point tools, as ordinary editing tools on editable layers under Edit. They're useful beyond annotation, and they feed the existing dl.train / raster.classifySupervised training data. If shapes need extra data, I'd prefer a general per-layer metadata mechanism over an annotation-only Dataset field.

    Not core for now: the annotation convention, the palette, image binding and the pixel-corner OBB export. They serve one workflow and would fix a file format we have to keep. A plugin fits better once the plugin API has map tools, file browsing (#1438) and settings (#1261), or an export option in the deep learning tools. I'd also start without a dedicated Annotate tab and revisit it if more uses appear.

    @giswqs, your call on the tab and the metadata.

  2. giswqs commented on Nov 10, 2026

    @giswqs
    Member

    It can be a group under the Analyze tab, not as top-level tab. Alternatively, it can be developed as a plugin, which allows top-level tab placement.

  3. robmarkcole commented on Nov 10, 2026

    @robmarkcole
    ContributorAuthor

    Thanks both. I’d like to refine the proposal to keep the complete workflow in core, with an Annotate group under Analyze and a dedicated annotation pane, following @giswqs’s suggested placement.

    The ribbon group would provide the main workflow actions: choose an image, open the annotation pane, activate drawing tools and save annotations. The pane would hold the target image/layer, classes and colours, shape-specific controls, and individual-object editing. This gives the expanding feature set a clear home without adding a top-level tab or requiring a plugin.

    The proposed core scope remains:

    • GeoTIFF affine and original-image-grid reading.
    • Rotated boxes, circles, polygons and points, integrated with editable layers, selection and undo.
    • Image binding, category/colour management, single-GeoJSON round-tripping and original-pixel OBB export.
    • Commands supporting the same operations through scripting and automation.

    Reusable geometry tools would also remain available under Edit. The annotation controls would share their commands and state, following the existing navigation, undo and export conventions.

    I agree with using a general per-layer metadata mechanism rather than an annotation-only Dataset field. Its representation and preservation through GeoJSON, projects, packages and plugin bridges still need agreement. The compatibility plan would preserve the existing format for ordinary documents, introduce a newer version only where necessary, and release reader support before newer-format writing.

    Once the scope is agreed, I’d split #1544 into the smaller PRs discussed above and carry the responsiveness, cancellation, data-preservation and interaction fixes into the relevant stages.

    @giswqs, would you support this complete core workflow with Analyze → Annotate plus an annotation pane, and a general per-layer metadata design agreed before the persistence work proceeds?

  4. robmarkcole commented on Nov 11, 2026

    @robmarkcole
    ContributorAuthor

    I’d like to simplify the initial scope around sharing ordinary GeoJSON, deferring custom metadata and saved-format changes.

    The workflow would be:

    1. Annotate an image and export standard GeoJSON geometry and feature properties.
    2. Another user loads the image and GeoJSON.
    3. To refine an annotation, they right-click it and choose Convert to → Rotated Box / Circle / Polygon / Point.
    4. RustGIS previews the conversion; confirmation enables the relevant handles and numeric controls.

    Conversion affects only the selected feature, preserves its ID and attributes, and is undoable. Fitting uses the selected image’s original pixel grid. Circles and boxes export as polygons; reopening does not promise exact recovery of their original construction. There would be no automatic detection or silent reshaping.

    Following the built-in extension direction—and now that #1752 has landed through #1764—I propose these scoped PRs:

    PR Scope Dependency
    1. Image-grid foundations Original GeoTIFF dimensions, full affine transform and coordinate conversion. Can start now.
    2. Rotated-box editing Construction, rotation paddle, resizing, numeric controls, selection and undo on ordinary editable layers. Can start now; use PR 1 for image-grid editing.
    3. Additional shapes Circle, polygon and point drawing/editing using shared interactions. Builds on PR 2; split further if needed.
    4. Manual conversion Right-click conversion, preview, confirmation, attribute preservation and undo. After the relevant shape tools.
    5. Annotation extension Built-in Annotate tab and dedicated pane, wiring together the tools and existing GeoJSON import/export. Develop alongside the tools; integrate incrementally.

    Each PR would include focused regressions, repository quality gates and UI evidence, carrying forward the relevant correctness fixes from #1544.

    Custom collection metadata, construction records, persistent image bindings, palette/colour round-tripping and project-format changes would be deferred. Class values remain ordinary feature properties.

    Would this smaller scope be an acceptable starting point for #1717?

  5. pick2510 commented on Nov 12, 2026

    @pick2510
    Contributor

    Thanks, this scope works for me. Plain GeoJSON with classes as ordinary properties, and no format changes, removes most of my concerns.

    PRs 1–4 look good to me as planned. Two points:

    1. The rotated box, circle, polygon and point tools should also be usable from Edit on any editable layer, not only through the annotation workflow. They're useful beyond annotation, for example for dl.train training data.
    2. A circle or box comes back as a polygon after a round-trip. The export should say so in its notes, so nobody expects the construction to survive.

    For PR 5: a new top-level tab is only possible for a plugin. As @giswqs said above, in core it's a group under Analyze. So either the Annotate group and pane go under Analyze, or the extension becomes a plugin with its own tab.

  6. robmarkcole commented on Nov 13, 2026

    @robmarkcole
    ContributorAuthor

    Thanks—I'll incorporate those requirements and use Analyze → Annotate with a dedicated annotation pane for the initial workflow UI.

    The revised scoped plan is:

    PR Scope Dependency
    1. Image-grid foundations Original GeoTIFF dimensions, full affine transform before display resampling, and tested coordinate conversion. No annotation metadata or format changes. Implemented on feat/image-grid-foundations, based on upstream main; ready for a scoped PR.
    2. Rotated-box editing Construction, paddle rotation, resizing, numeric controls, selection/deletion and undo on ordinary editable layers. Also accessible under Edit, independently of annotation. Can start independently; use PR 1 for image-grid editing.
    3. Additional shapes Circle, polygon and point drawing/editing, also available under Edit. Split further if needed. Builds on the shared editing infrastructure from PR 2.
    4. Manual conversion Right-click Convert to → Rotated Box / Circle / Polygon / Point, with preview, confirmation, preserved feature identity/attributes and undo. No automatic detection or silent reshaping. After the relevant shape tools.
    5. Annotation workflow UI Analyze → Annotate opens a dedicated pane for the target image/layer, classes and shape controls, reusing the core editing commands and existing GeoJSON import/export. No new top-level tab or separately installed plugin in this initial scope. Develop alongside PRs 2–4; integrate incrementally.

    The shape PRs will include export notes explaining that circles and boxes are exported as ordinary polygons and their construction does not survive round-tripping. Notices will be aggregated by shape kind with counts. A recipient explicitly converts an imported feature again when constrained editing is needed.

    Custom collection metadata, construction records, persistent image bindings, palette/colour exchange and project-format changes remain deferred. Class values stay ordinary feature properties. Each PR will carry the relevant correctness fixes from #1544, focused tests, the repository quality gates and UI evidence where applicable.

    PR 1 will be a reusable rustgis-data API; it will not introduce annotation commands, UI, dependencies or changes to existing raster display behaviour.

    AI assistance: drafted with OpenAI Codex.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions