Skip to content

Update spec to include tag history endpoint - #606

Open
jcarter3 wants to merge 9 commits into
opencontainers:mainfrom
jcarter3:tag_history
Open

Update spec to include tag history endpoint#606
jcarter3 wants to merge 9 commits into
opencontainers:mainfrom
jcarter3:tag_history

Conversation

@jcarter3

@jcarter3 jcarter3 commented Apr 23, 2026

Copy link
Copy Markdown
Contributor

Updates distribution spec to include new endpoint for retrieving the history of a given tag. The response is a json list of descriptors,including the created timestamp and sorted in descending order (newest first).
The list can be paged.

Comment thread spec.md Outdated
@jcarter3
jcarter3 force-pushed the tag_history branch 2 times, most recently from ba1f62d to dd534be Compare May 4, 2026 19:24
@jcarter3 jcarter3 changed the title **WIP** Update spec to include tag history endpoint Update spec to include tag history endpoint May 4, 2026
@jcarter3
jcarter3 marked this pull request as ready for review May 4, 2026 19:25

@mikebrow mikebrow left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

some thoughts

Comment thread extensions/_oci.md Outdated
Comment thread extensions/_oci.md
When `n` is zero, this endpoint MUST return an empty array.
When `n` is not specified, the registry MAY apply a default limit.

- **`before`** *string (RFC 3339 timestamp)*, OPTIONAL

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe from or ot (older than)

Suggested change
- **`before`** *string (RFC 3339 timestamp)*, OPTIONAL
- **`ot`** *string (RFC 3339 timestamp)*, OPTIONAL

Comment thread extensions/_oci.md Outdated

- **`before`** *string (RFC 3339 timestamp)*, OPTIONAL

When provided, only entries whose `org.opencontainers.tag.created` value is strictly less than (i.e. older than) `before` will be returned.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
When provided, only entries whose `org.opencontainers.tag.created` value is strictly less than (i.e. older than) `before` will be returned.
When provided, only entries whose `org.opencontainers.tag.created` value is strictly less than (i.e. older than) the `ot` timestamp will be returned.

Comment thread extensions/_oci.md
When `n` is zero, this endpoint MUST return an empty array.
When `n` is not specified, the registry MAY apply a default limit.

- **`before`** *string (RFC 3339 timestamp)*, OPTIONAL

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is there a chance to have two results with the exact same time stamp..

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps a registry could consider every update of a tag as a delete followed by a create and so the timestamp would be the same.

@northtyphoon northtyphoon Aug 22, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would think the history can be more compact for repush case. Actually two created events are better repressing what acutally happened compared to created/deleted/created.

Comment thread extensions/_oci.md
Signed-off-by: Jeff Carter <jeff.carter@docker.com>

@sudo-bmitch sudo-bmitch left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here's some early feedback. I still need to spend some time implementing this before considering approving, and we need a conformance test.

Comment thread extensions/_oci.md
Comment thread extensions/_oci.md Outdated
Comment thread extensions/_oci.md Outdated
Comment thread extensions/_oci.md Outdated
Comment thread extensions/_oci.md Outdated
]
```

Results MUST be sorted in descending order by the history entry timestamp, which is either the `org.opencontainers.tag.created` annotation value or the `org.opencontainers.tag.deleted` annotation value (i.e. the most recent entry appears first).

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In image spec, we use the prefix org.opencontainers.image. For distribution, perhaps we should standardize on org.opencontainers.distribution to avoid conflicts with other specs?

Comment thread extensions/_oci.md Outdated
Comment thread extensions/_oci.md
Comment thread extensions/_oci.md Outdated
Comment thread extensions/_oci.md Outdated
Comment on lines +155 to +177
- **`mediaType`** *string*, REQUIRED

MUST be `application/vnd.oci.empty.v1+json`.

- **`digest`** *string*, REQUIRED

MUST be `sha256:44136fa355b3678a1146ad16f7e8649e94fb4fc21fe77e8310c060f61caaff8a`.

- **`size`** *integer*, REQUIRED

MUST be `2`.

- **`data`** *string*, REQUIRED

MUST be `e30=`.

- **`annotations`** *map of strings*, REQUIRED

Annotations associated with the historical tag entry. MUST include:

- **`org.opencontainers.tag.deleted`** *string*, REQUIRED

The RFC 3339 timestamp at which the tag was deleted. Used as the sort key and as the cursor for time-based pagination.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd rephrase to use the descriptor of the deleted entry. And now with so much overlap, I think we can probably merge the two create/delete sections and then document the two possible annotations.

Suggested change
- **`mediaType`** *string*, REQUIRED
MUST be `application/vnd.oci.empty.v1+json`.
- **`digest`** *string*, REQUIRED
MUST be `sha256:44136fa355b3678a1146ad16f7e8649e94fb4fc21fe77e8310c060f61caaff8a`.
- **`size`** *integer*, REQUIRED
MUST be `2`.
- **`data`** *string*, REQUIRED
MUST be `e30=`.
- **`annotations`** *map of strings*, REQUIRED
Annotations associated with the historical tag entry. MUST include:
- **`org.opencontainers.tag.deleted`** *string*, REQUIRED
The RFC 3339 timestamp at which the tag was deleted. Used as the sort key and as the cursor for time-based pagination.
- **`mediaType`** *string*, REQUIRED
The media type of the manifest this tag previously pointed to at that point in time.
- **`size`** *integer*, REQUIRED
The size in bytes of the manifest.
- **`digest`** *string*, REQUIRED
The digest of the manifest, in the form `<algorithm>:<encoded>`.
- **`annotations`** *map of strings*, REQUIRED
Annotations associated with the historical tag entry. MUST include:
- **`org.opencontainers.distribution.tag.deleted`** *string*, REQUIRED
The RFC 3339 timestamp at which the tag was deleted. Used as the sort key and as the cursor for time-based pagination.

Comment thread extensions/_oci.md
Comment thread extensions/_oci.md Outdated

A successful request MUST return a `200 OK` response code.
If the repository does not exist, the registry MUST return a `404 Not Found` response code.
If the tag has no history or the registry does not implement this extension, the registry MUST return a `404 Not Found` response code.

@yosifkit yosifkit Jul 2, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this have a MUST/SHOULD respond with UNSUPPORTED in the error code for "registry does not implement this" case? Or perhaps some other recommended required error codes for the other 404 responses so that users of the API can know which it is. Or maybe the n=0 trick is the correct way to figure out if history is supported?


Related:

If the tag has no history [...] the registry MUST return a 404 Not Found

when n is zero, this endpoint MUST return an empty array

These seem conflicting. Perhaps a clarification is needed?

@yosifkit yosifkit Jul 2, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In other words, does n=0 override the "repository does not exist" 404 or just the "no history" 404 or "registry does not implement this extension" 404 and what about the pinned digest 404?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, those do some conflicting, good catch.

Currently most http routers on an unknown path would return 404 by default, hence the "return 404 if not supported". If we forced an UNSUPPORTED it'd require registries to explicitly update to say they aren't handling it, vs it being the default

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the appropriate thing to do is return an empty response with 200 OK if there is no history - this seems to make more sense?

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤔 Could it have no history? What does that mean? Does that mean nothing was ever tagged as this? Or just that the registry hasn't yet recorded history? But if it has at least the current tagged image and they support the history endpoint, it must have a minimum history of one, right? So, 404 is correct, the tag does not exist as an image, or it would have history. Returning an empty list for every tag that doesn't exist seems odd.

jcarter3 added 3 commits July 2, 2026 15:47
Signed-off-by: Jeff Carter <jeff.carter@docker.com>
Signed-off-by: Jeff Carter <jeff.carter@docker.com>
Signed-off-by: Jeff Carter <jeff.carter@docker.com>
Comment thread extensions/_oci.md Outdated

@sudo-bmitch sudo-bmitch left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adding our own annotations to the descriptors has me concerned. E.g. what if someone wants to pull up annotations in their own tag history API and there's a collision, perhaps even a collision with an annotation we haven't specified yet but add in a future release.

I'm playing around with my own implementation and have the following struct for each tag history entry:

type LayoutHistory struct {
	Time       time.Time  `json:"time"`
	Deleted    bool       `json:"deleted,omitempty"`
	Descriptor Descriptor `json:"descriptor"`
}

The descriptor ends up nested one extra layer in the responses, but we can now add other fields. E.g. I was considering if registries could add an audit log to each entry, identifying who (or what) was responsible for each entry.


Edit: A few more thoughts looking over this:

  • I'd want to use the Link header for pagination. It's cleaner from the client side to know when more results are available.
  • With the Link header, that gives flexibility for the server to reject some values of n for being too small or large, imposing it's own limits. Or the server can not support pagination and always return the full history.
  • Do we have guidance for servers implementing garbage collection of history responses? If the server has a hard upper limit on the number of history entries per tag, that makes it easier to not paginate responses.
  • When searching on a tag with a digest, I'd lean towards always returning a 200, and just having an empty list when no matches are found. Treat it like a filter on the history, where a tag could also have a particular digest assigned to it multiple times.
  • Is pushing a new tag value treated as a simultaneous delete of the old value, or can that be implied?

Comment thread extensions/_oci.md
When `n` is zero, this endpoint MUST return an empty array, useful for determining if history is available (200) or not (404) for the tag query.
When `n` is not specified, the registry MAY apply a default limit.

- **`before`** *string (RFC 3339 timestamp)*, OPTIONAL

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd love the opposite of this (after or since) to support the use case of "tailing" changes such that a client can cheaply check for updates to a tag to stay up to date. Some similar mechanisms would be If-Modified-Since headers or something like RSS. It would be great if we could lean on HTTP primitives for this.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've updated to include a since

@tianon

tianon commented Jul 23, 2026

Copy link
Copy Markdown
Member

On today's call we talked about a lot of good reasons why it makes sense for this to be shaped like an OCI index (like #579 + referrers), and on the problem of "what if a registry 'pulls up' annotations?" we discussed having said registry ignore any image annotations that start with org.opencontainers.distrubution. (as explicitly "owned" by the registry), for future possible growth (so that clients can assume they're from the registry, not the image).

Signed-off-by: Jeff Carter <jeff.carter@docker.com>

@michaelb990 michaelb990 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

a couple comments, but this seems like a cool idea!

for registries implementing this for existing repos, is there a way we should communicate to clients when we started tracking history? it seems like there's a difference between "this is the full history for a given tag" and "this is the history since 2026-08-15 for a given tag".

Comment thread extensions/_oci.md
Comment on lines +83 to +89
A successful request MUST return a `200 OK` response code.
If the registry does not implement this extension, the registry MUST return a `404 Not Found` response code.
The registry MUST NOT return a `404 Not Found` response code for any other condition.
If the repository does not exist or the tag has no history, the registry MUST return a `200 OK` response code with an empty manifest list.
Tag history SHOULD remain queryable after the tag is deleted.
Deleted manifests SHOULD NOT be removed from tag history.
Tag history MAY be deleted after the repository is deleted.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

from the example below, it seems like each registry operation (push or delete) will only create a single entry in the history. it seems a little ambiguous reading these requirements whether a tag moving between images creates 1 created event or 2 events (deleted/created).

can we add a requirement to clarify that a tag move, or a push to an existing tag, will only produce a single "created" event referencing the new image digest, assuming that's the direction we want to go?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I can add clarifying language there - the intent was to do a single "create" event when a tag moves, and "deleted" events only apply if it is specifically removed and no longer referencable in the registry.

Comment thread extensions/_oci.md
Deleted manifests SHOULD NOT be removed from tag history.
Tag history MAY be deleted after the repository is deleted.

Upon success, the response body MUST be an OCI Index containing a list of descriptor objects in the following format:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i'm not sure about image indexes being used to represent content that may/may not exist...

the referrers API was a pretty natural use of the image index since it's designed to contain content that's actually in the registry and is going to be pulled, but this feels forced -- any thoughts on using more of a bare bones JSON list, similar to #611?

potentially:

  GET /v2/team/web-app/_oci/tag-history/latest?n=3

  200 OK
  Content-Type: application/json

[
  {
    "event": "created",
    "timestamp": "2026-07-01T14:22:08.551204Z",
    "target": {
      "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
      "digest": "sha256:7d9f0e1c2b3a4958687776665554443332221110ffeeddccbbaa998877665544",
      "size": 3120
    }
  },
  {
    "event": "deleted",
    "timestamp": "2026-06-10T09:00:41.128870Z",
    "target": {
      "mediaType": "application/vnd.oci.image.index.v1+json",
      "digest": "sha256:e3a1c07f9b2d4856a0f1c2e3d4b5a69788c7d6e5f4039281a0b1c2d3e4f50617",
      "size": 5871
    }
  },
  {
    "event": "created",
    "timestamp": "2026-05-04T03:02:01.043916Z",
    "target": {
      "mediaType": "application/vnd.oci.image.index.v1+json",
      "digest": "sha256:e3a1c07f9b2d4856a0f1c2e3d4b5a69788c7d6e5f4039281a0b1c2d3e4f50617",
      "size": 5871
    }
  }
]

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We debated this a couple times - ultimately opted to stick with the index approach rather than having to define a new schema/model that everyone needs to copy/import. The data format isn't ideal but at least it is known. Happy to revisit if more people challenge it but that was the consensus before.

Comment thread extensions/_oci.md
A successful request MUST return a `200 OK` response code.
If the registry does not implement this extension, the registry MUST return a `404 Not Found` response code.
The registry MUST NOT return a `404 Not Found` response code for any other condition.
If the repository does not exist or the tag has no history, the registry MUST return a `200 OK` response code with an empty manifest list.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the repository does not exist or the tag has no history

the statement is a bit ambiguous, a tag must have a history, at least created if it ever exits. It's better to reword to something like if the repository or tag never exits

Comment thread extensions/_oci.md

The annotations MUST include both of the following:

- **`org.opencontainers.distribution.tag.timestamp`** *string*

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

timestamp is a universal concept, suggest remove tag from the annotation name

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants