Update spec to include tag history endpoint - #606
Conversation
ba1f62d to
dd534be
Compare
| When `n` is zero, this endpoint MUST return an empty array. | ||
| When `n` is not specified, the registry MAY apply a default limit. | ||
|
|
||
| - **`before`** *string (RFC 3339 timestamp)*, OPTIONAL |
There was a problem hiding this comment.
maybe from or ot (older than)
| - **`before`** *string (RFC 3339 timestamp)*, OPTIONAL | |
| - **`ot`** *string (RFC 3339 timestamp)*, OPTIONAL |
|
|
||
| - **`before`** *string (RFC 3339 timestamp)*, OPTIONAL | ||
|
|
||
| When provided, only entries whose `org.opencontainers.tag.created` value is strictly less than (i.e. older than) `before` will be returned. |
There was a problem hiding this comment.
| When provided, only entries whose `org.opencontainers.tag.created` value is strictly less than (i.e. older than) `before` will be returned. | |
| When provided, only entries whose `org.opencontainers.tag.created` value is strictly less than (i.e. older than) the `ot` timestamp will be returned. |
| When `n` is zero, this endpoint MUST return an empty array. | ||
| When `n` is not specified, the registry MAY apply a default limit. | ||
|
|
||
| - **`before`** *string (RFC 3339 timestamp)*, OPTIONAL |
There was a problem hiding this comment.
is there a chance to have two results with the exact same time stamp..
There was a problem hiding this comment.
Perhaps a registry could consider every update of a tag as a delete followed by a create and so the timestamp would be the same.
There was a problem hiding this comment.
I would think the history can be more compact for repush case. Actually two created events are better repressing what acutally happened compared to created/deleted/created.
Signed-off-by: Jeff Carter <jeff.carter@docker.com>
sudo-bmitch
left a comment
There was a problem hiding this comment.
Here's some early feedback. I still need to spend some time implementing this before considering approving, and we need a conformance test.
| ] | ||
| ``` | ||
|
|
||
| Results MUST be sorted in descending order by the history entry timestamp, which is either the `org.opencontainers.tag.created` annotation value or the `org.opencontainers.tag.deleted` annotation value (i.e. the most recent entry appears first). |
There was a problem hiding this comment.
In image spec, we use the prefix org.opencontainers.image. For distribution, perhaps we should standardize on org.opencontainers.distribution to avoid conflicts with other specs?
| - **`mediaType`** *string*, REQUIRED | ||
|
|
||
| MUST be `application/vnd.oci.empty.v1+json`. | ||
|
|
||
| - **`digest`** *string*, REQUIRED | ||
|
|
||
| MUST be `sha256:44136fa355b3678a1146ad16f7e8649e94fb4fc21fe77e8310c060f61caaff8a`. | ||
|
|
||
| - **`size`** *integer*, REQUIRED | ||
|
|
||
| MUST be `2`. | ||
|
|
||
| - **`data`** *string*, REQUIRED | ||
|
|
||
| MUST be `e30=`. | ||
|
|
||
| - **`annotations`** *map of strings*, REQUIRED | ||
|
|
||
| Annotations associated with the historical tag entry. MUST include: | ||
|
|
||
| - **`org.opencontainers.tag.deleted`** *string*, REQUIRED | ||
|
|
||
| The RFC 3339 timestamp at which the tag was deleted. Used as the sort key and as the cursor for time-based pagination. |
There was a problem hiding this comment.
I'd rephrase to use the descriptor of the deleted entry. And now with so much overlap, I think we can probably merge the two create/delete sections and then document the two possible annotations.
| - **`mediaType`** *string*, REQUIRED | |
| MUST be `application/vnd.oci.empty.v1+json`. | |
| - **`digest`** *string*, REQUIRED | |
| MUST be `sha256:44136fa355b3678a1146ad16f7e8649e94fb4fc21fe77e8310c060f61caaff8a`. | |
| - **`size`** *integer*, REQUIRED | |
| MUST be `2`. | |
| - **`data`** *string*, REQUIRED | |
| MUST be `e30=`. | |
| - **`annotations`** *map of strings*, REQUIRED | |
| Annotations associated with the historical tag entry. MUST include: | |
| - **`org.opencontainers.tag.deleted`** *string*, REQUIRED | |
| The RFC 3339 timestamp at which the tag was deleted. Used as the sort key and as the cursor for time-based pagination. | |
| - **`mediaType`** *string*, REQUIRED | |
| The media type of the manifest this tag previously pointed to at that point in time. | |
| - **`size`** *integer*, REQUIRED | |
| The size in bytes of the manifest. | |
| - **`digest`** *string*, REQUIRED | |
| The digest of the manifest, in the form `<algorithm>:<encoded>`. | |
| - **`annotations`** *map of strings*, REQUIRED | |
| Annotations associated with the historical tag entry. MUST include: | |
| - **`org.opencontainers.distribution.tag.deleted`** *string*, REQUIRED | |
| The RFC 3339 timestamp at which the tag was deleted. Used as the sort key and as the cursor for time-based pagination. |
Signed-off-by: Jeff Carter <jeff.carter@docker.com>
|
|
||
| A successful request MUST return a `200 OK` response code. | ||
| If the repository does not exist, the registry MUST return a `404 Not Found` response code. | ||
| If the tag has no history or the registry does not implement this extension, the registry MUST return a `404 Not Found` response code. |
There was a problem hiding this comment.
Should this have a MUST/SHOULD respond with UNSUPPORTED in the error code for "registry does not implement this" case? Or perhaps some other recommended required error codes for the other 404 responses so that users of the API can know which it is. Or maybe the n=0 trick is the correct way to figure out if history is supported?
Related:
If the tag has no history [...] the registry MUST return a
404 Not Found
when
nis zero, this endpoint MUST return an empty array
These seem conflicting. Perhaps a clarification is needed?
There was a problem hiding this comment.
In other words, does n=0 override the "repository does not exist" 404 or just the "no history" 404 or "registry does not implement this extension" 404 and what about the pinned digest 404?
There was a problem hiding this comment.
Yes, those do some conflicting, good catch.
Currently most http routers on an unknown path would return 404 by default, hence the "return 404 if not supported". If we forced an UNSUPPORTED it'd require registries to explicitly update to say they aren't handling it, vs it being the default
There was a problem hiding this comment.
I think the appropriate thing to do is return an empty response with 200 OK if there is no history - this seems to make more sense?
There was a problem hiding this comment.
🤔 Could it have no history? What does that mean? Does that mean nothing was ever tagged as this? Or just that the registry hasn't yet recorded history? But if it has at least the current tagged image and they support the history endpoint, it must have a minimum history of one, right? So, 404 is correct, the tag does not exist as an image, or it would have history. Returning an empty list for every tag that doesn't exist seems odd.
Signed-off-by: Jeff Carter <jeff.carter@docker.com>
Signed-off-by: Jeff Carter <jeff.carter@docker.com>
Signed-off-by: Jeff Carter <jeff.carter@docker.com>
There was a problem hiding this comment.
Adding our own annotations to the descriptors has me concerned. E.g. what if someone wants to pull up annotations in their own tag history API and there's a collision, perhaps even a collision with an annotation we haven't specified yet but add in a future release.
I'm playing around with my own implementation and have the following struct for each tag history entry:
type LayoutHistory struct {
Time time.Time `json:"time"`
Deleted bool `json:"deleted,omitempty"`
Descriptor Descriptor `json:"descriptor"`
}The descriptor ends up nested one extra layer in the responses, but we can now add other fields. E.g. I was considering if registries could add an audit log to each entry, identifying who (or what) was responsible for each entry.
Edit: A few more thoughts looking over this:
- I'd want to use the Link header for pagination. It's cleaner from the client side to know when more results are available.
- With the Link header, that gives flexibility for the server to reject some values of
nfor being too small or large, imposing it's own limits. Or the server can not support pagination and always return the full history. - Do we have guidance for servers implementing garbage collection of history responses? If the server has a hard upper limit on the number of history entries per tag, that makes it easier to not paginate responses.
- When searching on a tag with a digest, I'd lean towards always returning a 200, and just having an empty list when no matches are found. Treat it like a filter on the history, where a tag could also have a particular digest assigned to it multiple times.
- Is pushing a new tag value treated as a simultaneous delete of the old value, or can that be implied?
| When `n` is zero, this endpoint MUST return an empty array, useful for determining if history is available (200) or not (404) for the tag query. | ||
| When `n` is not specified, the registry MAY apply a default limit. | ||
|
|
||
| - **`before`** *string (RFC 3339 timestamp)*, OPTIONAL |
There was a problem hiding this comment.
I'd love the opposite of this (after or since) to support the use case of "tailing" changes such that a client can cheaply check for updates to a tag to stay up to date. Some similar mechanisms would be If-Modified-Since headers or something like RSS. It would be great if we could lean on HTTP primitives for this.
There was a problem hiding this comment.
I've updated to include a since
|
On today's call we talked about a lot of good reasons why it makes sense for this to be shaped like an OCI index (like #579 + referrers), and on the problem of "what if a registry 'pulls up' annotations?" we discussed having said registry ignore any image annotations that start with |
Signed-off-by: Jeff Carter <jeff.carter@docker.com>
michaelb990
left a comment
There was a problem hiding this comment.
a couple comments, but this seems like a cool idea!
for registries implementing this for existing repos, is there a way we should communicate to clients when we started tracking history? it seems like there's a difference between "this is the full history for a given tag" and "this is the history since 2026-08-15 for a given tag".
| A successful request MUST return a `200 OK` response code. | ||
| If the registry does not implement this extension, the registry MUST return a `404 Not Found` response code. | ||
| The registry MUST NOT return a `404 Not Found` response code for any other condition. | ||
| If the repository does not exist or the tag has no history, the registry MUST return a `200 OK` response code with an empty manifest list. | ||
| Tag history SHOULD remain queryable after the tag is deleted. | ||
| Deleted manifests SHOULD NOT be removed from tag history. | ||
| Tag history MAY be deleted after the repository is deleted. |
There was a problem hiding this comment.
from the example below, it seems like each registry operation (push or delete) will only create a single entry in the history. it seems a little ambiguous reading these requirements whether a tag moving between images creates 1 created event or 2 events (deleted/created).
can we add a requirement to clarify that a tag move, or a push to an existing tag, will only produce a single "created" event referencing the new image digest, assuming that's the direction we want to go?
There was a problem hiding this comment.
Yes, I can add clarifying language there - the intent was to do a single "create" event when a tag moves, and "deleted" events only apply if it is specifically removed and no longer referencable in the registry.
| Deleted manifests SHOULD NOT be removed from tag history. | ||
| Tag history MAY be deleted after the repository is deleted. | ||
|
|
||
| Upon success, the response body MUST be an OCI Index containing a list of descriptor objects in the following format: |
There was a problem hiding this comment.
i'm not sure about image indexes being used to represent content that may/may not exist...
the referrers API was a pretty natural use of the image index since it's designed to contain content that's actually in the registry and is going to be pulled, but this feels forced -- any thoughts on using more of a bare bones JSON list, similar to #611?
potentially:
GET /v2/team/web-app/_oci/tag-history/latest?n=3
200 OK
Content-Type: application/json
[
{
"event": "created",
"timestamp": "2026-07-01T14:22:08.551204Z",
"target": {
"mediaType": "application/vnd.docker.distribution.manifest.v2+json",
"digest": "sha256:7d9f0e1c2b3a4958687776665554443332221110ffeeddccbbaa998877665544",
"size": 3120
}
},
{
"event": "deleted",
"timestamp": "2026-06-10T09:00:41.128870Z",
"target": {
"mediaType": "application/vnd.oci.image.index.v1+json",
"digest": "sha256:e3a1c07f9b2d4856a0f1c2e3d4b5a69788c7d6e5f4039281a0b1c2d3e4f50617",
"size": 5871
}
},
{
"event": "created",
"timestamp": "2026-05-04T03:02:01.043916Z",
"target": {
"mediaType": "application/vnd.oci.image.index.v1+json",
"digest": "sha256:e3a1c07f9b2d4856a0f1c2e3d4b5a69788c7d6e5f4039281a0b1c2d3e4f50617",
"size": 5871
}
}
]
There was a problem hiding this comment.
We debated this a couple times - ultimately opted to stick with the index approach rather than having to define a new schema/model that everyone needs to copy/import. The data format isn't ideal but at least it is known. Happy to revisit if more people challenge it but that was the consensus before.
| A successful request MUST return a `200 OK` response code. | ||
| If the registry does not implement this extension, the registry MUST return a `404 Not Found` response code. | ||
| The registry MUST NOT return a `404 Not Found` response code for any other condition. | ||
| If the repository does not exist or the tag has no history, the registry MUST return a `200 OK` response code with an empty manifest list. |
There was a problem hiding this comment.
If the repository does not exist or the tag has no history
the statement is a bit ambiguous, a tag must have a history, at least created if it ever exits. It's better to reword to something like if the repository or tag never exits
|
|
||
| The annotations MUST include both of the following: | ||
|
|
||
| - **`org.opencontainers.distribution.tag.timestamp`** *string* |
There was a problem hiding this comment.
timestamp is a universal concept, suggest remove tag from the annotation name
Updates distribution spec to include new endpoint for retrieving the history of a given tag. The response is a json list of descriptors,including the created timestamp and sorted in descending order (newest first).
The list can be paged.