Skip to content

feat(media): model image, video, and 3D variants - #690

Open
igrigorik wants to merge 1 commit into
mainfrom
feat/media-sources
Open

feat(media): model image, video, and 3D variants#690
igrigorik wants to merge 1 commit into
mainfrom
feat/media-sources

Conversation

@igrigorik

Copy link
Copy Markdown
Contributor

The existing Media shape is image-oriented: one URL and one pair of dimensions cannot faithfully describe alternate video resolutions and codecs, adaptive-streaming manifests, device-specific 3D formats, poster images, or an externally hosted player. This update extends Media as an open type discriminator with well-known image, video, and model_3d variants. It follows the schema.org MediaObject family while preserving UCP's existing wire contract and generated-source APIs.

  • Adds per-type schemas for image, video, and 3D media using open if/then composition.
  • Adds sources for the complete set of available video renditions or 3D encodings.
  • Adds preview for a poster or thumbnail still.
  • Adds optional name, distinct from accessibility-oriented alt_text.
  • Adds video duration in seconds.
  • Models externally hosted playback as type: "video" plus embed_url, rather than introducing a redundant external_video type.
  • Adds source metadata for MIME type, format, dimensions, and byte size.
  • Keeps Media.type open: a Platform must ignore media types it does not recognize.

When sources is present and url identifies a rendition, the Business should include a source whose url matches the media item's url. The exception is an externally hosted video, where url identifies a watch page rather than a playable rendition.

Self-hosted video

The primary rendition appears in both url and the complete sources set:

{
  "type": "video",
  "url": "https://cdn.example.com/video-720.mp4",
  "name": "Product demonstration",
  "duration": 24,
  "preview": {
    "url": "https://cdn.example.com/video-poster.jpg",
    "width": 1280,
    "height": 720
  },
  "sources": [
    {
      "url": "https://cdn.example.com/video-720.mp4",
      "mime_type": "video/mp4",
      "width": 1280,
      "height": 720
    },
    {
      "url": "https://cdn.example.com/video.m3u8",
      "mime_type": "application/vnd.apple.mpegurl"
    }
  ]
}

3D model

A single logical model can advertise the formats supported by different devices:

{
  "type": "model_3d",
  "url": "https://cdn.example.com/product.glb",
  "preview": {
    "url": "https://cdn.example.com/product-poster.jpg"
  },
  "sources": [
    {
      "url": "https://cdn.example.com/product.glb",
      "mime_type": "model/gltf-binary"
    },
    {
      "url": "https://cdn.example.com/product.usdz",
      "mime_type": "model/vnd.usdz+zip"
    }
  ]
}

Externally hosted video

The watch page remains the primary URL while embed_url identifies third-party active player content:

{
  "type": "video",
  "url": "https://videos.example.com/watch/123",
  "embed_url": "https://videos.example.com/embed/123",
  "preview": {
    "url": "https://cdn.example.com/video-poster.jpg",
    "width": 1280,
    "height": 720
  }
}

A Platform may decline to load third-party active content under its security, privacy, or presentation policy and use the watch-page URL instead.

Backward compatibility

This is backward compatible for conforming uses of the released Media contract:

  • the required set remains type and url;
  • no existing field is removed, renamed, or retyped;
  • width and height remain on base Media for generated-source compatibility;
  • all new fields are optional;
  • existing payloads continue to validate;
  • unknown media types remain valid.

Checklist

  • Capability: New schemas (Discovery, Cart, etc.) or extensions. (Requires Maintainer approval)
  • Documentation: Updates to README, or documentations regarding schema or capabilities.
  • I have followed the Contributing Guide
  • I have updated the documentation (if applicable).
  • My changes pass all local linting and formatting checks.

The existing Media shape is image-oriented: one URL and one pair of
dimensions cannot describe alternate video resolutions and codecs, adaptive
streaming manifests, device-specific 3D formats, poster images, or an
externally hosted player's embed URL.

Model Media as an open type discriminator with image, video, and model_3d
variants. Keep `url` as the primary resource, add `sources` as the complete
set of negotiable renditions, and represent externally hosted playback as a
video with `embed_url` rather than a separate media type. Optional `preview`
and `name` fields provide a poster and human-readable label; source metadata
covers media type, format, dimensions, and byte size.

The shape follows the schema.org MediaObject family while using scalar seconds
for duration and bytes for file size. REST and MCP examples demonstrate
self-hosted video, 3D models, and embedded video without vendor-specific hosts.

This is additive and preserves generated-source compatibility: the base keeps
`required: [type, url]` plus `width` and `height`, shared constraints are
defined once on the base, and unknown media types remain valid but MUST be
ignored by Platforms.

Follow-up: teach the Schema Reference macro to merge inherited base constraints
when rendering description-only variant refinements.
@igrigorik igrigorik added this to the Working Draft milestone Aug 6, 2026
@igrigorik
igrigorik requested a review from gsmith85 August 6, 2026 16:27
@igrigorik igrigorik self-assigned this Aug 6, 2026
@igrigorik igrigorik added the TC review Ready for TC review label Aug 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants