Video example - #265
Merged
Merged
Video example#265
Conversation
Agrael1
added a commit
that referenced
this pull request
Sep 13, 2026
* Add Video extension with DX12/Vulkan support and codegen Introduced a new Video extension with CMake integration, C/C++ API headers, and backend implementations for DirectX 12 and Vulkan. Updated the code generator to support handle constructors with custom parameters from XML, enabling automatic generation of extension handle constructors. Added XML spec files for the Video extension, new headers for enums and handles, and backend-specific implementation files. Improved code generation for function parameters and enhanced parsing of <init> blocks in handle definitions. Comprehensive documentation was added for all new types and functions. * Add extensible video decode capability query API Replaces the old boolean "Supported" check with wisVideoDecodingExtensionQueryCodecCaps, enabling detailed queries for codec profiles, bit depths, chroma subsampling, and max resolution. Introduces new enums and structs for codec capabilities, updates C/C++ APIs, and implements backend support for DX12 and Vulkan. Updates documentation, codegen, and sample app to use the new API. Adds helper files for Vulkan function loading and enum conversion. * Add video format & profile support, refactor codec desc - Add NV12, P010, P012, P016 formats and 16-bit bit depth support - Refactor WisVideoCodecDesc: use data_format, add width/height - Expand H.265 profiles (Main12, Main16), renumber profiles - Update DX12/Vulkan backends for new formats and profiles - Improve DX12 Agility SDK handling in CMake/build scripts - Update example app and docs for new API and formats - Minor cleanups and improved error handling throughout * Add video format validation to Vulkan decoding extension Enhances the Vulkan video decoding extension by adding support for querying and validating video format compatibility. Introduces format mapping utilities, integrates format checks into codec capability queries, and improves error handling for unsupported formats and profiles. * Refactor QueryCodecCaps to return Result only Refactored the QueryCodecCaps function in both C and C++ APIs for DX12 and Vulkan to return only a Result type, removing all usage of the VideoDecodeInfo structure. Updated function signatures, documentation, and XML metadata accordingly. Added resolution bounds checking in the Vulkan implementation. Adjusted example usage to match the new API. * Add cross-platform video decoder creation API Introduces a unified API for creating and managing hardware-accelerated video decoders in the Wisdom video module, supporting both DX12 and Vulkan. Adds the WisVideoDecoderDesc structure, new decoder handle types, and functions for decoder creation and destruction. Updates C/C++ APIs, documentation, and registry. Refactors out the old WisVideoDecodeInfo. Implements backend logic for both DX12 and Vulkan, and updates sample usage and wrappers for type safety and RAII. * Refactor video API: rename fields, add video decode test Refactor video decoding API for clarity and consistency: - Rename data_format to image_format in all APIs and docs - Replace max_width/max_height with width/height in decoder desc - Update DX12/Vulkan backends and headers for new field names - Add comprehensive video decoding capability test (video.cpp) - Update CMake/test infra to include new test - Simplify test logging callbacks and improve error handling - Update XML API descriptions for renamed fields - Refactor Vulkan backend result handling for clarity These changes improve API usability, consistency, and test coverage. * Refactor Vulkan video decoder resource management Introduce VKVideoDecodingControlBlock for improved resource tracking and lifetime management. Increase WisVKVideoDecoder handle size from 3 to 4. Add vkBindVideoSessionMemoryKHR to VKVideoTable and update initialization. Implement batch memory allocation and binding with VMA, ensuring proper cleanup. Update destruction logic for extension and decoder to release resources and references. Enhance VKScopeGuard with a Release method. Update documentation and XML handle definitions to match new handle size. * Add video decode command list API for DX12 and Vulkan Introduce WisVideoDecodeCommandList and WisVideoDecoderParameters handles for both DX12 and Vulkan backends. Add API functions for creating, beginning, ending, and destroying video decode command lists and decoder parameters. Extend the C++ API with RAII wrappers and methods for these resources. Implement backend logic for command list management. Update documentation, code registry, and build files to support the new functionality. * - Introduce AV1 enums and structs for profiles, levels, frame types, color config, loop filter, quantization, segmentation, tile info, CDEF, restoration, global motion, film grain, sequence header, and decode info - Add std_video_av1.xml registry and integrate into video.xml - Add VideoDecodeInputDesc variant for bitstream input - Implement DecodeFrame method for VideoDecodeCommandList (DX12/Vulkan) - Update C/C++ API headers to expose new AV1 types and functions - Implement backend support for DecodeFrame (DX12: DecodeFrame1, Vulkan: vkCmdDecodeVideoKHR) - Integrate libavif for AVIF sample handling in examples - Add AVIF demuxer and sample loading to example app - Enhance codegen for bitfields and string enums - Update documentation and references for new AV1 features * Add VideoDecoderParameters API for explicit codec params Introduce VideoDecoderParameters handle and wisVideoDecoderCreateParameters function for explicit management of codec-specific parameter sets (e.g., AV1 sequence headers). Update DecodeFrame APIs to require decoder parameters, and implement backend support for DX12 and Vulkan. Update documentation, XML, and codegen to reflect the new workflow. This decouples parameter management from frame decoding, improving extensibility and hardware decode support. * Replace libavif with mBMFF for AVIF parsing Switch AVIF parsing from libavif to mBMFF, updating CMake and source code accordingly. Remove avif_demux.hpp and migrate demuxing logic to entry_main.cpp using mBMFF. Add functions to extract AV1 sequence headers and build decoder parameters. Update Vulkan video decoder to set pNext and codec type correctly. Update avif_sample.avif to match new parsing logic. * Added H265 parsing Video now requires support for both maintenance 1 and 2 Removed static session parameters. Implemented DecodeFrame partioally Implemented InsertBarrier for video command list * Fixed header only build Refactored Video Example * Added Command list view and submit * Added Decoding parameters * Added video flags for buffer and texture * Video example (#265) * Added slicing of the video * Fixed memory binding to video session and leak of it * Added NAL unit size parsing (QnD) * fixed NAL parsing error * Changed the query for alignment * Fixed empty reference info and width + height problem * Added back decoder params * All the errors are gon. Access violation still exists * all errors gon * Added mutable video formats and removed sampled usage from video formats * Device removal on submission * The decoding doesn't work, releasig 0.7.1 without video/raytracing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.