WebAssembly compilation

[bnason-nf]

Hi @traffaillac,

Thanks for creating this great library.

In case you're interested, to build this as a WebAssembly module, a few very minor changes were needed:

diff --git c/edge264.c w/edge264.c
index 9b47a57..38ddd04 100644
--- c/edge264.c
+++ w/edge264.c
@@ -135,7 +135,7 @@ Edge264Decoder *edge264_alloc(int n_threads, void (*log_cb)(const char *str, voi
        dec->taskPics_v = set8(-1);
        
        // select parser functions based on CPU capabilities and logs mode
-       #if defined(__SSE2__) // if compiled for Intel
+       #if defined(__SSE2__) && !defined(__wasm__) // if compiled for Intel
                __builtin_cpu_init();
                if (!__builtin_cpu_supports("cmov") || !__builtin_cpu_supports("sse2"))
                        return free(dec), NULL;
diff --git c/edge264_internal.h w/edge264_internal.h
index 337f4ca..2c0f7c5 100644
--- c/edge264_internal.h
+++ w/edge264_internal.h
@@ -660,7 +660,11 @@ static always_inline const char *unsup_if(int cond) { return cond ? " # unsuppor
  * _ ziphiN - interleave the high N-bit elements from two vectors
  */
 #if defined(__SSE2__)
+#if defined(__wasm__)
+	#include <immintrin.h>
+#else
        #include <x86intrin.h>
+#endif
        #define adds16(a, b) (i16x8)_mm_adds_epi16(a, b)
        #define addu8(a, b) (i8x16)_mm_adds_epu8(a, b)
        #define avgu8(a, b) (i8x16)_mm_avg_epu8(a, b)
@@ -714,7 +718,11 @@ static always_inline const char *unsup_if(int cond) { return cond ? " # unsuppor
        #define ziphi16(a, b) (i16x8)_mm_unpackhi_epi16(a, b)
        #define ziphi32(a, b) (i32x4)_mm_unpackhi_epi32(a, b)
        #define ziphi64(a, b) (i64x2)_mm_unpackhi_epi64(a, b)
+#if defined(__wasm__)
+       static always_inline size_t shld(size_t l, size_t h, int i) {return (size_t)((uint64_t)h << (uint64_t)i) | ((uint64_t)l >> (uint64_t)(SIZE_BIT - i));}
+#else
        static always_inline size_t shld(size_t l, size_t h, int i) {asm("shld %%cl, %1, %0" : "+rm" (h) : "r" (l), "c" (i)); return h;}
+#endif
        #ifdef __SSE4_1__
                #define cvtlo8u16(a) (i16x8)_mm_cvtepu8_epi16(a)
                #define cvtlo8s16(a) (i16x8)_mm_cvtepi8_epi16(a)

And more substantial changes were required for the make file, so I ended up creating a new CMake file instead, which I can provide if you like. I'm also happy to create a pull request for the above compilation fixes.

-Benbuck

[traffaillac]

Wow that was unexpected, thanks!!!

I had plans to add WebAssembly sometime later with LLVM vector extensions but that makes it much simpler :D

Do you want to make a MR that I can validate?

Can you provide a few commands showing how to compile for WebAssembly ? (feel free to edit README.md directly if through a MR)

As for the CMake I just approved an old MR that had been pending for too long. If you have time to edit from it that would be perfect, otherwise I'll gladly take your input and merge the two :)

[bnason-nf]

Hi @traffaillac,

Thanks for the response. Before I submit any build file changes, can we please discuss the approach? The thing with WebAssembly is that it has it's own SIMD instructions, which you can't directly write from C or C++. However, clang is able to convert many x86/64 and ARM SIMD instructions to WASM equivalents. The Emscripten project documentation discusses some of the details here: https://emscripten.org/docs/porting/simd.html.

I think this means that for Edge264 WASM builds, you need to pick either x86/64 or ARM as the build type. For my CMake file, I chose to build as x86/64 with all of the instructions sets (SSE2, SSE3, AVX, AVX2, etc.) enabled. I haven't done the equivalent ARM build, and so I can't compare the resulting WASM output instructions, and I don't have the time to work on that right now. Would you be OK with only having a x86/64 equivalent WASM build for now?

Also, I haven't gotten the log mode to work yet. I may spend more time on it later, but for now wouldn't be able to provide a way to have that with the WASM CMake builds. Would that be OK with you also?

I'm attaching the very simple CMake file I used to test with. Now that you have a CMake file in the project also, these would probably need to integrated as you say, but I'm providing it here just for reference:

cmake_minimum_required(VERSION 3.22)

project(edge264)

if(EMSCRIPTEN)
    set(WASM 1)
endif()

if (WASM)
    add_executable(edge264 edge264.c)
else()
    add_library(edge264 STATIC edge264.c)
endif()

target_compile_options(edge264
    PRIVATE
        -w

        $<$<CONFIG:Debug>:
            # using -O0 results in:
            # WebAssembly.compileStreaming(): Compiling function "decode_inter_luma" failed: local count too large
            -O1
        >
)

if(NOT WASM)
    target_compile_options(edge264 PRIVATE -flax-vector-conversions)

    add_executable(edge264_test edge264_test.c)
    target_link_libraries(edge264_test edge264)
else()
    target_compile_definitions(edge264
        PRIVATE
            __i386__
            __x86_64__
    )
    target_compile_options(edge264
        PRIVATE
            -msimd128
            -msse
            -msse2
            -msse3
            -mssse3
            -msse4.1
            -msse4.2
            -mavx
            -mavx2
    )
    target_link_options(edge264
        PRIVATE
            -lexports.js
            --no-entry
            -sALLOW_MEMORY_GROWTH=1
            -sEXPORTED_FUNCTIONS=_malloc,_free,_edge264_find_start_code,_edge264_alloc,_edge264_flush,_edge264_free,_edge264_decode_NAL,_edge264_get_frame,_edge264_return_frame
    )
endif()

Thanks,
Benbuck

[bnason-nf]

Here's another change needed for WebAssembly compilation if you enable HAS_X86_64_V2 or V3:

diff --git a/edge264.c b/edge264.c
index 38ddd04..6e6c7a5 100644
--- a/edge264.c
+++ b/edge264.c
@@ -149,11 +149,14 @@ Edge264Decoder *edge264_alloc(int n_threads, void (*log_cb)(const char *str, voi
        dec->parse_nal_unit[14] = dec->parse_nal_unit[20] = ADD_VARIANT(parse_nal_unit_header_extension);
        #ifdef HAS_X86_64_V2
                // macOS's clang does not support x86-64-v3/v2 feature string yet
+       #ifndef __wasm__
                if (__builtin_cpu_supports("popcnt") &&
                        __builtin_cpu_supports("sse3") &&
                        __builtin_cpu_supports("sse4.1") &&
                        __builtin_cpu_supports("sse4.2") &&
-                       __builtin_cpu_supports("ssse3")) {
+                       __builtin_cpu_supports("ssse3"))
+       #endif
+               {
                        dec->worker_loop = worker_loop_v2;
                        dec->parse_nal_unit[1] = dec->parse_nal_unit[5] = parse_slice_layer_without_partitioning_v2;
                        dec->parse_nal_unit[7] = dec->parse_nal_unit[15] = parse_seq_parameter_set_v2;
@@ -163,11 +166,14 @@ Edge264Decoder *edge264_alloc(int n_threads, void (*log_cb)(const char *str, voi
                }
        #endif
        #ifdef HAS_X86_64_V3
+       #ifndef __wasm__
                if (__builtin_cpu_supports("avx") &&
                        __builtin_cpu_supports("avx2") &&
                        __builtin_cpu_supports("bmi") &&
                        __builtin_cpu_supports("bmi2") &&
-                       __builtin_cpu_supports("fma")) {
+                       __builtin_cpu_supports("fma"))
+       #endif
+               {
                        dec->worker_loop = worker_loop_v3;
                        dec->parse_nal_unit[1] = dec->parse_nal_unit[5] = parse_slice_layer_without_partitioning_v3;
                        dec->parse_nal_unit[7] = dec->parse_nal_unit[15] = parse_seq_parameter_set_v3;

But unless I made a mistake in my testing, the generated WASM code is identical for these different modes except for the function names.

[bnason-nf]

I said something wrong earlier:

The thing with WebAssembly is that it has it's own SIMD instructions, which you can't directly write from C or C++.

This just isn't true, you can #include <wasm_simd128.h> and use the WASM 128 bit vector intrinsics directly. However, it would obviously have to be a new implementation path in addition to the x86/64 and ARM paths you already have.

Sorry for any confusion,
Benbuck

[traffaillac]

Intel compilation is perfectly fine for WebAssembly!

This just isn't true, you can #include <wasm_simd128.h> and use the WASM 128 bit vector intrinsics directly. However, it would obviously have to be a new implementation path in addition to the x86/64 and ARM paths you already have.

Don't worry I could search online and understand the full picture :) Actually I have been considering for a few months how to extend edge264 to be fully portable, and have been thinking about adding a 3rd implementation path that would be general-purpose and compile to all other architectures (with tolerance for suboptimal performance, hence the "Intel is fine for wasm"). I intended to go for Clang's vector builtins but looked this weekend and they are really incomplete at the moment. Other options are OpenCL but which has a bloated API, and WebAssembly but I am not sure whether possible to turn the resulting file into a static RISC-V or PowerPC lib, or (better) make clang output directly a proper lib. So this 3rd path will wait for the moment...

In the meantime I added wasm support as requested, yet without being able to test it at the moment. I also added a FORCEINTRIN make option so you can force the code to compile for Intel/ARM despite missing __SSE2__/__ARM_NEON. And I made the other inline asm optional so you could compile through ARM too :)

But unless I made a mistake in my testing, the generated WASM code is identical for these different modes except for the function names.

x86-64-v2 and x86-64-v3 variants are meant for runtime selection of binary code paths (one presumably compiled for x86-64-v1 that uses SSE2 instructions at most, one compiled for x86-64-v2 that uses SSE4 at most, and the last that uses AVX2 at most). Since WASM is itself an IR it will be translated to machine code, so you may want to compile for the most capable CPU and the later translation to machine code will replace the missing instructions if necessary. FORCEINTRIN=x86 does that, by making it look like a full-featured Intel CPU (it injects macros SSE2/SSSE3/SSE4_1/BMI1 that edge264 tests to output later instructions). It should improve performance by a good margin since previously you didn't have SSSE3 enabled.

If instead you want to make a runtime selection of WASM binaries then you need to find an alternative to __builtin_cpu_supports, tell me if this is the case and we can discuss it more here.

Once I'm able to test compilation for WASM I'll add a snippet to show how to compile for it.

Cheers,
Thibault

[traffaillac]

Hi @bnason-nf,

I took some time to try to compile to wasm, and was definitely wrong in my previous post, we don't need a FORCEINTRIN option as emcc has -m options already (sorry I didn't read well enough your cmake file 😅). And I erroneously assumed your code didn't have SSSE3 enabled but it certainly was. I was mislead by the +#if defined(__wasm__) next to the #if defined(__SSE2__) and not the other __SSSE3__ and __SSE4_1__ tests, but emcc handles all of that so no need to add new tests.

With emmake make CFLAGS="-msimd128 -mavx" BUILD_TEST=no (and by removing -march=native) I managed to compile edge264 itself, but still need to investigate on edge264_test. -mavx2does not work on my machine, but that is not a problem here as AVX2 is not used yet.

And I did test wasm compiling through ARM NEON which does not work, need to work further here.

Cheers,
Thibault

[yume-chan]

I also tried to build it for WebAssembly before seeing this issue: https://github.com/tvlabs/edge264/compare/master...yume-chan:edge264:master?w=1 (just run emmake make)

Baseline profile level 4 works, and the performance is about 5x comparing to tinyh264.

---

Baseline profile level 1.2 fails this assertion

edge264/src/edge264_headers.c

Line 1213 in b87eb34

assert(__builtin_popcount(reference_frames & same_views) <= sps->max_num_ref_frames);

because max_num_ref_frames is 0 (MaxDpbFrames is 0)

edge264/src/edge264_headers.c

Lines 1924 to 1925 in b87eb34

	int MaxDpbFrames = min((MaxDpbMbs[min(level_idc, 63)] / (unsigned)(sps.pic_width_in_mbs * sps.pic_height_in_mbs)) << mvc, 16);
	sps.max_num_ref_frames = min(max_num_ref_frames, MaxDpbFrames >> mvc);

EDIT: I think it's the encoder mislabels the level, some decoders ignore the level and decode anyway, but not edge264

---

Main/High profile level 4.2 decodes several frames then leaving ready_task as 0

edge264/src/edge264_headers.c

Line 1252 in b87eb34

dec->ready_tasks |= ((dec->task_dependencies[task_id] & ~ready_frames(dec)) == 0) << task_id;

causing worker_loop to pick up an invalid task

edge264/src/edge264_headers.c

Line 446 in b87eb34

int task_id = __builtin_ctz(c.d->ready_tasks); // FIXME arbitrary selection for now

---

Really don't know whether these are issues in my porting process or the library itself.

---

adding a 3rd implementation path that would be general-purpose and compile to all other architectures (with tolerance for suboptimal performance, hence the "Intel is fine for wasm").

Like https://github.com/simd-everywhere/simde? For example with shuffle3

edge264/src/edge264_internal.h

Line 871 in b411fc7

static always_inline i8x16 shuffle3(const i8x16 *p, i8x16 m) {return vqtbl3q_s8(*(int8x16x3_t *)p, m);}

You can just leave this one (or change to simde_vqtbl3q_s8 if not enable native alias), and simde will simulate it on other platforms.

---

If instead you want to make a runtime selection of WASM binaries then you need to find an alternative to __builtin_cpu_supports, tell me if this is the case and we can discuss it more here.

Currently there is only one batch of WASM SIMD instructions, so a browser either supports all or none. On unsupported browsers, .wasm files with SIMD instructions simply don't load.

So for best compatibility we need to compile two .wasm files, one with SIMD and one without. But all major browsers since 2023 already supports SIMD so it's mostly unnecessary.