Skip to content

Two-pass min1/min2 cnProc for BG1 and BG2. #282

Open
rorsc wants to merge 9 commits into
developfrom
cnproc-twopass
Open

Two-pass min1/min2 cnProc for BG1 and BG2. #282
rorsc wants to merge 9 commits into
developfrom
cnproc-twopass

Conversation

@rorsc

@rorsc rorsc commented Jul 9, 2026

Copy link
Copy Markdown
Member

nrLDPC_cnProc_BG1_2pass and nrLDPC_cnProc_BG2_2pass replace the LUT-exclude-self approach with a two-pass algorithm:

  • Pass 1: single sweep over all numBN inputs to collect min1, min2 (two smallest |vk|) and the full sign product via sign_epi8.
  • Pass 2: re-read each vk, select min2 where |vk|==min1 (tie approx), remove self sign with sign_epi8(vsgn_all, vk), and store.

Memory reads per CN group: 2*numBN vs numBN*(numBN-1) for LUT approach:

  numBN= 4:  8 vs  12  (1.5x  fewer)
  numBN= 7: 14 vs  42  (3.0x  fewer)
  numBN=10: 20 vs  90  (4.5x  fewer)
  numBN=19: 38 vs 342  (9.0x  fewer)

A generic nrLDPC_cnProc_group_2pass(buf, res, numBN, M, off) helper
does the work; the BG1/BG2 wrappers loop over their degree groups and
derive the BN stride from lut_numCnInCnGroups_BG1/2_R13/15 exactly as
the existing functions do. No new LUT tables are required.

Placed inside the #else (non-AVX512) block alongside the existing
BG1/BG2 functions; AVX512 path is unchanged.

It also removes the cnProc/bnProc/bnProcPC generators and simplifies the original implementation (several hundreds of lines of
repetitive code replaced by generic functions.)

RaymondKnopp and others added 9 commits July 9, 2026 18:04
nrLDPC_cnProc_BG1_2pass and nrLDPC_cnProc_BG2_2pass replace the
LUT-exclude-self approach with a two-pass algorithm:

  Pass 1: single sweep over all numBN inputs to collect min1, min2
          (two smallest |vk|) and the full sign product via sign_epi8.
  Pass 2: re-read each vk, select min2 where |vk|==min1 (tie approx),
          remove self sign with sign_epi8(vsgn_all, vk), and store.

Memory reads per CN group: 2*numBN vs numBN*(numBN-1) for LUT approach:
  numBN= 4:  8 vs  12  (1.5x  fewer)
  numBN= 7: 14 vs  42  (3.0x  fewer)
  numBN=10: 20 vs  90  (4.5x  fewer)
  numBN=19: 38 vs 342  (9.0x  fewer)

A generic nrLDPC_cnProc_group_2pass(buf, res, numBN, M, off) helper
does the work; the BG1/BG2 wrappers loop over their degree groups and
derive the BN stride from lut_numCnInCnGroups_BG1/2_R13/15 exactly as
the existing functions do.  No new LUT tables are required.

Placed inside the #else (non-AVX512) block alongside the existing
BG1/BG2 functions; AVX512 path is unchanged.

Assisted-by: Claude:claude-sonnet-4.6
Signed-off-by: Raymond Knopp <raymond.knopp@eurecom.fr>
Add nrLDPC_cnProc_group_2pass_128() — a native 128-bit variant of the
two-pass min1/min2 CN processor — and restructure the file so the
two-pass section compiles on every target.

Changes:
- Close the #if !AVX512BW guard immediately after the existing generic
  BG1/BG2 256-bit functions (line ~857); the two-pass section is now
  outside that guard so it is visible on all platforms.
- Add #if defined(__AVX2__) || defined(__AVX512BW__) guard around the
  256-bit nrLDPC_cnProc_group_2pass kernel; the 128-bit kernel below
  it is always compiled.
- nrLDPC_cnProc_BG1_2pass / _BG2_2pass wrappers now dispatch at
  compile time:
    AVX2 / AVX512: 256-bit path, M = ceil(numCN*Z/32), off >>= 5
    aarch64 / SSE2: 128-bit path, M = ceil(numCN*Z/16), off >>= 4
  Each simde__m128i op maps to a single NEON instruction on aarch64,
  matching the code-generation strategy used by the existing
  cnProc128/ generated functions.

Compile-tested: -mavx2       → exit 0 (256-bit path taken)
  -mno-avx2    → exit 0 (128-bit path taken, one pre-existing warning)
  -mavx512bw   → pre-existing errors in nrLDPC_cnProc_avx512.h
                 (ones512_epi8 undeclared); not caused by this change.
Signed-off-by: Raymond Knopp <raymond.knopp@eurecom.fr>
Assisted-by: Claude:claude-sonnet-4.6
Add //#define TWOPASS_CN_PROC alongside UNROLL_CN_PROC=1.
When UNROLL_CN_PROC is commented out and TWOPASS_CN_PROC is
defined, all four cnProc call sites (first-iter BG1/BG2,
loop BG1/BG2) dispatch to nrLDPC_cnProc_BG1_2pass /
nrLDPC_cnProc_BG2_2pass instead of the LUT-based variants.

To benchmark: comment UNROLL_CN_PROC and uncomment TWOPASS_CN_PROC.
To revert to LUT generic: comment both.

Assisted-by: Claude:claude-sonnet-4.6
Signed-off-by: Raymond Knopp <raymond.knopp@eurecom.fr>
Remove TWOPASS_CN_PROC guard now that the two-pass path is the only
non-unrolled option:

  - Comment out #define UNROLL_CN_PROC 1 so the two-pass path is
    active by default on all targets (AVX2, AVX512, aarch64/NEON).
  - Collapse the four dispatch sites from
      #ifndef UNROLL_CN_PROC
        #ifdef TWOPASS_CN_PROC ... #else ... #endif
      #else  <unrolled switch>
    to the simpler
      #ifndef UNROLL_CN_PROC
        nrLDPC_cnProc_BG{1,2}_2pass(...)
      #else  <unrolled switch>
  - The unrolled path (UNROLL_CN_PROC) is retained for benchmarking
    and will be removed in a follow-up once the two-pass path is
    confirmed as the permanent default.

Tested on: AMD Ryzen (AVX2), Intel (AVX512), Rockchip A76 (NEON),
           NVIDIA Neoverse (DGX Spark, GH200).

Signed-off-by: Raymond Knopp <raymond.knopp@eurecom.fr>
Assisted-by: Claude:claude-sonnet-4.6
Add nrLDPC_cnProc_group_2pass_512() — a native 512-bit variant of the
two-pass min-sum CN processor — and update the BG1/BG2 wrappers to
dispatch to the widest available register set.

Kernel design:
- Uses AVX512BW mask-register comparisons (simde__mmask64) and
  predicated blend (simde_mm512_mask_blend_epi8) for single-cycle
  select instead of the vector-mask XOR trick used in the 256-bit path.
- Pass 1: simde_mm512_abs/xor/min_epu8/max_epu8 — same structure as
  256-bit kernel, 64 CNs per iteration.
- Pass 2: eq_mask = cmpeq_epi8_mask(|vk|, vmin1);
          out_mag = mask_blend_epi8(eq_mask, vmin1, vmin2);
          neg_mask = cmpgt_epi8_mask(zeros, other_xor);
          result = mask_blend_epi8(neg_mask, out_mag, -out_mag)

Wrapper dispatch (compile-time):
  __AVX512BW__: 512-bit, M = ceil(numCN*Z/64), off >>= 6
  __AVX2__:     256-bit, M = ceil(numCN*Z/32), off >>= 5
  else:         128-bit, M = ceil(numCN*Z/16), off >>= 4

All buffer strides (lut_numCnInCnGroups_{BG1_R13,BG2_R15}[grp] *
NR_LDPC_ZMAX) are divisible by 64, so the 512-bit stride is exact.

Compile-tested: -mavx512bw   → exit 0 (512-bit path)
  -mavx2       → exit 0 (256-bit path)
  -mno-avx2    → exit 0 (128-bit path, pre-existing Wpsabi warning)
Signed-off-by: Raymond Knopp <raymond.knopp@eurecom.fr>
Assisted-by: Claude:claude-sonnet-4.6
Replace 1360-line bnProc.h (BG1-only, 256-bit only, 13 copy-pasted degree
groups) and all 36 generated bnProc/bnProcPc #includes with two generic
loops over the 30 BN-degree groups, dispatching at compile time to:

  - AVX512BW: 512-bit accumulation in bnProcPc (mm512_cvtepi8_epi16 +
    mm512_cvtsepi16_epi8), 512-bit subs in bnProc
  - AVX2:     existing 256-bit paired-128 widen/accumulate/pack pattern
  - 128-bit:  hi/lo split via mm_srli_si128, always compiled

Both functions now cover all BG variants (BG1/BG2, all rates) without
rate-specific dispatch in the decoder.  bnProc includes degree-1 BNs
(previously special-cased) via the generic loop.  nrLDPC_decoder.c
dispatch blocks for UNROLL_BN_PROC / UNROLL_BN_PROC_PC are removed;
UNROLL_CN_PROC remains commented-in for benchmarking.

Assisted-by: Claude:claude-sonnet-4.6
Signed-off-by: Raymond Knopp <raymond.knopp@eurecom.fr>
…build

Signed-off-by: Raymond Knopp <raymond.knopp@eurecom.fr>
Signed-off-by: Robert Schmidt <robert.schmidt@openairinterface.org>
@rorsc rorsc added the 5G-NR Perform 5G Tests label Jul 9, 2026
@rorsc rorsc temporarily deployed to ci-approval July 9, 2026 16:07 — with GitHub Actions Inactive
@rorsc rorsc requested a review from LaurentThomas July 9, 2026 16:08
Comment thread doc/Doxyfile
@CMAKE_CURRENT_SOURCE_DIR@/../openair1/PHY/CODING/nrLDPC_encoder/ldpc_BG2_Zc208_byte.c \
@CMAKE_CURRENT_SOURCE_DIR@/../openair1/PHY/CODING/nrLDPC_encoder/ldpc240_byte.c \
@CMAKE_CURRENT_SOURCE_DIR@/../openair1/PHY/CODING/nrLDPC_encoder/ldpc_BG2_Zc352_byte_128.c \
@CMAKE_CURRENT_SOURCE_DIR@/../openair1/PHY/CODING/nrPolar_tools/nr_polar_defs.h \

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wrong, revert

int8_t bnProcBuf[NR_LDPC_SIZE_BN_PROC_BUF] __attribute__ ((aligned(64)));
int8_t bnProcBufRes[NR_LDPC_SIZE_BN_PROC_BUF] __attribute__ ((aligned(64)));
int8_t llrRes[NR_LDPC_MAX_NUM_LLR] __attribute__ ((aligned(64)));
int8_t llrProcBuf[NR_LDPC_MAX_NUM_LLR] __attribute__((aligned(64)));

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

check which ones really don't need to be memsetted

@rorsc rorsc added the retrigger-ci Re-run CI label Jul 9, 2026
@rorsc rorsc temporarily deployed to ci-approval July 9, 2026 16:33 — with GitHub Actions Inactive
@github-actions github-actions Bot removed the retrigger-ci Re-run CI label Jul 9, 2026
@durantabot

Copy link
Copy Markdown
Collaborator

CI Build: #771 | Failed on the following stages:

Validation: SHA recognized in 3fa6e71, using "origin/cnproc-twopass" as branch name
The following commit(s) are missing a Signed-off-by:

3fa6e71

Please use 'git commit -s' to sign your commits.
For detailed instructions, refer to the CONTRIBUTING file at the root of this repository.

@durantabot

Copy link
Copy Markdown
Collaborator

CI Build: #773 | Failed on the following stages:

Validation: SHA recognized in 3fa6e71, using "origin/cnproc-twopass" as branch name
The following commit(s) are missing a Signed-off-by:

3fa6e71

Please use 'git commit -s' to sign your commits.
For detailed instructions, refer to the CONTRIBUTING file at the root of this repository.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

5G-NR Perform 5G Tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants