Feature/optimization round 2 - #2
Open
LoveProgrammingMint wants to merge 3 commits into
Open
Conversation
… small-file shortcut - ComputeByteHistogram32/ComputeBlockEntropyStats accept Span/ReadOnlySpan to avoid array copies. - Standard block-entropy and ByteAnalysisHelper block stats use unsafe pointer loops to skip bounds checks. - ParsePeHeader works over ReadOnlySpan<byte>. - Flash path returns early with a single head region when file size <= FlashRegionSize.
…ion entropy - Revert ProHybrid from single-pass unified RegionStats back to calling optimized Standard/Flash/RawStat extractors. - Replace section entropy sampling with min(actual, 128KB) truncation for predictable cost. - Removes ~400 lines of complex segmented stat bookkeeping.
Contributor
Author
This will be my last commit |
Contributor
Author
此次提交不签署MINT协议，自动签署源仓库所签署的协议，保留原始作署权 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
PR 标题
perf(feature-engineering): simplify ProHybrid and vectorize loops without changing output
PR 正文
目标
在上一个 PR 合并后的基础上，继续优化
Xdows-Model-Invoker特征工程性能。保持特征维度与特征值不变，通过向量化/Span/unsafe 指针和算法简化降低耗时。改动范围
本次 PR 仅包含
upstream/master（b1060ad）之后的 3 个提交：debbefcperf(FeatureExtractor): Span-based APIs, unsafe block loops and Flash small-file shortcut2366cb4refactor(ProHybrid): simplify to composed extractors with capped section entropyd019094feat(Caller): add-dumpfeaturesmode for feature vector export主要改动
1.
ByteAnalysisHelper底层加速ByteClass字节分类查找表替代逐字节if/else。Math.Log2(p)。stackalloc long[256]替代堆分配。2.
ProHybridFeatureExtractor简化RegionStats分段单次遍历代码。FeatureExtractor+FlashFeatureExtractor+ProRawStatExtractor+ExtractStructuralFeatures。min(实际大小, 128 KB)截断顺序计算，成本可预测。3. 调试工具
-dumpfeatures <file> <out.csv>，用于导出 Standard/Flash/Pro 三组特征向量，方便验证特征一致性。性能对比（Debug 构建，相对
upstream/master）kernel32.dll(836 KB)kernel32.dll(836 KB)kernel32.dll(836 KB)onnxruntime.dll(10.5 MB)onnxruntime.dll(10.5 MB)onnxruntime.dll(10.5 MB)精度影响
使用
-dumpfeatures对比kernel32.dll与onnxruntime.dll的 Standard/Flash/Pro 三组特征向量：即本次优化在提速的同时完全未改变特征输出，对模型精度无影响。