Audit a node's firewall reality when it changes, not on every poll - #55
Merged
Conversation
Measured on the production control plane: 29,958 of the last 30,000 audit events were netguard.reality.report, about 250k rows a day from 33 nodes reporting on an unattended timer. Every audit event is held in memory for the life of the process, so this one action was the engine behind today's OOM loop, and it would have refilled the memory that alpha-0.2.2a40 just freed within days. An unattended poll is not a security event. The reported firewall changing is. Gate the row on a fingerprint of the stored, canonicalized reality, with a six-hour floor so the trail still shows a quiet node reporting. Node id and collected_at are cleared before fingerprinting, because those are exactly the fields that differ between two reports of an unchanged firewall. This is not a new mechanism: shouldAuditSingBoxDiscovery already did precisely this for singbox.discover.report, which is why that action dropped from 28,984 of the first 30,000 events to 34 of the last 30,000. netguard reality never got the same treatment. The gate is cleared when a node is deleted, so a re-enrolled node with the same id records its reality again instead of inheriting a stale fingerprint. Tested: three tests covering the unchanged poll, the change plus the periodic floor, and per-node isolation including the delete path. Mutation-verified by forcing the gate open, which fails all three. Full suite and go vet green.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
生产实测:最近 30000 条审计事件里有 29958 条是
netguard.reality.report,33 台节点按定时器无人值守上报,约每天 25 万条。每条审计事件都要在内存里存活到进程结束,所以这一个 action 就是今天 OOM 循环的发动机,而且它会在几天内把alpha-0.2.2a40刚腾出来的内存重新填满。无人值守的一次轮询不是安全事件,防火墙变了才是。改成对存储后已规范化的 reality 取指纹来判断是否记录,并保留六小时下限,让安静的节点在轨迹里仍然看得见。取指纹前清掉 node id 和 collected_at,因为这两个字段正是同一份防火墙的两次上报之间唯一会变的东西。
这不是新机制:
shouldAuditSingBoxDiscovery早就为singbox.discover.report做过一模一样的事,所以那个 action 从最早 30000 条里的 28984 条降到最近 30000 条里的 34 条。netguard reality 一直没跟上。节点删除时清掉该节点的 gate,同 id 重新入网时会重新记录一次 reality,不会继承过期指纹。
测试:三个,覆盖不变轮询、变更加周期下限、跨节点隔离含删除路径。变异验证:把 gate 强制打开,三个全红。全量测试与 go vet 通过,server 包在 -race 下通过。