Feat(#29): NB 모델 재학습 및 PII 마스킹 토큰 통일

[ch0412]

📝 개요

Spring PiiMaskingService 단일 마스킹 전환(#69) 이후 학습 데이터와 운영 입력 간 토큰 불일치 문제를 해결하기 위해 _normalize_text를 Spring 기준 토큰으로 통일하고 NB 모델을 재학습했습니다.

🔗 관련 이슈

Closes #29

🎯 주요 변경 사항

• train_sms.py, naive_bayes_analyzer.py — _normalize_text 토큰을 [PHONE], [ACCOUNT], [CARD], [RRN], [EMAIL]로 통일, _RE_LONG_NUM 제거, \b → lookaround 패턴 교체, _mask_pii 헬퍼 분리 (EMAIL을 URL보다 먼저 처리), has_phone OR 조건 추가
• phishing_model_artifact.pkl, phishing_vectorizer.pkl — 재학습 아티팩트 교체
• test_naive_bayes_analyzer.py — 토큰 변경에 따른 단위 테스트 수정
• test_nb_masking_accuracy.py — 마스킹 정확도 테스트 추가, is_available 체크 추가

📸 사진

✅ PR 체크리스트

• 관련 이슈를 연결했습니다.
• 구현 범위와 변경 이유를 설명했습니다.
• 로컬 테스트를 통과했습니다. (단위 테스트 5/5, 마스킹 정확도 평균 0.0점)
• API 변경 사항이 있다면 Swagger / API 명세에 반영했습니다.
• 민감 정보(API Key, 시크릿 키 등)가 코드·로그·테스트 데이터에 포함되지 않았습니다.
• 프론트엔드 또는 메인 백엔드(Spring)에 영향을 주는 응답 스키마 또는 Enum 변경이 있다면 팀에 공유했습니다.
• 병합(Merge) 전 작업 브랜치를 삭제하지 않았습니다.

Summary by CodeRabbit

• Bug Fixes

  • Improved detection and masking of sensitive information, including identification numbers, card numbers, phone numbers, account numbers, and email addresses.
  • Improved fraud and phishing analysis consistency when messages contain masked personal information.
  • Standardized masking labels across text analysis and message scoring.

• Tests

  • Added coverage to verify masking accuracy and maintain reliable risk scores after sensitive data is removed.

[coderabbitai]

📝 Walkthrough

Walkthrough

The change aligns Naive Bayes PII normalization with Spring-compatible bracketed tokens. It adds masking for RRN, card, account, phone, and email values, recognizes normalized phone markers, and validates score stability after masking.

Changes

PII token alignment

Layer / File(s)	Summary
PII normalization contracts app/analysis/text/naive_bayes_analyzer.py, data_science/SMSModel/train_sms.py	The analyzer and SMS model use dedicated PII patterns and replace values with [RRN], [CARD], [PHONE], [ACCOUNT], [EMAIL], [URL], and [AMOUNT]. Generic long-number normalization and legacy markers are removed.
Normalized phone feature recognition app/analysis/text/naive_bayes_analyzer.py, data_science/SMSModel/train_sms.py	Structural phone detection recognizes raw phone numbers and existing [PHONE] markers in batch and single-message paths.
Masking accuracy validation tests/analysis/text/test_nb_masking_accuracy.py, tests/analysis/text/test_naive_bayes_analyzer.py, pytest.ini	The tests apply Spring-matching PII masks, compare masked and unmasked analyzer scores, update token expectations, and enable automatic asyncio mode.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Sequence Diagram(s)

sequenceDiagram
  participant MessageInput
  participant NaiveBayesNormalizer
  participant PhoneFeatureExtractor
  participant ScoreEvaluator
  MessageInput->>NaiveBayesNormalizer: normalize PII values
  NaiveBayesNormalizer->>PhoneFeatureExtractor: pass text containing [PHONE]
  PhoneFeatureExtractor->>ScoreEvaluator: set structural phone feature
  ScoreEvaluator-->>MessageInput: return analyzer score

Loading

Possibly related issues

• SafeFam/SafeFam_BE issue 69: The normalization tokens and masking patterns match the Spring PII masking implementation.

Possibly related PRs

• SafeFam/SafeFam_AI#20: It modifies the same Naive Bayes normalization and structural phone detection paths.

Suggested reviewers: pearseona

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 inconclusive)

Check name	Status	Explanation	Resolution
Linked Issues check	❓ Inconclusive	The code updates token handling and phone-marker detection, but recall and replacement of excluded model artifacts cannot be verified.	Provide evidence for phishing recall of at least 0.96 and verify the excluded .pkl artifacts were replaced.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Out of Scope Changes check	✅ Passed	The configuration and test changes support the PII normalization and model validation objectives.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly describes the main changes: retraining the NB model and unifying PII masking tokens.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch feat/29-nb-retrain-pii-token

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 3

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@app/analysis/text/naive_bayes_analyzer.py`:
- Around line 46-52: Reorder the masking calls in the normalizer at
app/analysis/text/naive_bayes_analyzer.py lines 46-52 and the corresponding
normalizer at data_science/SMSModel/train_sms.py lines 89-96 so _RE_EMAIL runs
before _RE_URL; leave the remaining substitutions unchanged to ensure email
addresses are fully masked before URL processing.
- Around line 20-23: Update the numeric PII regexes _RE_RRN, _RE_CARD,
_RE_PHONE, and _RE_ACCOUNT in app/analysis/text/naive_bayes_analyzer.py at lines
20-23 to use (?<!\d) and (?!\d) instead of digit boundaries, while preserving
their existing numeric patterns. Apply the same boundary changes to the
corresponding regexes in data_science/SMSModel/train_sms.py at lines 71-74 so
both normalizers handle values adjacent to Hangul consistently.

In `@tests/analysis/text/test_nb_masking_accuracy.py`:
- Around line 55-60: Ensure both analyze_text_with_naive_bayes results report
is_available before extracting risk_score in the accuracy test. Fail the test
immediately when either inference is unavailable, then retain the existing score
calculation and difference assertion for successful results.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 60adb9a1-ffc9-4de6-87b5-128f96d0dc67

📥 Commits

Reviewing files that changed from the base of the PR and between ea97e9b and 0e643d4.

⛔ Files ignored due to path filters (2)

• data_science/SMSModel/phishing_model_artifact.pkl is excluded by !**/*.pkl
• data_science/SMSModel/phishing_vectorizer.pkl is excluded by !**/*.pkl

📒 Files selected for processing (5)

• app/analysis/text/naive_bayes_analyzer.py
• data_science/SMSModel/train_sms.py
• pytest.ini
• tests/analysis/text/test_naive_bayes_analyzer.py
• tests/analysis/text/test_nb_masking_accuracy.py