feat(f1): 독립 pairing validator + 평가 파이프라인 연결 (V1 완료) - #34
Open
ohchanju3 wants to merge 29 commits into
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
요약
F1(V1) 파트 전체 완료 - 독립 pairing validator, source_type별 분리 집계, evaluate_ip.py/eval_best.py 연결, LLM, ASCP 결과 재채점 adapter
포함된 것
C1 - 독립 validator (
evaluation/validator.py)RL/environment.py::get_mask()를 재사용하지 않고 14개 violation code 전부 새로 구현 (base 출발/복귀, duty 내외 공항 연속성, connection/rest, duty/leg/pairing 기간, duplicate, unknown flight 등)duty_break_indices(실제 선택한 EndDuty 지점) 지원 - 있으면 그걸로, 없으면 gap 기반 휴리스틱으로 duty 분리validator_version/constraint_hashprovenance 기록 분리C2 - source_type별 분리 집계 (
evaluation/validation_report.py)aggregate_by_source()/aggregate_by_source_per_chunk()- policy_direct/salvage/repair/forced 4종류 분리(pairing 수, coverage, invalid 수, duplicate, deadhead 수, ManDays, FTC - 전부 flights 데이터로 독립 재계산)
C3 - 평가 파이프라인 연결 (
evaluation/evaluate_ip.py,evaluation/eval_best.py)collect_pool_full()이 pairing마다 실제 생성 base(_gen_base_airport)를 남기도록 수정validate_selected_pairings()— 최종 선택된 pairing을 독립 재검증,--strict-validation플래그로 invalid 발견 시 실패 처리 여부 선택save_result_json()(--save-json PATH) — 선택된 pairing(flight ID 포함) + 검증 결과를 JSON으로 저장하도록 수정eval_mode(strict/legacy)를 wandb run 이름, config, log에 반영eval_best.py는 evaluate_ip.py의 새 출력을 파싱해서 invalid selected pairing 수를 비교에 반영 - correctness가 성능(dead_time/deadhead)보다 항상 우선C4/C5 - adapter + 테스트
evaluation/llm_adapter.py- LLM 원시 출력 파싱 → pairing_record 변환evaluation/ascp_output_adapter.py-save_result_json()결과를 다시 읽어 재검증