ablation_matrix.csv

ablation_id,research_question,full_component,removed_or_replaced_component,simple_alternative,controlled_variables,expected_observation,experiment_ids,status,conclusion
A-V01,"Does visual marking improve contact grounding?","raw plus marked identity packet","marked view removed or color swapped","raw full image only","model prompt candidate bank query budget","grounding gain without marker-dependent identity errors","VLM-S01","planned",""
A-V02,"Does verification add utility beyond generation?","proposer plus balanced verifier","verifier removed","proposal frequency only","fixed candidates geometry optimizer","positive residual utility and reduced confirmation errors","VLM-S02;VLM-S08","planned",""
A-V03,"How many samples are justified?","N=5","N=1/3/10","single deterministic generation","model prompt K events","utility and consistency saturate before cost explodes","VLM-S03","planned",""
A-V04,"Which consistency probe predicts harm?","all consistency probes","remove paraphrase flip swap or crop one at a time","sampling entropy only","fixed edge geometry and query budget","at least one probe adds fixed-candidate utility signal","VLM-S04;VLM-S05","planned",""
A-V05,"Is verbal confidence trustworthy?","calibrated multi-evidence","self-reported confidence only","option margin only","same candidates and calibration split","raw verbal confidence is worse calibrated and less discriminative","VLM-S06","planned",""
A-V06,"Are VLM features model agnostic?","per-model standardized evidence","raw features","small target recalibration","same events candidates optimizer","standardization or recalibration recovers part of transfer loss","VLM-S07","planned",""
A-V07,"Can queries be budgeted adaptively?","uncertainty-triggered probes","always run all probes","single proposer and verifier","same HarmRate target and coverage","similar safety with fewer queries","VLM-S09","planned",""
A-V08,"Does 3D counterfactual rendering add evidence?","RGB plus highlighted SMPL-X renders","3D renders removed","RGB marked image","same candidates token budget and selector capacity","3D render adds utility beyond RGB or is rejected as redundant","VLM-S10","planned",""
A-I1-01,"Can semantic contact truth substitute for intervention utility?","paired vector outcome","utility reduced to semantic GT","semantic GT filter","same116 records and outcome protocol","semantic filter leaves nonzero harm and loses helpful interventions","CIU-BENCH-PILOT-V1","completed","harm30.0% safety26.7% helpful retention32.5%; truth is not utility"
A-I1-02,"Does a reversible K100 probe predict both benefit and safety?","K100 response plus K1000 endpoint","K1000 endpoint removed","K100 observable policy","balanced81 candidates and frozen5mm/safety thresholds","high benefit correlation but residual late safety failures","CIU-BENCH-PILOT-V1","completed","Spearman0.907 but9/38 accepted candidates violate full safety"
A-I1-03,"Is evaluator-congruent per-step safety cheaper than paired endpoints?","TwinGuard full-mesh certificate","stepwise certificate removed","direct O0+O1 endpoints","same BR1-10-2 initializer edge optimizer weights and device","certificate must remain below1.25x paired runtime","GATE-I1-TWINGUARD-TARGETED-R4","completed; rejected","runtime1.77195x; remaining matrix stopped"
A-I1-04,"Should VLM and geometry evidence be concatenated or assigned separate roles?","VLM proposer plus geometry harmful-risk filter","geometry risk removed or raw early feature concatenation","VLM-only and Geometry+VLM logistic selectors","same124 outcomes five grouped folds model capacity and coverage","role-specific geometry risk should reduce harmful Qwen proposals without collapsing helpful retention","CIU-CORE-SELECTOR-V1;CIU-VAL-HELDOUT-CONFIRMATION-V1","completed;strict VAL gate failed","Early fusion fails;role-specific filter transfers safety on VAL with0/2 harm but retains only1/4 helpful at frozen floor-25% point"
A-I1-05,"Is a nominal25% policy well-defined for a small abstaining proposal set?","pre-outcome floor-25% accept rule","rounding and minimum accepted count varied only posthoc","ranked prefix diagnostic","same frozen scores and11 revealed VAL outcomes;no refit or relabel","discrete coverage may explain retention failure without invalidating risk ranking","CIU-VAL-HELDOUT-CONFIRMATION-V1","completed as posthoc diagnosis only","floor gives2/11 and25% retention;top3 diagnostic gives0 harm and50% retention but cannot override failed Gate or define TEST policy"