Unity 6.5 EditMode: 14 pre-existing test failures (triage, none caused by EntityId rename)

[Scaler0222]

Background

Forking branch: unity6-compat.
Embedded copy in downstream project: Packages/com.besty.unity-skills/ at host project commit 5f81301 ("Embed local Unity 6.5 compat fork").

After migrating the package to Unity 6000.5.0b8 (the InstanceID -> EntityId rename across 33 files), 110 / 124 internal EditMode tests pass. The remaining 14 are pre-existing upstream defects or test fixture issues — none are caused by the EntityId rename. The Forge production baseline (761 tests) is unaffected.

Filing this so the work isn't lost if the package is ever re-pulled. Full triage doc is in the downstream project at Packages/com.besty.unity-skills/UNITY6_TEST_TRIAGE.md and reproduced below.

Test-suite numbers

Suite	Total	Pass	Fail
UnitySkills.Tests.Editor.dll	124	110	14

Source of truth: TestResults/EditMode-results.xml (run 2026-05-28; retries=3, all retry attempts failed).

Failures (a) — test fixture / env (2)

Same root cause for both: the test saves a .unity file under Assets/CodexTemp/RealValidation/ but the fixture never creates that folder. Unity emits Parent directory must exist before creating asset and EditorSceneManager.SaveScene returns false.

1. PerceptionSkillsTests.SceneDiff_SnapshotOnlyIncludesActiveSceneObjects — Tests/Editor/Core/PerceptionSkillsTests.cs:249
2. TestInfrastructureTests.TestRun_WhenAnotherRunIsActive_ReturnsErrorInsteadOfStartingConcurrentRunner — Tests/Editor/Core/TestInfrastructureTests.cs:85

Fix recipe

Add to the [SetUp] of each fixture (the pattern already exists in SelectionDrivenSkillTests.cs:22-25):

if (!AssetDatabase.IsValidFolder("Assets/CodexTemp"))
    AssetDatabase.CreateFolder("Assets", "CodexTemp");
if (!AssetDatabase.IsValidFolder("Assets/CodexTemp/RealValidation"))
    AssetDatabase.CreateFolder("Assets/CodexTemp", "RealValidation");

Failures (b) — upstream code <-> test drift (9)

SkillsModeManager never-list removal — 3 tests still expect old behavior

Editor/Skills/SkillsModeManager.cs:89-94 documents that the historical _explicitNeverList (scene_clear / scene_new / batch_apply) was removed in v1.9.x, because metadata flags now cover the 75 forbidden skills.

But three tests still call MakeSkill("scene_clear") with default op=Modify, risk="low" and assert Forbidden:

3. SkillsModeManagerTests.CheckAccess_ApprovalMode_NeverInSemiSkill_Forbidden — SkillsModeManagerTests.cs:312-314 — "Expected: Forbidden / But was: NeedsGrant"
4. SkillsModeManagerTests.IsForbiddenInSemi_CoversAllAutoJudgementBranches — SkillsModeManagerTests.cs:405-407 — "Names in the explicit never list must be forbidden / Expected: True / But was: False"
5. SkillsModeManagerTests.Allowlist_OverridesForbiddenInSemi_HighRiskSkillAllowed — SkillsModeManagerTests.cs:523-524 — "Expected: Forbidden / But was: NeedsGrant"

Fix: rewrite each MakeSkill("scene_clear") to a forbidden-by-metadata flavour (e.g. MakeSkill("del", op: SkillOperation.Delete)) — or drop the now-dead never-list coverage.

CurrentMode fresh-install default — code says Auto, test says Approval

6. SkillsModeManagerTests.CurrentMode_FreshInstall_NoKeys_DefaultsToApproval — SkillsModeManagerTests.cs:475 — "Expected: Approval / But was: Auto"

SkillsModeManager.cs:143 returns SkillsOperatingMode.Auto on fresh install. The test (and its #17 matrix label) expects Approval. Pick one: either the code regressed the default (revert line 143 to Approval) or the test is stale.

SkillRouter no longer emits unknownParams field

7. SkillValidationTests.Execute_WithUnknownTransformParameters_ReturnsStructuredErrorAndSuggestions — SkillValidationTests.cs:19 — "Expected: not null / But was: null"
8. SkillValidationTests.Execute_WithUnknownShaderParameter_SuggestsCanonicalParameter — SkillValidationTests.cs:35 — "Expected: not null / But was: null"

Both expect a top-level unknownParams JArray in the error envelope. That field is now absent. Either re-add it to the envelope or migrate the tests to the new shape.

SKILL.md <-> code drift (one test, many findings)

9. SkillDocumentationConsistencyTests.SkillDocumentation_ShouldMatchCodeDefinitions — SkillDocumentationConsistencyTests.cs:70

Concrete drift:

• 6 packages missing schema-first Exact Signatures block: addressables-design, dotween, dotween-design, netcode-design, unitask-design, yooasset-design.
• ~25 extra dotween_pro_add_animation.* / dotween_pro_stagger_animations.* params documented but not in the signatures.
• dotween_pro_batch_add_animation.dotween_pro_add_animation extra param.
• job_status.recentCount extra param.

Fix: regenerate the listed SKILL.md files from current code, or add the missing Exact Signatures: blocks. Runtime is unaffected.

Test-isolation defect — discovery cache leaks across tests

10. TestInfrastructureTests.TestList_WhenNoCachedDiscoveryExists_StartsAsyncDiscovery — TestInfrastructureTests.cs:240 — "Expected: False / But was: True"

The test asserts TestSkills.TestList() returns success=false when no cache exists, but the cache is populated by sibling tests / a previous CLI test pass. Need a [SetUp] that wipes the discovery cache (or a dedicated reset hook in BatchPersistence).

Failures (c) — Unity 6.5 / URP 17.5 regressions to verify (4)

SmartReplaceObjects returns "error"

11. SelectionDrivenSkillTests.SmartReplaceObjects_ReplacesSelectedObjectsWithPrefab — SelectionDrivenSkillTests.cs:132

The router envelope status is "error", so SmartReplaceObjects returned one of:

• "Prefab not found: ..." — AssetDatabase.LoadAssetAtPath<GameObject>(prefabPath) returned null. Likely a Unity 6.5 timing change around PrefabUtility.SaveAsPrefabAsset — the test destroys the source GO immediately after saving and never calls AssetDatabase.Refresh().
• "No objects selected" — Selection.gameObjects was empty.

Editor/Skills/SmartSkills.cs:516-537 does NOT use any renamed API. Candidate test-side repair: add AssetDatabase.Refresh(); AssetDatabase.SaveAssets(); after PrefabUtility.SaveAsPrefabAsset.

Shader Graph node / sub-graph operations fail on URP 17.5

The "is Shader Graph installed?" probe passes (URP 17.5 ships SG), so the tests don't Assert.Pass — they reach the actual operations and fail there.

12. ShaderGraphNodeEditingTests.ShaderGraphNodeEditing_WorksOnGraph — fails at ShaderGraphAddNode("Vector1Node", …) — ShaderGraphNodeEditingTests.cs:87
13. ShaderGraphNodeEditingTests.ShaderGraphNodeEditing_WorksOnSubGraph_AndRejectsInvalidOperations — fails at ShaderGraphCreateSubGraph(...) — ShaderGraphNodeEditingTests.cs:120
14. ShaderGraphSkillsTests.ShaderGraphConstrainedEditing_WorksWhenPackageInstalled — fails at ShaderGraphCreateSubGraph(...) — ShaderGraphSkillsTests.cs:69

The sibling ShaderGraphCreateGraph_WorksWithTemplateOrBlankFallback does pass, so blank-graph creation still works. Three of the four failures involve sub-graph creation or node addition specifically — likely a Unity 6.5 + URP 17.5 internal API change inside Editor/Skills/ShaderGraphSkills.cs.

Out of scope for this issue

The InstanceID -> EntityId rename itself (33 files) — that lives in the downstream project as commit 5f81301 and is not the cause of any failure here.

[Scaler0222]

Close-out

Three PRs shipped covering 12 of the 14 EditMode failures listed above. The remaining 2 are filed as their own follow-up issues with concrete acceptance criteria and reproducers, so neither is "stuck on this issue".

PRs

PR	Scope	Fixes
#3 — fix(tests): quick-win	Test-only, zero production .cs	#1.#1, #1.#2, #1.#7, #1.Besty0728#8, #1.Besty0728#10, #1.Besty0728#14 (6 fixed) + #1.Besty0728#13 quarantined to follow-up #2
#5 — fix(skills): drift	D2 SkillRouter envelope + R1/R2 fixtures + 7 docs	#1.#6, #1.Besty0728#11, #1.Besty0728#12 (3 fixed)
#7 — fix(shadergraph): URP 17.5 compat	AssetDatabase.Refresh() + InvokeMethod optional-defaults + R3 positive-lane gate	#1.#3, #1.#4, #1.#5 (3 fixed)

12 of 14 originally-failing tests now pass on Unity 6000.5.0b8 + URP 17.5.0. Forge baseline (761 host-project production tests) holds throughout. Both remaining failures are tracked separately:

• Unity 6.5 EditMode: 14 pre-existing test failures (triage, none caused by EntityId rename) #1.skill安装成功后是否需要一直开着unity内的server？ Besty0728/Unity-Skills#9 CurrentMode_FreshInstall_NoKeys_DefaultsToApproval → SkillsModeManager fresh-install default: revert Auto→Approval (deferred from #1 PR#2 — needs test-env setup first) #4 (D1 default-revert deferred; needs a test-environment [SetUpFixture] before the one-line code change can land safely)
• Unity 6.5 EditMode: 14 pre-existing test failures (triage, none caused by EntityId rename) #1.fix: 修复Debug Skill无法从Unity Console获取日志的问题 Besty0728/Unity-Skills#13 TestList_WhenNoCachedDiscoveryExists_StartsAsyncDiscovery → BatchPersistence discovery-cache leaks across tests (TestList_WhenNoCachedDiscoveryExists) #2 (BatchPersistence discovery-cache reset hook; cross-cutting design work)

Plus one regression surfaced in flight, tracked as:

• SmartReplaceObjects: ImportAsset(ForceSynchronousImport) no longer reliable after PR#2 test ordering shift #6 SmartReplaceObjects_… order-sensitivity (PR#5's R1/R2 test additions shifted NUnit's ordering enough that PR#3's ImportAsset(ForceSynchronousImport) fix is no longer reliable; needs Refresh() + retry-loop hardening)

Process notes (for the curious)

Run through Claude Octopus /octo:embrace — Phase 1 multi-LLM probe, Phase 2 consensus synthesis, Define→Develop adversarial debate gate, Phase 3 implementation, Phase 4 cross-PR validation.

The gate produced 4 remediations (R1-R4) that ended up materially shipping:

• R1 → PR#5 Execute_UnknownParamErrorEnvelope_PreservesNestedDetailsPath_ForLegacyClients (pins nested details.unknownParams after D2 widens the surface)
• R2 → PR#5 CurrentMode_MigrationMatrix_AllThreeBranchesHonorIntent + companion (proven its worth: PR#5's D1 attempt triggered exactly the 8-fixture regression cascade Codex predicted at this gate, leading to the D1 deferral to SkillsModeManager fresh-install default: revert Auto→Approval (deferred from #1 PR#2 — needs test-env setup first) #4)
• R3 → PR#7 ShaderGraphAdapter_BlankGraphPlusSingleNode_RoundTrips_R3PositiveLane (proven its worth: round 1 of PR#7 with only AssetDatabase.Refresh() saw R3.a pass but R3.b fail with the exact reflection error that diagnosed the InvokeMethod issue)
• R4 → PR#3 Assert.Ignore quarantine of the BatchPersistence test linked to BatchPersistence discovery-cache leaks across tests (TestList_WhenNoCachedDiscoveryExists) #2

Closing this issue when all 3 PRs merge.