Skip to content

fix(hw-gate): accept a decide verdict written as a markdown headline - #711

Merged
Kaden-Schutt merged 1 commit into
masterfrom
fix/hw-gate-fable-markdown-decision
Sep 4, 2026
Merged

fix(hw-gate): accept a decide verdict written as a markdown headline#711
Kaden-Schutt merged 1 commit into
masterfrom
fix/hw-gate-fable-markdown-decision

Conversation

@Kaden-Schutt

Copy link
Copy Markdown
Collaborator

Summary

Run 33905366422 (#691): Fable's decide phase returned a complete, correct investigation — as a markdown report headed ## hw-gate decide — PR #691: **merge-staging** instead of a JSON object. review.py reported "no JSON object in assistant text", decision=None, decision_final=hold — red run on a green verdict, and the staging merge never happened.

(This class of failure is why the ladder loses a run: the seat did the work and the gate lost it at the serializer.)

Change

  • _markdown_decision() recognizes the decide headline / bold decision word (**merge-staging**, **decision:** hold, decision: block) within the report head and synthesizes the decision dict; the full text is kept as announcement/rationale and in fable_raw.markdown_fallback.
  • Wired into both decide paths: omp_investigate's no-JSON branch and omp_review's retry loop (decide phase only).
  • fable_raw is now also set on the non-investigate failure path (the omp_review error message carries the assistant tail).

Evidence

test_decide_markdown_headline_is_a_decision: a headline verdict yields decision_final=merge-staging and the staging merge fires; prose without a verdict word yields hold with fable_error + raw tail in the artifact. Full suite: 104/104.

Which surface(s) does this touch?

  • policy filesscripts/hw-gate/review.py (hard floor: a human merges this)

Run 33905366422 (#691): Fable's decide phase returned a complete, correct
investigation — as a markdown report headed `## hw-gate decide — PR #691:
**merge-staging**` instead of a JSON object. review.py reported "no JSON
object in assistant text", decision=None, decision_final=hold, and the
run went red on a complete green verdict.

_markdown_decision() recognizes the decide headline / bold decision word
(`**merge-staging**`, `**decision:** hold`, `decision: block`) and
synthesizes the decision dict; wired into both decide paths
(omp_investigate's no-JSON branch and omp_review's retry loop).
fable_raw is now also set on the non-investigate failure path.

test_decide_markdown_headline_is_a_decision: headline verdict →
merge-staging decision + staging merge; prose without a verdict word →
hold with fable_error + raw tail in the artifact. 104/104.
@Kaden-Schutt
Kaden-Schutt merged commit 3bd2914 into master Sep 4, 2026
@hipfire-sol

hipfire-sol Bot commented Sep 4, 2026

Copy link
Copy Markdown

hw-gate sol prelim

summary: Adds a decide-phase fallback that recognizes a Fable verdict expressed in the first 2,000 characters of a markdown report, synthesizes the decision object, preserves raw assistant text, and allows the existing floor and staging-merge logic to consume it; also preserves the assistant tail when the non-investigate path remains unparseable.

run_hardware: true
run_hardware_reasons: The changed Python policy code only parses already-captured assistant text and records it in gate artifacts; it adds no network, credential, filesystem, dependency, toolchain, build, unsafe, or process-spawning behavior beyond the gate's existing merge path.; Policy-file changes require a human decision even when tests pass.; Execution should focus on the isolated hw-gate test suite; model fixtures cannot meaningfully validate this serializer/parser behavior.

routes:

mode tag source why
no routes

unavailable_routes:

(none)

claim_assessment: The author claims markdown-headline verdicts now produce decision_final=merge-staging and invoke the staging merge, while verdict-free prose remains hold with error and raw-tail evidence, with 104 tests passing. This is proved by running the complete scripts/hw-gate test suite and inspecting focused positive, negative, and adversarial parser cases; model-generation routes do not exercise the claim.

questions_for_author:

  • Why do the second and third fallback patterns search anywhere in the first 2,000 characters rather than requiring a heading or an unambiguous standalone verdict line, given that quoted prior output such as “previous decision: merge-staging was wrong” could authorize a merge?
  • Please add negative tests for quoted, negated, and conflicting verdict phrases before relying on this fallback as merge authorization.

@hipfire-sol

hipfire-sol Bot commented Sep 4, 2026

Copy link
Copy Markdown

hw-gate evidence — 2 lane(s) — verdict pass

lane hiptrx (gfx1201)

hw-gate evidence

field value
base f025b2ec9c842833109b3c858937997b9cb36cc7
head 473128588b82bdf2d22a91a12c775d3579a64ea2
buckets kernel,load,serve
host gfx gfx1201
host rocm 7.15.26333-0000000
device 3
runner hiptrx
daemon_md5 1315aac7d3e7f11c031be856e521375c
hipfire_md5 970c4e5d77df047a37c1d938902d51dc
build_seconds 11.835463047027588
verdict pass
logs_dir hw-gate-logs

fixtures

qwen3.6:27b

source: bucket sha256_ok: ✅ size_ok: ✅ status: pass reason:

battery — exit 0 seconds 33.2 status pass

mode idx genre finish ctx cached gen ans_words prefill_tok_s decode_tok_s attractor empty runaway recall_ok
battery 0 stop 34 0 132 62 3.1 15.1 False False False True
battery 1 stop 30 0 79 66 504.2 36.1 False False False True
battery 2 stop 31 0 16 13 522.3 36.2 False False False True
battery 3 stop 47 0 31 8 560.8 36.1 False False False True
battery 4 stop 47 0 83 45 563.5 36.0 False False False True
qwen3.6:27b battery turn 0
```python
def longest_substring_without_repeating(s: str) -> str:
    char_index = {}
    start = 0
    max_len = 0
    max_start = 0

    for end, char in enumerate(s):
        if char in char_index and char_index[char] >= start:
            start = char_index[char] + 1
        char_index[char] = end
        if end - start + 1 > max_len:
            max_len = end - start + 1
            max_start = start

    return s[max_start:max_start + max_len]
```
qwen3.6:27b battery turn 1
During the day, sunlight scatters off gas molecules in the atmosphere, with shorter blue wavelengths deflecting more easily than other colors. However, at sunset, the sun is lower on the horizon, forcing light to travel through a much thicker layer of air. This extended path scatters away most of the blue and green light, allowing only the longer red and orange wavelengths to reach your eyes.
qwen3.6:27b battery turn 2
The capital of France is Paris, and the River Seine runs through it.
qwen3.6:27b battery turn 3
{
  "name": "Alice",
  "age": 34,
  "city": "Lisbon"
}
qwen3.6:27b battery turn 4
To calculate 17 + 26:

1.  Add the ones digits: 7 + 6 = 13. Write down 3 and carry over 1.
2.  Add the tens digits along with the carry: 1 + 2 + 1 = 4.
3.  Combine the results: 43.

Answer: 43

chain — exit 0 seconds 21.5 status pass

mode idx genre finish ctx cached gen ans_words prefill_tok_s decode_tok_s attractor empty runaway recall_ok
chain 0 stop 44 0 113 46 508.2 35.8 False False False True
chain 1 stop 213 158 207 123 445.8 35.7 False False False True
chain 2 stop 446 421 80 68 299.8 35.7 False False False True
chain 3 stop 560 527 89 66 304.9 35.6 False False False True
chain 4 stop 681 650 80 58 359.1 35.6 False False False True
qwen3.6:27b chain turn 0
```python
def merge_sorted(a, b):
    """Merge two sorted lists into one sorted list."""
    merged = []
    i, j = 0, 0
    while i < len(a) and j < len(b):
        if a[i] <= b[j]:
            merged.append(a[i])
            i += 1
        else:
            merged.append(b[j])
            j += 1
    merged.extend(a[i:])
    merged.extend(b[j:])
    return merged
```
qwen3.6:27b chain turn 1
To find the total distance traveled, we calculate the distance for each leg of the trip separately and then add them together.

**Step 1: Calculate the distance for the first leg**
*   Speed = 60 mph
*   Time = 2.5 hours
*   Distance = Speed × Time
*   Distance₁ = 60 × 2.5 = 150 miles

**Step 2: Calculate the distance for the second leg**
*   Speed = 40 mph
*   Time = 1.5 hours
*   Distance = Speed × Time
*   Distance₂ = 40 × 1.5 = 60 miles

**Step 3: Calculate the total distance**
*   Total Distance = Distance₁ + Distance₂
*   Total Distance = 150 + 60 = 210 miles

**Final Answer:**
The train traveled a total of **210** miles.
qwen3.6:27b chain turn 2
The seasons on Earth are caused by the planet's tilted axis, which remains oriented in the same direction as it orbits the Sun. This tilt means that different hemispheres receive varying amounts of direct sunlight throughout the year, rather than the distance from the Sun being the primary factor. Consequently, when a hemisphere is tilted toward the Sun, it experiences summer, while the hemisphere tilted away experiences winter.
qwen3.6:27b chain turn 3
Elias had tended the Cape Hatteras lighthouse for thirty years, weathering countless storms without surprise. One foggy morning, however, a peculiar wooden box lodged between the jagged rocks caught his eye. Inside, he discovered not driftwood or debris, but a perfectly preserved, leather-bound journal from a ship lost a century ago. The final entry described a map to a hidden cove, rewriting Elias's quiet life forever.
qwen3.6:27b chain turn 4
1. Use descriptive and consistent naming conventions for variables, functions, and classes.
2. Write modular code by breaking functionality into small, reusable functions.
3. Keep functions short and focused on a single responsibility.
4. Write clear, concise comments that explain why, not just what the code does.
5. Consistently follow a style guide and use automated formatting tools.

ornith-1.5:35b-a3b-mq4r

source: bucket sha256_ok: ✅ size_ok: ✅ status: pass reason:

battery — exit 0 seconds 31.8 status pass

mode idx genre finish ctx cached gen ans_words prefill_tok_s decode_tok_s attractor empty runaway recall_ok
battery 0 stop 34 0 104 44 23.2 12.4 False False False True
battery 1 stop 30 0 92 81 452.1 55.2 False False False True
battery 2 stop 31 0 16 13 462.2 61.6 False False False True
battery 3 stop 47 0 22 6 663.5 69.3 False False False True
battery 4 stop 47 0 17 7 681.2 49.9 False False False True
ornith-1.5:35b-a3b-mq4r battery turn 0
```python
def longest_substring_without_repeating(s: str) -> str:
    seen = set()
    left = 0
    longest = ""
    for right in range(len(s)):
        while s[right] in seen:
            seen.remove(s[left])
            left += 1
        seen.add(s[right])
        if right - left + 1 > len(longest):
            longest = s[left:right + 1]
    return longest
```
ornith-1.5:35b-a3b-mq4r battery turn 1
During the day, the sky appears blue because sunlight interacts with atmospheric gases through a process called Rayleigh scattering, in which shorter blue wavelengths scatter much more efficiently than longer wavelengths, filling the sky with that characteristic blue hue. At sunset, sunlight travels through a much thicker portion of the atmosphere to reach your eyes, so nearly all of the scattered blue light is deflected away before arriving, leaving predominantly the longer red and orange wavelengths to dominate the visible sky.
ornith-1.5:35b-a3b-mq4r battery turn 2
The capital of France is Paris, and the Seine River runs through it.
ornith-1.5:35b-a3b-mq4r battery turn 3
{"name": "Alice", "age": 34, "city": "Lisbon"}
ornith-1.5:35b-a3b-mq4r battery turn 4
17 + 26 = 43

Answer: 43

chain — exit 0 seconds 21.4 status pass

mode idx genre finish ctx cached gen ans_words prefill_tok_s decode_tok_s attractor empty runaway recall_ok
chain 0 length 44 0 256 113 712.6 66.3 False False True True
chain 1 stop 353 0 117 47 175.6 59.1 False False False True
chain 2 stop 496 470 91 69 420.3 48.8 False False False True
chain 3 stop 621 587 119 89 533.6 43.0 False False False True
chain 4 stop 772 740 82 54 441.7 53.0 False False False True
ornith-1.5:35b-a3b-mq4r chain turn 0
```python
def merge_sorted(a, b):
    """Merge two sorted lists into a single sorted list (ascending).

    Uses the classic merge step from merge sort: two pointers advance
    through `a` and `b`, always appending the smaller current element.
    Runs in O(len(a) + len(b)) time and does not call sorted().
    """
    merged = []
    i = j = 0

    while i < len(a) and j < len(b):
        if a[i] <= b[j]:
            merged.append(a[i])
            i += 1
        else:
            merged.append(b[j])
            j += 1

    # Append any remaining elements from either list.
    merged.extend(a[i:])
    merged.extend(b[j:])

    return merged
```

**Example:**

```python
print(merge_sorted([1, 3, 5], [2, 4, 6]))  # [1, 2, 3, 4, 5, 6]
print(merge_sorted([1, 2, 2], [2, 3, 4]))  # [1, 2
ornith-1.5:35b-a3b-mq4r chain turn 1
# Total Distance Calculation

**Leg 1:** 60 mph for 2.5 hours

$$60 \times 2.5 = 150 \text{ miles}$$

**Leg 2:** 40 mph for 1.5 hours

$$40 \times 1.5 = 60 \text{ miles}$$

**Total:**

$$150 + 60 = \boxed{210 \text{ miles}}$$

The train traveled **210 miles** in total.
ornith-1.5:35b-a3b-mq4r chain turn 2
The seasons are caused by the 23.5-degree tilt of Earth's axis relative to its orbital plane around the Sun. As Earth orbits the Sun throughout the year, different hemispheres lean toward or away from the Sun, changing both the intensity and duration of sunlight. When a hemisphere tilts toward the Sun, it experiences summer with warmer, more direct rays; when it tilts away, winter arrives with cooler, slanted sunlight.
ornith-1.5:35b-a3b-mq4r chain turn 3
After forty years of patting the salty cliffs at dawn, Maren had never once wondered what the sea might offer beyond her usual driftwood and gulls. That morning, however, a tangle of kelp cradled something pale and smooth, half-buried where the tide had dragged it high. She knelt and brushed away the salt, revealing the worn face of a compass she could have sworn her father had lost over the decades. Its needle, dead for a long time, now turned—slowly, steadily—to point not north, but back toward the water.
ornith-1.5:35b-a3b-mq4r chain turn 4
1. Keep functions small and focused on a single, well-defined responsibility.
2. Use consistent, descriptive names for variables, functions, and modules.
3. Write comments that explain *why*, not *what*, when intent isn't obvious.
4. Follow a consistent style and formatting standard across the codebase.
5. Avoid premature optimization; favor clarity and simplicity over cleverness.

lfm2.5:1.2b

source: bucket sha256_ok: ✅ size_ok: ✅ status: pass reason:

battery — exit 0 seconds 8.1 status pass

mode idx genre finish ctx cached gen ans_words prefill_tok_s decode_tok_s attractor empty runaway recall_ok
battery 0 stop 0 0 121 58 None 207.9 False False False True
battery 1 stop 0 0 68 62 None 209.23 False False False True
battery 2 stop 0 0 18 16 None 209.3 False False False True
battery 3 stop 0 0 26 8 None 208.0 False False False True
battery 4 stop 0 0 28 18 None 208.96 False False False True
lfm2.5:1.2b battery turn 0
```python
def longest_unique_substring(s):
    seen = {}
    start = 0
    max_length = 0
    max_sub = ""

    for i, char in enumerate(s):
        if char in seen and start <= seen[char]:
            start = seen[char] + 1
        else:
            if i - start + 1 > max_length:
                max_length = i - start + 1
                max_sub = s[start:i+1]

        seen[char] = i

    return max_sub
```
lfm2.5:1.2b battery turn 1
The sky appears blue during the day because of the scattering of sunlight by particles in the atmosphere, with shorter wavelengths like blue being spread out in all directions. At sunset, the sun is low on the horizon, so its light travels through more atmosphere, scattering the red and orange wavelengths out and leaving the longer blue light to reach our eyes.
lfm2.5:1.2b battery turn 2
The capital of France is Paris, and the river that runs through it is the Seine.
lfm2.5:1.2b battery turn 3
{
  "name": "Alice",
  "age": 34,
  "city": "Lisbon"
}
lfm2.5:1.2b battery turn 4
Let's add the two numbers together:  
17 + 26 = 43  
So the final answer is:  
Answer: 43

chain — exit 0 seconds 14.6 status pass

mode idx genre finish ctx cached gen ans_words prefill_tok_s decode_tok_s attractor empty runaway recall_ok
chain 0 stop 0 0 231 123 None 203.7 False False False True
chain 1 stop 0 0 182 110 None 188.8 False False False True
chain 2 stop 0 0 65 58 None 181.06 False False False True
chain 3 stop 0 0 77 59 None 175.4 False False False True
chain 4 stop 0 0 71 51 None 171.5 False False False True
lfm2.5:1.2b chain turn 0
```python
def merge_sorted(a, b):
    """
    Merge two already-sorted lists into a single sorted list.
    
    Parameters:
    a (list): First sorted list.
    b (list): Second sorted list.
    
    Returns:
    list: A new sorted list containing elements from both a and b.
    """
    i, j = 0, 0
    merged = []
    # Traverse both lists and append smaller element to merged
    while i < len(a) and j < len(b):
        if a[i] < b[j]:
            merged.append(a[i])
            i += 1
        else:
            merged.append(b[j])
            j += 1
    # Append remaining elements from list a
    while i < len(a):
        merged.append(a[i])
        i += 1
    # Append remaining elements from list b
    while j < len(b):
        merged.append(b[j])
        j += 1
    
    return merged
```

This function efficiently merges two sorted lists without using the `sorted()` function.
lfm2.5:1.2b chain turn 1
To solve this problem, let's break it down step by step:

**Step 1: Calculate distance traveled at each speed**  
We are given two scenarios:

1. First, the train goes at **60 mph for 2.5 hours**  
   Distance = speed × time  
   Distance�� = 60 × 2.5 = 150 miles

2. Then, it goes at **40 mph for 1.5 hours**  
   Distance�� = 40 × 1.5 = 60 miles

**Step 2: Add both distances together**  
Total distance = Distance�� + Distance��  
Total distance = 150 miles + 60 miles = 210 miles

**Final Answer:**  
The train traveled a total of **210 miles**.

Note: The calculation uses the formula: distance = speed × time.
lfm2.5:1.2b chain turn 2
The seasons on Earth are caused by the tilt of the planet's axis relative to its orbit around the sun, which affects the amount of sunlight different regions receive. As the Earth orbits the sun, various parts of the planet experience different seasonal changes throughout the year. This axial tilt creates the cyclical pattern of seasons we observe.
lfm2.5:1.2b chain turn 3
One stormy evening, the lighthouse keeper noticed a delicate, shiny object washed up on the rocky shore. Curious, he carefully examined the item and discovered it was a tiny, ancient map. Excited, he wondered what stories it might tell him about the coastline. With each passing day, the keeper's curiosity grew, leading him further down the path of discovery.
lfm2.5:1.2b chain turn 4
1. Write clean and readable code with consistent formatting.  
2. Use comments to explain complex logic or unclear sections.  
3. Organize your code into modular functions with clear responsibilities.  
4. Regularly update and test your code to maintain reliability.  
5. Document your codebase to make it easier for others to understand.

qwen3.8:27b-mq4-xt

source: bucket sha256_ok: ✅ size_ok: ✅ status: pass reason:

battery — exit 0 seconds 11.5 status pass

mode idx genre finish ctx cached gen ans_words prefill_tok_s decode_tok_s attractor empty runaway recall_ok
battery 0 stop 34 0 132 62 63.2 45.3 False False False True
battery 1 stop 30 0 96 81 346.1 63.2 False False False True
battery 2 stop 31 0 16 13 370.0 62.9 False False False True
battery 3 stop 47 0 31 8 446.7 58.4 False False False True
battery 4 stop 47 0 17 7 447.6 67.5 False False False True
qwen3.8:27b-mq4-xt battery turn 0
```python
def longest_substring_without_repeating(s: str) -> str:
    start = 0
    max_start = 0
    max_length = 0
    last_index = {}

    for end, char in enumerate(s):
        if char in last_index and last_index[char] >= start:
            start = last_index[char] + 1
        last_index[char] = end
        if end - start + 1 > max_length:
            max_length = end - start + 1
            max_start = start

    return s[max_start:max_start + max_length]
```
qwen3.8:27b-mq4-xt battery turn 1
Sunlight is scattered by gas molecules in the atmosphere through a process known as Rayleigh scattering, which affects shorter wavelengths more intensely than longer ones. During the day, this preferential scattering sends a significant amount of blue light in all directions, making the sky appear blue when we look up. At sunset, sunlight travels through a thicker layer of the atmosphere, scattering away most of the blue light and allowing the longer, less-scattered red and orange wavelengths to dominate the view.
qwen3.8:27b-mq4-xt battery turn 2
The capital of France is Paris, and the River Seine runs through it.
qwen3.8:27b-mq4-xt battery turn 3
{
  "name": "Alice",
  "age": 34,
  "city": "Lisbon"
}
qwen3.8:27b-mq4-xt battery turn 4
17 + 26 = 43

Answer: 43

chain — exit 0 seconds 18.4 status pass

mode idx genre finish ctx cached gen ans_words prefill_tok_s decode_tok_s attractor empty runaway recall_ok
chain 0 stop 44 0 120 51 371.0 63.9 False False False True
chain 1 stop 220 164 212 124 432.6 50.4 False False False True
chain 2 stop 458 432 78 64 357.4 36.8 False False False True
chain 3 stop 570 536 105 86 363.0 32.6 False False False True
chain 4 stop 707 675 73 48 415.7 54.2 False False False True
qwen3.8:27b-mq4-xt chain turn 0
```python
def merge_sorted(a, b):
    """Merge two sorted lists into a single sorted list."""
    result = []
    i, j = 0, 0
    while i < len(a) and j < len(b):
        if a[i] <= b[j]:
            result.append(a[i])
            i += 1
        else:
            result.append(b[j])
            j += 1
    # Append remaining elements
    result.extend(a[i:])
    result.extend(b[j:])
    return result
```
qwen3.8:27b-mq4-xt chain turn 1
To find the total distance traveled, we need to calculate the distance for each segment of the trip and then add them together.

**Step 1: Calculate the distance for the first segment**
*   Speed = 60 mph
*   Time = 2.5 hours
*   Distance = Speed × Time
*   Distance₁ = $60 \times 2.5 = 150$ miles

**Step 2: Calculate the distance for the second segment**
*   Speed = 40 mph
*   Time = 1.5 hours
*   Distance = Speed × Time
*   Distance₂ = $40 \times 1.5 = 60$ miles

**Step 3: Calculate the total distance**
*   Total Distance = Distance₁ + Distance₂
*   Total Distance = $150 + 60 = 210$ miles

**Final Answer:**
The train traveled a total of **210 miles**.
qwen3.8:27b-mq4-xt chain turn 2
The primary cause of seasons is the 23.5-degree tilt of Earth's rotational axis relative to its orbital plane around the Sun. As Earth orbits the Sun, this tilt causes different hemispheres to receive varying amounts of direct sunlight and daylight hours throughout the year. This variation in solar intensity and duration leads to the cyclical change in temperature and weather patterns known as seasons.
qwen3.8:27b-mq4-xt chain turn 3
Elias wiped the salt spray from his goggles and stepped out onto the jagged rocks just as the tide receded. There, nestled in a tangle of kelp, was not a fish or a bottle, but a pristine, silver telescope with no brand marking. He lifted the heavy instrument, feeling the cold metal vibrate against his palm as if it were still listening to the sea. As he brought it to his eye, the dark horizon blurred into a shimmering, impossible galaxy he had never seen before.
qwen3.8:27b-mq4-xt chain turn 4
1. Write descriptive, self-explanatory variable and function names.
2. Keep functions small and focused on a single responsibility.
3. Avoid duplication by extracting shared logic into reusable modules.
4. Include meaningful docstrings and comments for non-obvious logic.
5. Consistently apply a style guide and use automated linting tools.

kernel

status: pass

report pass: True

lane hipx (gfx1100)

hw-gate evidence

field value
base f025b2ec9c842833109b3c858937997b9cb36cc7
head b106a4aac3d8b9f6aadfaf9a3693c1c2601593ca
buckets
host gfx gfx1100
host rocm 7.15.26333-0000000
device 0
runner hipx
daemon_md5 17ba7dfab49aa1c1110d3f412f072648
hipfire_md5 cac7ede8ee13de92e25338bf2ca40a55
build_seconds 45.118409156799316
verdict pass
logs_dir hw-gate-logs

fixtures

no fixtures

kernel

not run

@hipfire-sol

hipfire-sol Bot commented Sep 4, 2026

Copy link
Copy Markdown

hw-gate sol verdict

{
  "claim_verdict": "not-exercised",
  "confidence": 0.98,
  "coverage": {
    "gaps": [
      "The hardware report exercises kernel, load, and serve behavior, not the changed markdown decision parser or staging-merge authorization path.",
      "The reported hw-run result is failure.",
      "No adversarial tests cover quoted, negated, or conflicting verdict phrases."
    ],
    "surfaces_evidenced": [],
    "surfaces_touched": [
      "policy"
    ]
  },
  "decision": "block",
  "eyeball": [
    "Review the unanchored fallback patterns at scripts/hw-gate/review.py:447-450; quoted or explanatory prior verdicts can become authoritative.",
    "The supplied model outputs are generally coherent, but they are unrelated to the policy-parser claim."
  ],
  "phase": "verdict",
  "rationale": "Block because the diff introduces a demonstrated merge-authorization ambiguity: an unanchored quoted or negated `decision: merge-staging` phrase is accepted as the current verdict. The supplied hardware evidence passes unrelated runtime routes but does not exercise this parser, and the overall hw-run result is failure. Narrow the fallback to an unambiguous verdict heading or standalone line, reject conflicting matches, and add positive plus quoted/negated/conflicting end-to-end tests.",
  "regressions": [
    {
      "beta_behavior": "The fallback accepts a decision phrase anywhere within the first 2,000 characters. Text such as \u201cPrevious **decision:** merge-staging was wrong\u201d matches the second pattern and can authorize a staging merge despite explicitly rejecting that verdict.",
      "evidence": "Diff at scripts/hw-gate/review.py:449 uses an unanchored search for **decision:** followed by merge-staging/hold/block; no negative, quotation, conflict, or unique-verdict checks are present.",
      "file": "scripts/hw-gate/review.py",
      "line": 449,
      "master_behavior": "Non-JSON decide output is rejected and held for human review.",
      "severity": "high"
    }
  ]
}

Floor: hard=['hw_run_result=failure', 'policy_paths: scripts/hw-gate/review.py,scripts/hw-gate/tests/test_review.py'] soft=["coverage_gaps: ['The hardware report exercises kernel, load, and serve behavior, not the changed markdown decision parser or staging-merge authorization path.', 'The reported hw-run result is failure.', 'No adversarial tests cover quoted, negated, or conflicting verdict phrases.']"] model_decision=block final=block

@hipfire-sol hipfire-sol Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hw-gate sol verdict block: Block because the diff introduces a demonstrated merge-authorization ambiguity: an unanchored quoted or negated decision: merge-staging phrase is accepted as the current verdict. The supplied hardware evidence passes unrelated runtime routes but does not exercise this parser, and the overall hw-run result is failure. Narrow the fallback to an unambiguous verdict heading or standalone line, reject conflicting matches, and add positive plus quoted/negated/conflicting end-to-end tests.

@hipfire-fable

hipfire-fable Bot commented Sep 4, 2026

Copy link
Copy Markdown

announcement: Blocking, and the block is on the code rather than the floor: the headline pattern is sound and does merge the #691 shape end-to-end, but the two unanchored decision: patterns take the first verdict word in the report, and a report that says block while quoting a prior merge-staging merges to staging on this diff — I ran that exact text through the suite's own _run_decide harness ($HW_GATE_EVIDENCE/e2e_wrong_direction_merge.json: PR merges with an approving review, base holds). A report with no verdict that mentions the daemon's admission decision posts block, and a report quoting Sol's block before its own merge-staging posts block. Keep the headline pattern, drop or line-anchor the other two with a single-distinct-verdict rule, prefer a toolless re-ask for the JSON over regex (the investigate path has no retry today), stop duplicating text[:6000] into both comment fields, and add the quoted/negated/conflicting/no-verdict cases as tests asserting hold. One thing for you rather than the PR: the hardware evidence comment on this PR is another commit's (hiptrx lane head 31dd5e3) — lanes/ in the reused runner workspace is not cleared and merge_evidence.py never checks a lane's head against the run's; details in $HW_GATE_EVIDENCE/stale-lane-evidence.txt.

investigation:

| question | route | result | evidence |

|---|---|---|---|

| Does the 104/104 claim hold, and does the new test pass on the PR head (base for comparison)? | cd pr && python3 -m pytest scripts/hw-gate/tests -q -p no:cacheprovider ; same in base ; pr: -k test_decide_markdown_headline_is_a_decision -vv | PR: 104 passed in 4.56s. Base: 103 passed. New test: 1 passed. The claim is true, and the suite has no negative case for a verdict word that is not the report's verdict. | pytest-pr-markdown-test.txt |

| What does _markdown_decision (review.py:438-464) accept when the text is a realistic Fable report rather than the one fixture in the test? | python3 $HW_GATE_EVIDENCE/markdown_decision_probe.py — 17 named cases run through the PR's _markdown_decision, each re-matched against the three patterns to record which fired | Headline cases (pattern 0, anchored ^#{1,4}\s+hw-gate\s+decide) are correct in every ordering, including a headline placed after a quoted Sol decision. Patterns 1 and 2 are wrong in 5 of 17 cases: The previous run's **decision:** merge-staging was wrong; ... My decision: block. -> merge-staging (pattern 1); Sol's verdict was **decision:** block, ... **decision:** merge-staging -> block; Sol's decision: block.\n...decision: merge-staging -> block; The daemon's admission decision: block. That is master behavior... with no verdict at all -> block (pattern 2); **decision:** hold\n(edit after the A/B: **decision:** merge-staging) -> hold. Truncated JSON and a headline past the 2000-char window return None (documented limits, not defects). | markdown_decision_probe.json |

| Does a markdown verdict merge end-to-end through _run_decide, including the #691 shape (Sol needs-human), and what does base do with the same text? | python3 $HW_GATE_EVIDENCE/e2e_markdown_probe.py — the suite's own _run_decide harness (fake gh/omp) on the PR checkout for 7 scenarios and on base for 4 | PR: headline + Sol greenlight -> merge-staging, merged. Headline + Sol needs-human (the #691 case) -> merge-staging, merged, override={of: needs-human, why: } — the fix does cover the run it was written for. Headline + policy_paths floor -> hold, no merge (floor precedence intact). Prose with no verdict word -> hold, fable_error set, fable_raw.assistant_text_tail present. Quoted **decision:** block followed by the real **decision:** merge-staging -> decision_final=block, a request-changes review posted. A 9000-char report copies text[:6000] into BOTH announcement and rationale (fable-decision comment 12,106 bytes). Base: all four -> hold with no JSON object in assistant text. | *.decision.json |

| Can a report whose verdict is block be merged to staging by the fallback? (Sol's regression, review.py:449-453) | python3 $HW_GATE_EVIDENCE/e2e_wrong_direction_merge.py — same harness, Sol greenlight, fable text = "The previous run's decision: merge-staging was wrong; the regression is real. ... turn 3 emits a single-token attractor ... My decision: block." | PR: decision_final=merge-staging, markdown_fallback=true, one POST to /merges, merged.merge_sha set, an --approve review posted. Base: decision_final=hold, decision=null, fable_error carries the text. A report that says block and describes an attractor merges to staging on this diff. | wrong_direction_block_report_merges.decision.json |

| Why did the floor fire hw_run_result=failure when the merged evidence says pass on both lanes, and is the hardware evidence even this PR's? | stat/jq over the reused hiptrx workspace (hw-gate.json, lanes/, evidence/, prelim/*), git log for the recorded SHAs; then base/scripts/hw-gate/run.py --repo pr --buckets "" [--routes evidence/routes.json] --skip-build --device 3 with this PR's select (empty buckets, routes []) | The hiptrx lane in evidence/hw-gate.json is head 31dd5e3 / base e98b461 — a different PR's run (mtime 20:07:33, lanes/ copy 20:10:29); only the hipx lane (656 bytes, buckets [], no fixtures, head b106a4a) belongs to this PR (downloaded 20:12:05). merge_evidence.py takes base/head verbatim from the first lane (merge_evidence.py:89-95) and hard_floor never compares evidence.head to --head (review.py:94-151); sol-verdict and fable-decide download into lanes/ with continue-on-error and no rm -rf (hw-gate.yml:286-297, 374-385). So every battery, chain, and Redline number in the evidence comment on #711 is another commit's. run.py with this PR's actual selection exits 0 and writes verdict=pass with head b106a4a (binaries md5 17ba7dfa/cac7ede8, matching the hipx lane), so the hiptrx job failure was not the runner script on this input; the leg simply produced no artifact for this head. | runpy-empty-buckets-{routes,noroutes}.{json,md,log} |

unproven:

  • Whether the real fix(dflash): transactional draft ctors, emitter-error rollback, one ctx-cap predicate #691 report (run 33905366422) would have been parsed by pattern 0 alone: the assistant text of that run is not on this host; the PR's own fixture and the probe's headline cases say yes, so the narrowing below does not lose the motivating case.

  • Why the hiptrx hw-run leg for head b106a4a produced no artifact: the job log is not readable from the sandbox; run.py with this PR's exact selection exits 0 here, so it is an infra/concurrency failure of that leg, not the runner script.

rationale: The hard floor fired twice and neither reason is mine to lift: policy_paths (scripts/hw-gate/review.py, tests/test_review.py) is a hold by rule, and hw_run_result=failure is a block by rule. On the merits I also block, for a regression I demonstrated rather than inferred. _markdown_decision (review.py:438-464) tries three patterns; the first, anchored to the ## hw-gate decide headline, is correct in every case I threw at it, including a headline after a quoted Sol decision, and it merges the #691 shape end-to-end over a Sol needs-human with the report as override.why ($HW_GATE_EVIDENCE/e2e_markdown_probe.json S2). The second and third (review.py:452-453) search the first 2000 characters for any decision: <word> and return the first hit, and the first hit in a real report is routinely a quotation: $HW_GATE_EVIDENCE/e2e_wrong_direction_merge.json runs a report that says block and describes an attractor through the PR's own harness and gets decision_final=merge-staging, one /merges call and an approving review; base holds the same text with fable_error. The other direction is also live: a report quoting Sol's block before its own merge-staging posts block (e2e S3), and a sentence about the daemon's admission decision in a report with no verdict posts block (probe case prose_daemon_decision). Test evidence is 104/104 on the PR and 103/103 on base ($HW_GATE_EVIDENCE/pytest-*.txt), and none of those tests contains a verdict word that is not the verdict. What changes this to a merge: keep pattern 0; delete patterns 1-2 or replace them with a line-anchored standalone verdict (^\s*\**decision:?\**\s*[:=]?\s*?(merge-staging|hold|block)?\s*$, MULTILINE) that returns None when more than one distinct verdict word matches; in omp_review try the existing 'Return only the JSON object' retry before falling back (the investigate path has no retry at all — a second, toolless omp call that hands back the report and asks for the JSON is safer than any regex); make announcement the headline plus first paragraph rather than a second copy of text[:6000]; and add the probe's quoted/negated/conflicting/no-verdict cases as tests asserting hold. Separately and not this PR's fault: the hardware evidence posted on #711 is not this PR's. evidence/hw-gate.json's hiptrx lane is head 31dd5e3 / base e98b461 (a different PR, written 20:07:33 on this reused hiptrx workspace); only the hipx lane (buckets [], no fixtures, head b106a4a) is real. merge_evidence.py copies base/head from the first lane verbatim (merge_evidence.py:89-95), hard_floor never checks evidence.head against --head (review.py:94-151), and both sol-verdict and fable-decide download into lanes/ without clearing it (hw-gate.yml:286-297, 374-385) — the same class #705 fixed for fable-evidence. This time hw_run_result=failure caught it; a stale lane under a successful or cancelled leg would pass the floor. Forensics in $HW_GATE_EVIDENCE/stale-lane-evidence.txt. run.py with this PR's empty selection exits 0 and writes head b106a4a ($HW_GATE_EVIDENCE/runpy-empty-buckets.json), so re-running hw-run clears the failure reason; the policy hold and the parser fix remain.

hard floor: ['hw_run_result=failure', 'policy_paths: scripts/hw-gate/review.py,scripts/hw-gate/tests/test_review.py']

soft floor: ["coverage_gaps: ['The hardware report exercises kernel, load, and serve behavior, not the changed markdown decision parser or staging-merge authorization path.', 'The reported hw-run result is failure.', 'No adversarial tests cover quoted, negated, or conflicting verdict phrases.']"]

@hipfire-fable hipfire-fable Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocking, and the block is on the code rather than the floor: the headline pattern is sound and does merge the #691 shape end-to-end, but the two unanchored decision: patterns take the first verdict word in the report, and a report that says block while quoting a prior merge-staging merges to staging on this diff — I ran that exact text through the suite's own _run_decide harness ($HW_GATE_EVIDENCE/e2e_wrong_direction_merge.json: PR merges with an approving review, base holds). A report with no verdict that mentions the daemon's admission decision posts block, and a report quoting Sol's block before its own merge-staging posts block. Keep the headline pattern, drop or line-anchor the other two with a single-distinct-verdict rule, prefer a toolless re-ask for the JSON over regex (the investigate path has no retry today), stop duplicating text[:6000] into both comment fields, and add the quoted/negated/conflicting/no-verdict cases as tests asserting hold. One thing for you rather than the PR: the hardware evidence comment on this PR is another commit's (hiptrx lane head 31dd5e3) — lanes/ in the reused runner workspace is not cleared and merge_evidence.py never checks a lane's head against the run's; details in $HW_GATE_EVIDENCE/stale-lane-evidence.txt. The hard floor fired twice and neither reason is mine to lift: policy_paths (scripts/hw-gate/review.py, tests/test_review.py) is a hold by rule, and hw_run_result=failure is a block by rule. On the merits I also block, for a regression I demonstrated rather than inferred. _markdown_decision (review.py:438-464) tries three patterns; the first, anchored to the ## hw-gate decide headline, is correct in every case I threw at it, including a headline after a quoted Sol decision, and it merges the #691 shape end-to-end over a Sol needs-human with the report as override.why ($HW_GATE_EVIDENCE/e2e_markdown_probe.json S2). The second and third (review.py:452-453) search the first 2000 characters for any decision: <word> and return the first hit, and the first hit in a real report is routinely a quotation: $HW_GATE_EVIDENCE/e2e_wrong_direction_merge.json runs a report that says block and describes an attractor through the PR's own harness and gets decision_final=merge-staging, one /merges call and an approving review; base holds the same text with fable_error. The other direction is also live: a report quoting Sol's block before its own merge-staging posts block (e2e S3), and a sentence about the daemon's admission decision in a report with no verdict posts block (probe case prose_daemon_decision). Test evidence is 104/104 on the PR and 103/103 on base ($HW_GATE_EVIDENCE/pytest-*.txt), and none of those tests contains a verdict word that is not the verdict. What changes this to a merge: keep pattern 0; delete patterns 1-2 or replace them with a line-anchored standalone verdict (^\s*\**decision:?\**\s*[:=]?\s*?(merge-staging|hold|block)?\s*$, MULTILINE) that returns None when more than one distinct verdict word matches; in omp_review try the existing 'Return only the JSON object' retry before falling back (the investigate path has no retry at all — a second, toolless omp call that hands back the report and asks for the JSON is safer than any regex); make announcement the headline plus first paragraph rather than a second copy of text[:6000]; and add the probe's quoted/negated/conflicting/no-verdict cases as tests asserting hold. Separately and not this PR's fault: the hardware evidence posted on #711 is not this PR's. evidence/hw-gate.json's hiptrx lane is head 31dd5e3 / base e98b461 (a different PR, written 20:07:33 on this reused hiptrx workspace); only the hipx lane (buckets [], no fixtures, head b106a4a) is real. merge_evidence.py copies base/head from the first lane verbatim (merge_evidence.py:89-95), hard_floor never checks evidence.head against --head (review.py:94-151), and both sol-verdict and fable-decide download into lanes/ without clearing it (hw-gate.yml:286-297, 374-385) — the same class #705 fixed for fable-evidence. This time hw_run_result=failure caught it; a stale lane under a successful or cancelled leg would pass the floor. Forensics in $HW_GATE_EVIDENCE/stale-lane-evidence.txt. run.py with this PR's empty selection exits 0 and writes head b106a4a ($HW_GATE_EVIDENCE/runpy-empty-buckets.json), so re-running hw-run clears the failure reason; the policy hold and the parser fix remain.

@hipfire-fable hipfire-fable Bot added the hw-gate-blocked hw-gate reviewer blocked on evidence label Sep 4, 2026
ghazni101 pushed a commit to ghazni101/hipfire that referenced this pull request Sep 5, 2026
…e-gate a merged PR

Two defects observed on 2026-09-04 while driving the ladder, both of which
cost live rungs their hardware lane:

1. The decide phase takes `flock --exclusive` on all five GPUs around the
   whole `review.py --phase decide` process, for its full 45-minute budget.
   warpfront#711's seat had no provider credits and could never produce a verdict, yet
   it held that lock for ~20 minutes while warpfront#687 and warpfront#688 sat queued with
   'hardware (gfx1201)' unable to start. A seat that cannot answer must not
   own the hardware.

   Fixed with a preflight: one 90 s toolless probe of the decide model before
   the lock is taken. If it replies, the locked phase runs exactly as before.
   If it does not, the locked step is skipped entirely and a new unlocked step
   records the hold via `--decider-unavailable REASON`, which short-circuits
   the model call in review.py and lets the floors, comment, and labels run as
   usual. The GPUs are never claimed.

2. `pull_request_target` fires on `labeled` even for a merged PR, so label
   churn re-ran the entire gate on warpfront#711 five minutes after it merged (run
   33915818350) — taking the runner and the exclusive lock from live rungs.
   `select` now refuses any event whose PR is already merged; `workflow_dispatch`
   is unaffected, so a manual re-gate still works.

Test: `test_decide_unavailable_seat_holds_without_calling_the_model` asserts
hold, the reason recorded, no merge, and no model invocation at all (the fake
never opens its log). 105/105 hw-gate tests pass.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

hw-gate-blocked hw-gate reviewer blocked on evidence

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant