web_fetch: panic in strip_html on pages with multi-byte UTF-8 characters

[jamesbrink]

Bug

strip_html() in src/tools/web_fetch.rs panics when processing HTML pages that contain multi-byte UTF-8 characters (e.g., ñ, é, CJK characters, emoji).

Panic message

thread 'tokio-runtime-worker' panicked at src/tools/web_fetch.rs:138:44:
byte index 206384 is not a char boundary; it is inside 'ñ' (bytes 206383..206385)

Root cause

The function iterates by char index (i counts characters via chars.collect() on line 132), but then uses i as a byte index when slicing the &str directly:

let chars: Vec<char> = html.chars().collect();   // line 132
let lower_chars: Vec<char> = lower.chars().collect(); // line 133
let len = chars.len();  // len = number of chars, NOT bytes
// ...
// BUG: `i` is a char index, but str[i..i+7] is byte-indexed in Rust
if !in_tag && i + 7 < len && &lower[i..i + 7] == "<script" {  // line 138

In Rust, str slice indexing (s[a..b]) operates on byte offsets, not character offsets. When the HTML contains multi-byte UTF-8 characters, the char index i drifts ahead of the actual byte position, and the slice eventually lands in the middle of a multi-byte character — causing a panic.

Affected lines

All of these use the same incorrect char-index-as-byte-index pattern:

• Line 138: &lower[i..i + 7] — <script check
• Line 144: &lower[i..i + 9] — </script> check
• Line 149: &lower[i..i + 6] — <style check
• Line 155: &lower[i..i + 8] — </style> check

Reproduction

Fetch any page containing non-ASCII text. The CNN homepage reliably triggers this because it contains Spanish-language content with ñ.

Suggested fix

The lower_chars vec is already computed on line 133 but is only used later (line 171) for block-element detection. The tag comparisons should use lower_chars slices instead of lower byte slices:

// Instead of:
&lower[i..i + 7] == "<script"

// Use char-based comparison:
lower_chars[i..i + 7].iter().collect::<String>() == "<script"

Alternatively, track a separate byte offset alongside the char index so that str slicing remains valid. Or replace the hand-rolled parser with a proper HTML-aware approach that handles UTF-8 correctly.