Separate the chunk diagnostics and harden the separator input

[arash77]

Follow-up to #2, from a second review round. Two of these produce silently wrong output.

A start index can be wrong without being negative

Once the splitter returns one negative index, the positions it derives from that one stay positive but point at an earlier part of the input, so they looked correct and were written to the JSON as such. Reproducible with a repetitive input, recursive splitter, token length, chunk size 10, overlap 3:

before:  [0, 67, 131, 186, null, 67, 131, 186, null, 67, 131]
after:   [0, 67, 131, 186, null, null, null, null, null, null, null]

Chunks 6 to 8 claimed the positions of chunks 2 to 4. The position is now checked for order and for content, so all of them are reported as null and named in the warning.

Chunks that the splitter altered were reported as a wrong position

Splitting between tokens cuts the text at token boundaries, which can fall inside a character that is encoded in several bytes and replaces it with the Unicode replacement character. With the default gpt2 encoding, chunk size 200 and overlap 50, a Chinese input gives 6 corrupted chunks out of 9. Those chunks were reported as an invalid start index, which points the user at a known upstream issue instead of at the corrupted text. They now get their own warning naming the cause and suggesting a character based splitter.

A custom separator was silently replaced by an X

Galaxy's default sanitizer only keeps letters, digits and -=_.()/+*^,:?!. Anything else, including | and every non-ASCII character, was replaced by a literal X. So a user splitting on | got the text split at every letter X in the document. The sanitizer now keeps | ; ~ % & $ @ < > " [ ] { }, and a validator rejects anything that would still be changed, pointing at the escape syntax. The apostrophe stays rejected on purpose, since the separator is passed inside single quotes.

Smaller

• Write the JSON output as UTF-8 as well. It was the one output still using the system locale encoding.
• special_token_error() returns the error to raise, so the special token handling has one contract instead of an unreachable re-raise at both call sites.
• New test for a separator that the sanitizer used to mangle.

planemo lint clean, planemo test 21/21.