Skip to content

Fix Korean (and other rewriting IME) input from the on-screen keyboard - #733

Open
ginlimeb-ship-it wants to merge 1 commit into
moonlight-stream:masterfrom
ginlimeb-ship-it:korean-onscreen-keyboard
Open

ginlimeb-ship-it wants to merge 1 commit into
moonlight-stream:masterfrom
ginlimeb-ship-it:korean-onscreen-keyboard

Conversation

@ginlimeb-ship-it

Copy link
Copy Markdown

Fixes #731

Problem

onKeyboardPressed: sends each newly typed character and then resets the text field to the "0" sentinel. The iOS Korean keyboard builds a syllable by rewriting the character it just typed (ㅎ → 하 → 한). By the time it rewrites, that character has already been sent and removed from the field. As a result only the first jamo reaches the host: typing 되나 sends just ㄷ, and 안녕하세요 shows ㅇㄴ. This was confirmed with Sunshine's debug log; details are in #731.

Change

  • Send a diff instead of resetting after every change. The field keeps its contents while typing. On each editing change, the text is compared with what was already sent, character by character (composed character sequences, so a Hangul syllable or an emoji counts as one). Backspaces are sent for the removed characters, then the new text. For plain appended typing this sends exactly what was sent before.
  • Marked text. For Japanese and Chinese, only the committed text before markedTextRange is sent; the rest waits until it's committed, because marked text can contain Latin letters (romaji, pinyin) that shouldn't reach the host as key presses. Marked Hangul is real text that only gets replaced, so it is sent right away and the diff corrects it. Otherwise the syllable being typed wouldn't show on the host until the next one started.
  • The field resets to the sentinel only at safe points: whitespace, punctuation, Return, an empty field (Backspace), or 256+ characters. A reset never happens while text is marked.
  • Holding Backspace. iOS can delete a whole word at once, sentinel included. When the field becomes empty, the text already sent is deleted on the host (at least one Backspace, as before).
  • Return commits first. textFieldShouldReturn: commits any marked text and sends it before Return.
  • Ordering. Keyboard-field events (Backspace, text, Return) now go through a serial queue. Before, each key went through dispatch_get_global_queue, which is concurrent, so a Backspace could reach the host after the text meant to replace it. sendLowLevelEvent: is split so this path can send synchronously on that queue. Other callers are unchanged.

Unchanged: hardware keyboard handling, the keyboard toolbar keys, ASCII text being sent as key events (UTF-8 is used only for non-ASCII), and Backspace on an empty field.

The diff logic was also checked with a small simulation against three possible behaviors of the iOS Korean keyboard: plain replacement, a marked syllable, and a plain first jamo that then becomes marked. All the cases below passed in each, plus word-delete and Japanese-style marked input.

Testing

Tested on an iPad mini (6th generation) and an iPhone 15 Pro Max (both on iPadOS/iOS 27.2) with the built-in Korean – Standard (2-set) keyboard, streaming to Sunshine 2026.914 on Windows 11, typing into Notepad.

Keys Expected iPad mini iPhone
e h l s k 되나 ✅
d k s s u d g k t p d y 안녕하세요 ✅ ✅
q k s r k d n j 반가워 ✅
r k s k 가나 ✅
d k s g d k 않아 ✅ ✅
d l f r j 일거 ✅
z z z ㅋㅋㅋ ✅ ✅
g k s, Delete ×3 하 → ㅎ → (empty) ✅
안녕하세요, Delete 안녕하세ㅇ (same as Notes) ✅ ✅
Hold Delete after a word word removed on the host too ✅
안녕? 반가워! punctuation kept ✅
감사합니다 수고하세요, typed fast no reordering ✅
ㅋ, then wait 3 s shows up right away ✅
안녕 + Return 안녕 then newline ✅
Close and reopen the keyboard, keep typing nothing re-sent ✅
Backspace on an empty field one Backspace on the host ✅
Emoji 😀, Delete inserted, removed with one Delete ✅
Numbers 123 123 ✅
Toolbar Ctrl + a select all ✅
Japanese Romaji: nihon → 日本 only 日本 reaches the host ✅
Chinese Pinyin: nihao → 你好 only 你好 reaches the host ✅

For comparison, with Moonlight 9.0.2 from the App Store on the same iPad, typing 안녕 hello 안녕 반가워 produced ㅇㄴ ㅗ디ㅣㅐ ㅇㄴㅂㄱ on the host (the Latin part comes from the host being in Hangul mode, which this PR doesn't change).

Deleting right after typing removes one jamo at a time, also across earlier syllables. That's what the iOS keyboard itself does (same in Notes), and the host just mirrors it.

Known limitation: when a syllable changes (아 → 안) the host briefly shows the old syllable removed before the new one appears, since the only way to change already-sent text is Backspace followed by new text. Removing the short pause between the two made it worse: Backspace goes over the keyboard channel and text over the UTF-8 channel, which aren't ordered against each other, so the text could arrive first.

🤖 Generated with Claude Code

The on-screen keyboard handler sent each newly typed character and then
reset the text field to the sentinel. IMEs that build a character by
rewriting the one before it, like the iOS Korean keyboard (ㅎ -> 하 -> 한),
lost everything after the first jamo because the character they wanted to
rewrite was already gone.

Instead, keep the text field contents while typing and send a diff against
what was already sent: backspaces for removed characters, then the new
text. Marked text from other IMEs (Japanese, Chinese) is held back until
it's committed, since it can contain Latin letters. Marked Hangul is sent
right away and fixed up by the diff as it changes. The field is reset to
the sentinel only at whitespace, punctuation, Return, an empty field, or
when the text gets long.

Keyboard text events are now sent in order on a serial queue. Before, each
key went through a concurrent global queue, so a backspace could be sent
after the text meant to replace it. Return commits any
marked text and sends it before the Return key.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Korean (Hangul) input from the on-screen keyboard only sends the first consonant

1 participant