Add chat history and document attachments to the Mac app - #15
Open
dmitrymalakhov wants to merge 5 commits into
Open
Add chat history and document attachments to the Mac app#15dmitrymalakhov wants to merge 5 commits into
dmitrymalakhov wants to merge 5 commits into
Conversation
Add multi-turn chat state, context-window management, document text extraction, attachment handling, app presentation updates, and comprehensive test coverage.
dmitrymalakhov
marked this pull request as ready for review
July 28, 2026 20:21
added 3 commits
July 29, 2026 16:46
NeelM0906
added a commit
to NeelM0906/Mference
that referenced
this pull request
Aug 1, 2026
Replace the single prompt string in AppGenerationRequest with an ordered [AppGenerationMessage] list and extend the decode-service request the same way, keeping the old prompt-based initializers and a computed `prompt` accessor for existing callers. AppGenerationContextWindow fits a conversation into the available context by binary-searching user-turn suffixes and measuring each candidate with the model tokenizer. Rendering stays behind MFTokenizer.applyChatTemplate, so the same code path produces Gemma turn markers or Qwen 3.6 ChatML depending on the installed checkpoint's tokenizer. AppPromptContext composes attachment text into a bounded prompt block and marks documents that were trimmed to fit. Ports drumih/turbo-fieldfare#15 by @dmitrymalakhov.
NeelM0906
added a commit
to NeelM0906/Mference
that referenced
this pull request
Aug 1, 2026
AppModel now owns a list of AppChats instead of a single prompt field. Each chat keeps its messages, draft, draft attachments, and a rolling context summary; the archive is written next to the model directory on a serialized background queue and flushed on termination. An unreadable archive is quarantined as mac-app-chats.recovery-*.json rather than deleted. Before a turn is committed to history the client prepares the request with the real tokenizer. Turns that no longer fit are replaced in model context by a locally generated rolling summary, surfaced as the new .compressing generation phase. The pending user turn is never silently dropped: if it alone overflows, the run fails and the text stays in the composer. AppModelLocation remembers an explicitly chosen model directory in UserDefaults while keeping Mference's per-family install-directory resolution, so the remembered path wins over the gemma4/qwen36 default without changing which family the descriptor selects. Ports drumih/turbo-fieldfare#15 by @dmitrymalakhov.
NeelM0906
added a commit
to NeelM0906/Mference
that referenced
this pull request
Aug 1, 2026
DocumentTextExtractor reads PDF via PDFKit and DOCX/PPTX/XLSX by streaming selected archive members through /usr/bin/unzip into a SAX parser. Extraction is bounded at every step — 32 MiB per archive member, 64 MiB of selected XML, 240k extracted characters, and an early break once a PDF exceeds twice that — so a large document cannot pull unbounded content into the app process. AppChromePresentation carries the sidebar toggle copy, header layout metrics, and model action labels; AppAppearance persists the System/Light/Dark choice. The transcript document controller renders prior turns before the streaming answer and rebuilds cleanly when the selected chat changes. Ports drumih/turbo-fieldfare#15 by @dmitrymalakhov.
NeelM0906
added a commit
to NeelM0906/Mference
that referenced
this pull request
Aug 1, 2026
Add a collapsible chat sidebar with rename/delete, a paperclip importer that extracts document text off the main actor, a Choose Model Folder… picker in the Model menu and install view, and System/Light/Dark appearance commands. The status header now hosts both sidebar toggles with symmetric padding, and the output pane renders the full chat transcript instead of a single turn. Document the chat sidebar, attachments, and history compression in the README and docs/RUNTIME_CONTROLS.md. Ports drumih/turbo-fieldfare#15 by @dmitrymalakhov.
he-be
added a commit
to he-be/turbo-fieldfare
that referenced
this pull request
Aug 21, 2026
本線 oQ4e-g64 の lm_head は 8-bit g64 で、既存の融合ヘッドは INT4 しか 読めなかった。INT4 版 (logit.metal) と同じ 2 段の形で INT8 を書いた: 語彙幅の logit をどこにも書かず、8 行ごとの argmax を summaries に落として畳む。 INT4 から写すときに変わるのは 2 か所だけで、どちらも静かに壊れる。 行の刻みが N/2 ではなく N であること、そして 4-bit の sym と違って INT8 の零点は本物のデータなので b·Σx を落とせないこと。両方とも負例にした。 - qwen_lm_head_greedy_int8_rows_chunk_raw / _reduce (qwen.metal) - QwenLMHeadChainInt8。vocab には 248,077 を渡す。末尾 243 行は未学習で、 マスクのコードは要らず引数だけで済むことを検査で確かめた - --qwen は 39 本 (drumih#15 で 10 本増、うち 4 本は負例)。 dequant_int8_gemv_simd を二人目の証人にして、参照との一致と chain との直接比較の 2 通りに使う 実物の語彙 (248,077 x 2048) で 1 トークン 4.008/4.039/4.047 ms、 540 MB を 133〜135 GB/s。LM head だけで decode の天井が約 248 tok/s。 帯域の実測が導出の前提 150 GB/s より 11% 低いので、01-MODEL §5-5 の 79 tok/s は 71 tok/s の側で読む。 docs/qwen35moe/19-LM-HEAD-INT8.md Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Motivation
The Mac app previously focused on isolated prompt-and-response runs and could not incorporate document content into a conversation.
This change adds persistent multi-turn conversations and bounded document ingestion while preserving the runtime’s context and memory constraints. It also improves the app layout, provides explicit appearance controls, and keeps the chat workflow compatible with the latest upstream runtime and prompt settings.
User impact
Users can:
Document text is extracted locally. The current user turn is never silently discarded when fitting conversation history into the model context.
Upstream integration
The branch has been merged with the current
upstream/main.Conflict resolution preserves both the PR functionality and newer upstream behavior, including:
Validation
Scripts/test.sh— 777 tests in 132 suites passed onfd5069cruby Scripts/check_markdown_links.rb— 22 Markdown files passedruby Scripts/check_app_version.rb— app version 0.4.3 matches the latest release