feat: compress ApplyData payloads with gzip and Base93 - #1015
Open
boomzero wants to merge 29 commits into
Open
Conversation
hendragon-bot
Bot
added
the
user-script
Contributor
Reviewer's GuideImplements a new ApplyData transport pipeline that replaces Base64/exception-based stderr encoding with bracket-safe Base93 plus gzip or NSC3 numeric compression, adds client-side decoders and UI controls for numeric mode and line preservation, auto-creates the ApplyData container, and bumps the script/package version to 3.6.3. Sequence diagram for updated ApplyData data fetch pipelinesequenceDiagram
actor User
participant Browser as XMOJ.user.js
participant Judge as xmoj.tech
participant Helper as Cpp_helper
User->>Browser: click GetDataButton
Browser->>Browser: Build Code (NumberStreamEnabled / GzipCode)
Browser->>Judge: fetch submit.php (Code)
Judge->>Helper: run helper program
Helper->>Helper: rd()
alt NumberStreamEnabled
Helper->>Helper: ns()
else Default gzip mode
Helper->>Helper: gz()
end
Helper->>Helper: b93()
Helper->>Judge: write [payload] to stderr
Helper->>Helper: abort()
Judge-->>Browser: reinfo.php HTML (stderr in #errtxt)
Browser->>Browser: ExtractData(ErrorData)
loop for each [payload]
Browser->>Browser: DecodePayload(payload)
alt NSC1/NSC3 header
Browser->>Browser: NumberStreamDecode(rawData)
else gzip data
Browser->>Browser: GzipDecode(rawData)
end
Browser->>Browser: render decoded data in ApplyDiv
end
File-Level Changes
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
Deploying xmoj-script-dev-channel with
|
| Latest commit: |
0d5fef7
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://0d2f4a42.xmoj-script-dev-channel.pages.dev |
| Branch Preview URL: | https://feat-apply-data-gzip-base91.xmoj-script-dev-channel.pages.dev |
langningchen
approved these changes
Aug 21, 2026
boomzero
changed the title
boomzero
marked this pull request as ready for review
August 21, 2026 14:08
Member
Author
|
Tested and confirmed to work |
![sourcery-ai[bot]](https://avatars.githubusercontent.com/in/48477?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Hey - I've found 2 issues, and left some high level feedback:
- Both C++ helpers inline a full Base93 encoder (
b93) andrdimplementation separately; consider factoring out the shared pieces so future changes to the transport format need to be updated in only one place. GzipDecoderelies onDecompressionStream('gzip'), which is not available in all browsers; adding a capability check and a clearer failure path (or a fallback decoder) would make the ApplyData feature degrade more gracefully.- The numeric/line-preservation toggle buttons update
aria-pressedstates in a somewhat inverted way (e.g. line-preservation button uses!PreserveLineBreaks); it may be worth double-checking the ARIA semantics so assistive technologies reflect the actual on/off state of each mode.
Prompt for AI Agents
Please address the comments from this code review:
## Overall Comments
- Both C++ helpers inline a full Base93 encoder (`b93`) and `rd` implementation separately; consider factoring out the shared pieces so future changes to the transport format need to be updated in only one place.
- `GzipDecode` relies on `DecompressionStream('gzip')`, which is not available in all browsers; adding a capability check and a clearer failure path (or a fallback decoder) would make the ApplyData feature degrade more gracefully.
- The numeric/line-preservation toggle buttons update `aria-pressed` states in a somewhat inverted way (e.g. line-preservation button uses `!PreserveLineBreaks`); it may be worth double-checking the ARIA semantics so assistive technologies reflect the actual on/off state of each mode.
## Individual Comments
### Comment 1
<location path="XMOJ.user.js" line_range="5571-5577" />
<code_context>
+ LineBreakButton.disabled = !NumberStreamEnabled;
+ LineBreakButton.classList.toggle("btn-outline-secondary", PreserveLineBreaks || !NumberStreamEnabled);
+ LineBreakButton.classList.toggle("btn-outline-warning", !PreserveLineBreaks && NumberStreamEnabled);
+ LineBreakButton.setAttribute("aria-pressed", String(!PreserveLineBreaks));
+ }
+ LineBreakButton.addEventListener("click", () => {
</code_context>
<issue_to_address>
**suggestion (bug_risk):** The aria-pressed state is inverted relative to the label and might confuse accessibility tooling.
The button text is "保留换行：是/否" and the visually active (ON) state is PreserveLineBreaks=true, but aria-pressed is bound to !PreserveLineBreaks. This causes assistive tech to report the opposite state from what’s shown. Please bind aria-pressed to PreserveLineBreaks (the ON state), or omit aria-pressed if this control should behave as a simple toggle without pressed semantics.
```suggestion
function UpdateLineBreakButton() {
LineBreakButton.innerText = "保留换行：" + (PreserveLineBreaks ? "是" : "否");
LineBreakButton.disabled = !NumberStreamEnabled;
LineBreakButton.classList.toggle("btn-outline-secondary", PreserveLineBreaks || !NumberStreamEnabled);
LineBreakButton.classList.toggle("btn-outline-warning", !PreserveLineBreaks && NumberStreamEnabled);
LineBreakButton.setAttribute("aria-pressed", String(PreserveLineBreaks));
}
```
</issue_to_address>
### Comment 2
<location path="XMOJ.user.js" line_range="5740" />
<code_context>
});
}
- document.getElementById("apply_data").addEventListener("click", () => {
+ ApplyDataElement.addEventListener("click", () => {
let ApplyElements = document.getElementsByClassName("data");
for (let i = 0; i < ApplyElements.length; i++) {
</code_context>
<issue_to_address>
**question (bug_risk):** Binding the click handler to the apply_data container instead of a dedicated control may change the interaction area and event behavior.
Previously the listener was bound to the specific `#apply_data` element, likely a dedicated control. Now it’s bound to `ApplyDataElement`, which is used as a generic container while buttons are appended to its parent. This makes any click within the container (including empty areas) toggle all `.data` elements, while clicks on the new buttons won’t trigger the handler. To preserve the original behavior, consider attaching the listener to a specific toggle button rather than the container.
</issue_to_address>Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.
Comment on lines
+5571
to
+5577
| function UpdateLineBreakButton() { | ||
| LineBreakButton.innerText = "保留换行：" + (PreserveLineBreaks ? "是" : "否"); | ||
| LineBreakButton.disabled = !NumberStreamEnabled; | ||
| LineBreakButton.classList.toggle("btn-outline-secondary", PreserveLineBreaks || !NumberStreamEnabled); | ||
| LineBreakButton.classList.toggle("btn-outline-warning", !PreserveLineBreaks && NumberStreamEnabled); | ||
| LineBreakButton.setAttribute("aria-pressed", String(!PreserveLineBreaks)); | ||
| } |
Contributor
There was a problem hiding this comment.
suggestion (bug_risk): The aria-pressed state is inverted relative to the label and might confuse accessibility tooling.
The button text is "保留换行：是/否" and the visually active (ON) state is PreserveLineBreaks=true, but aria-pressed is bound to !PreserveLineBreaks. This causes assistive tech to report the opposite state from what’s shown. Please bind aria-pressed to PreserveLineBreaks (the ON state), or omit aria-pressed if this control should behave as a simple toggle without pressed semantics.
Suggested change
| function UpdateLineBreakButton() { | |
| LineBreakButton.innerText = "保留换行：" + (PreserveLineBreaks ? "是" : "否"); | |
| LineBreakButton.disabled = !NumberStreamEnabled; | |
| LineBreakButton.classList.toggle("btn-outline-secondary", PreserveLineBreaks || !NumberStreamEnabled); | |
| LineBreakButton.classList.toggle("btn-outline-warning", !PreserveLineBreaks && NumberStreamEnabled); | |
| LineBreakButton.setAttribute("aria-pressed", String(!PreserveLineBreaks)); | |
| } | |
| function UpdateLineBreakButton() { | |
| LineBreakButton.innerText = "保留换行：" + (PreserveLineBreaks ? "是" : "否"); | |
| LineBreakButton.disabled = !NumberStreamEnabled; | |
| LineBreakButton.classList.toggle("btn-outline-secondary", PreserveLineBreaks || !NumberStreamEnabled); | |
| LineBreakButton.classList.toggle("btn-outline-warning", !PreserveLineBreaks && NumberStreamEnabled); | |
| LineBreakButton.setAttribute("aria-pressed", String(PreserveLineBreaks)); | |
| } |
| }); | ||
| } | ||
| document.getElementById("apply_data").addEventListener("click", () => { | ||
| ApplyDataElement.addEventListener("click", () => { |
Contributor
There was a problem hiding this comment.
question (bug_risk): Binding the click handler to the apply_data container instead of a dedicated control may change the interaction area and event behavior.
Previously the listener was bound to the specific #apply_data element, likely a dedicated control. Now it’s bound to ApplyDataElement, which is used as a generic container while buttons are appended to its parent. This makes any click within the container (including empty areas) toggle all .data elements, while clicks on the new buttons won’t trigger the handler. To preserve the original behavior, consider attaching the listener to a specific toggle button rather than the container.
![cubic-dev-ai[bot]](https://avatars.githubusercontent.com/in/1082092?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
2 issues found and verified against the latest diff
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="XMOJ.user.js">
<violation number="1" location="XMOJ.user.js:5535">
P2: Exact mode loses bytes when the payload starts with a UTF-8 BOM or contains invalid UTF-8. `TextDecoder` strips the BOM and substitutes invalid bytes with U+FFFD, so use a byte-preserving representation/decoder for the returned data (and define how non-UTF-8 bytes are displayed).</violation>
<violation number="2" location="XMOJ.user.js:5576">
P2: When `PreserveLineBreaks` is true, the button displays `保留换行：是` but exposes `aria-pressed="false"`, so assistive technology reports the opposite state. Bind `aria-pressed` to `PreserveLineBreaks`.</violation>
</file>
Reply with feedback, questions, or to request a fix.
Re-trigger cubic
| if (rawData.length >= 4 && rawData[0] === 78 && rawData[1] === 83 && rawData[2] === 67 && (rawData[3] === 49 || rawData[3] === 51)) { | ||
| return NumberStreamDecode(rawData); | ||
| } | ||
| return new TextDecoder().decode(await GzipDecode(rawData)); |
Contributor
There was a problem hiding this comment.
P2: Exact mode loses bytes when the payload starts with a UTF-8 BOM or contains invalid UTF-8. TextDecoder strips the BOM and substitutes invalid bytes with U+FFFD, so use a byte-preserving representation/decoder for the returned data (and define how non-UTF-8 bytes are displayed).
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At XMOJ.user.js, line 5535:
<comment>Exact mode loses bytes when the payload starts with a UTF-8 BOM or contains invalid UTF-8. `TextDecoder` strips the BOM and substitutes invalid bytes with U+FFFD, so use a byte-preserving representation/decoder for the returned data (and define how non-UTF-8 bytes are displayed).</comment>
<file context>
@@ -5174,10 +5174,428 @@ async function main() {
+ if (rawData.length >= 4 && rawData[0] === 78 && rawData[1] === 83 && rawData[2] === 67 && (rawData[3] === 49 || rawData[3] === 51)) {
+ return NumberStreamDecode(rawData);
+ }
+ return new TextDecoder().decode(await GzipDecode(rawData));
+ }
+
</file context>
| LineBreakButton.disabled = !NumberStreamEnabled; | ||
| LineBreakButton.classList.toggle("btn-outline-secondary", PreserveLineBreaks || !NumberStreamEnabled); | ||
| LineBreakButton.classList.toggle("btn-outline-warning", !PreserveLineBreaks && NumberStreamEnabled); | ||
| LineBreakButton.setAttribute("aria-pressed", String(!PreserveLineBreaks)); |
Contributor
There was a problem hiding this comment.
P2: When PreserveLineBreaks is true, the button displays 保留换行：是 but exposes aria-pressed="false", so assistive technology reports the opposite state. Bind aria-pressed to PreserveLineBreaks.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At XMOJ.user.js, line 5576:
<comment>When `PreserveLineBreaks` is true, the button displays `保留换行：是` but exposes `aria-pressed="false"`, so assistive technology reports the opposite state. Bind `aria-pressed` to `PreserveLineBreaks`.</comment>
<file context>
@@ -5174,10 +5174,428 @@ async function main() {
+ LineBreakButton.disabled = !NumberStreamEnabled;
+ LineBreakButton.classList.toggle("btn-outline-secondary", PreserveLineBreaks || !NumberStreamEnabled);
+ LineBreakButton.classList.toggle("btn-outline-warning", !PreserveLineBreaks && NumberStreamEnabled);
+ LineBreakButton.setAttribute("aria-pressed", String(!PreserveLineBreaks));
+ }
+ LineBreakButton.addEventListener("click", () => {
</file context>
Suggested change
| LineBreakButton.setAttribute("aria-pressed", String(!PreserveLineBreaks)); | |
| LineBreakButton.setAttribute("aria-pressed", String(PreserveLineBreaks)); |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR aim to accomplish?
Upgrade the existing
ApplyData(“获取数据”) component from plain Base64 transport to compressed, bracket-safe transports that support larger inputs while retaining a safe exact mode.How does this PR accomplish the above?
[and]from Base93 so payload boundaries remain unambiguous;long longrange with JavaScriptBigInt;#apply_datawhen the page does not provide it;IOFile;cerr, then callsabort()to triggerSIGABRTwithout thelogic_errorwrapper;[payload]stderr lines and legacywhat(): [payload]output;Mode details
getline/character-level APIs.long longrange. NSC3 separates values from per-line number counts and compresses both streams using packed raw, delta, frame-of-reference, run, and back-reference blocks.cin >> valueorscanf("%lld", ...).getline, line-based parsing, meaningful row/column boundaries, or code that requires a final newline. Mixed/noncanonical input still falls back verbatim and is not flattened.The “保留换行” control is available only when “高速数值模式” is enabled and defaults to “是”. Enabling numeric mode alone therefore does not alter line layout.
Known limitation
With exact line preservation, 200,000 uniformly random line lengths from 1–8 carry roughly 75 KB of unavoidable structure. The measured NSC3 result is 79,714 bytes / about 97,510 Base93 characters, which can exceed the judge output limit. When line boundaries are semantically irrelevant, disabling preservation reduces that tested payload to 26 Base93 characters.
Verification
XMOJ.user.jspassesnode --check;SIGABRTwhile retaining a recoverable stderr payload;Summary by Sourcery
Upgrade ApplyData transport and controls to support smaller, exact payloads and optimized numeric input generation.
New Features:
Bug Fixes:
Enhancements:
Build:
Summary by cubic
Compresses ApplyData payloads with gzip + bracket‑safe Base93 and adds an optional high‑speed numeric stream (NSC3). Previously we Base64‑encoded and threw an exception; now we write a bracketed Base93 payload to stderr and abort, reducing size and improving extraction while preserving exact bytes.
[and]excluded; optional NSC3 for signed 64‑bit input, withNSC_IGNORE_LINESto flatten lines; writes [payload] to stderr, flushes, then aborts; removes the Base64/exception path.what(): [payload]; decodes Base93 to gzip or NSC1/NSC3; reconstructs 64‑bit values via BigInt; supports multiple payloads; ignores malformed ones; treats no payload as a normal failure.#apply_datais missing; adds toggles for “高速数值模式” and “保留换行”(default on) with inline explanations.package.jsonandUpdate.json; no migrations.Written for commit 0d5fef7. Summary will update on new commits.
Summary by Sourcery
Upgrade ApplyData transport and decoding to support smaller exact payloads and an optional high-speed numeric mode while preserving compatibility and input fidelity.
New Features:
Bug Fixes:
Enhancements:
Build: