fix(engine): persist Last-Modified for hint-mode multi-segment downloads - #214
Merged
Merged
Conversation
use_server_time silently degraded to local completion time for large files downloaded via the browser extension takeover path. Hint-mode downloads (hint_file_size != 0) skip the probe phase to protect one-time signed URLs, leaving FileInfo.last_modified empty. The single-stream path self-heals by latching Last-Modified from its own real response (latched_last_modified), but the multi-segment path only kept the first segment's validator in an in-memory mutex for cross-segment consistency checking (check_cross_segment_validators) and never surfaced it to apply_server_mtime. - segment_coordinator::do_segment now persists the cross-segment- validated (ETag, Last-Modified) pair into tasks.orig_etag/ orig_last_modified via db.set_task_validator, reusing the existing validator column instead of adding new state. - downloader::run_download_inner falls back to reading that DB validator when both the single-stream latch and resume_last_modified are empty, before calling apply_server_mtime. Fixes #150
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Repro
开启「使用服务器文件时间」(
use_server_time)后,通过 FluxDown 浏览器扩展从 https://dotnet.microsoft.com/zh-cn/download/dotnet/8.0 下载 ASP.NET Core 运行时 / .NET 桌面运行时安装包(数十 MB,触发多段下载),完成后文件修改时间是本地下载完成时间,而非服务器Last-Modified。等价静态复现:curl -sI -L https://aka.ms/dotnet/8.0/aspnetcore-runtime-win-x64.exe与同 URL 加-H 'Range: bytes=0-0',确认服务器(重定向到builds.dotnet.microsoft.com,Azure Blob 存储)对 HEAD 与 Range GET 均正常返回Last-Modified,排除服务器端异常。Cause
浏览器扩展拦截下载时携带
hint_file_size,run_download_inner(native/engine/src/downloader.rs)命中p.hint_file_size != 0分支,为保护一次性签名 URL 而跳过 probe 阶段,FileInfo.last_modified恒为空串,resume_last_modified随之为空。安装包体积走多段下载(download_multi_segment→segment_coordinator::run_coordinated_download),该路径只在do_segment内把首个 206 响应的(ETag, Last-Modified)锁存进内存first_validators,仅用于check_cross_segment_validators的跨段一致性校验,从未落库或回传给下载完成后的apply_server_mtime调用点。该调用点在多段路径(single_result为None)下只会unwrap_or(&resume_last_modified)(空串),apply_server_mtime因last_modified.is_empty()提前返回,文件保留本地完成时间。单流路径不受影响:download_single会从【实际响应】直接锁存Last-Modified(latched_last_modified字段),不依赖 probe。Fix
native/engine/src/segment_coordinator.rs:do_segment在 hint 模式(expected_etag/expected_last_modified均为空)下,跨段一致性校验通过后,把(resp_etag, resp_lm)落库到tasks.orig_etag/orig_last_modified(复用既有db.set_task_validator,与非 hint 模式的持久化路径一致,每段重复写入同一份值,幂等无害)。native/engine/src/downloader.rs:run_download_inner的use_server_time分支新增第三级回退——单流锁存值 →resume_last_modified→ (均为空时)回读 DB validator,再交给apply_server_mtime。Verification
未在机器人环境中构建或测试——运行环境无法承载本项目的 Rust 工具链(2 核 / 1.8GB 容器)。本改动经静态审查:
run_download_inner(downloader.rs:2383-3291)完整调用链,确认resume_last_modified在改动点仍处于作用域内且未被移动(此前仅以&resume_last_modified借用,无所有权转移)。do_segment/check_cross_segment_validators(segment_coordinator.rs:4302-4644),确认db: &Db、task_id: &str、resp_etag/resp_lm: &str均已在作用域内;db.set_task_validator(id: &str, etag: &str, last_modified: &str) -> Result<(), DbError>签名与既有调用点(downloader.rs:2647-2656)一致。DbError经#[derive(Error, Debug)](thiserror,db.rs:11-15)实现Display,新增log_info!("...{}...", e)格式化合法。if cond && let Err(e) = ... {}而非嵌套if { if let },匹配本文件既有 let-chain 风格(如segment_coordinator.rs中expected_etag/expected_last_modified守卫),规避clippy::collapsible_if。native/engine/tests/{realtest,corruption_test,multi_cdn}.rs等直接调用run_coordinated_download的测试文件(其调用点参数不变)。请维护者执行:
已知未覆盖:未新增自动化测试覆盖 hint + 多段 +
use_server_time组合(需要本地 HTTP 测试服务器模拟浏览器扩展 hint 下载并强制多段,native/engine/tests/realtest.rs已有类似基础设施run_full_server_time/use_server_time_applies_last_modified_to_file_mtime,可参照扩展一个 hint 模式版本)。Fixes #150