From bc58ae3efd5f8d16e2cfb5bf5e4426c831153160 Mon Sep 17 00:00:00 2001 From: abruption Date: Wed, 21 Oct 2026 11:30:11 +0900 Subject: [PATCH 1/4] feat: add release checks notifications and verified manual updates --- CHANGELOG.md | 7 +- README.ja.md | 2 + README.ko.md | 2 + README.md | 2 + README.zh-CN.md | 2 + docs/usage.ja.md | 56 +++ docs/usage.ko.md | 56 +++ docs/usage.md | 56 +++ docs/usage.zh-CN.md | 56 +++ dots-ts | 942 ++++++++++++++++++++++++++++++++++++ source-manifest.json | 4 +- tailscale-setup.sh | 7 +- tests/test_update_apply.py | 287 +++++++++++ tests/test_update_check.py | 196 ++++++++ tests/test_update_notice.py | 157 ++++++ 15 files changed, 1828 insertions(+), 4 deletions(-) create mode 100644 tests/test_update_apply.py create mode 100644 tests/test_update_check.py create mode 100644 tests/test_update_notice.py diff --git a/CHANGELOG.md b/CHANGELOG.md index 111e596..59bf69d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,7 +6,12 @@ - Require a matching source manifest for new installations; preserve unknown metadata for legacy installations instead of guessing a version. - Add package integrity to `doctor` and document the development feature in all four languages. -This development work has not been deployed to Dots. Release checks, notifications and verified updates remain separate work in #28–#30. +- Add bounded stable-release checks with explicit download proxy selection and validated release manifests (#28). +- Add opt-in, cached local notifications with expiry, deduplication and no network work during startup (#29). + +- Add detached manual updates from fixed releases or explicitly reviewed local bundles, with code-only backups, a persistent transaction journal and bounded shim rollback (#30). + +This development work is undergoing validation; live deployment evidence will be recorded separately. ## v0.3.1 — 2026-10-19 diff --git a/README.ja.md b/README.ja.md index bb706d1..1c5387c 100644 --- a/README.ja.md +++ b/README.ja.md @@ -51,6 +51,8 @@ bash /workspace/.dots-tailscale/start.sh login 開発版では `dots-ts version` と `dots-ts version --json` で、dots-tailscale パッケージと Tailscale バイナリのバージョンを別々に確認できます。v0.3.1 にはまだ含まれていません。詳しくは[パッケージのバージョンとファイルの確認](docs/usage.ja.md#package-version)を参照してください。 +開発版には、安定版リリースの確認、任意で有効にするキャッシュ通知、コードのバックアップとロールバックに対応した手動更新も追加されます。実行条件、独立した復旧用ターミナルの確認、別途行う SSH 検証は[更新ガイド](docs/usage.ja.md#update-management)を参照してください。 + ## 同梱内容 | パス | 用途 | diff --git a/README.ko.md b/README.ko.md index ec6c6f4..87aa60e 100644 --- a/README.ko.md +++ b/README.ko.md @@ -51,6 +51,8 @@ bash /workspace/.dots-tailscale/start.sh login 개발 버전의 `dots-ts version`과 `dots-ts version --json`은 dots-tailscale 패키지 버전과 내부 Tailscale 버전을 따로 보여줍니다. v0.3.1에는 아직 없는 명령입니다. 자세한 내용은 [패키지 버전과 파일 확인](docs/usage.ko.md#package-version)을 참고하세요. +개발 버전에는 정식 릴리스 확인, 선택적으로 켜는 캐시 알림, 코드 백업과 원복을 갖춘 수동 업데이트도 추가됩니다. 실행 조건, 복구 터미널 확인과 별도 SSH 검증은 [업데이트 안내](docs/usage.ko.md#update-management)를 참고하세요. + ## 구성 요소 | 경로 | 역할 | diff --git a/README.md b/README.md index af450d6..759b315 100644 --- a/README.md +++ b/README.md @@ -51,6 +51,8 @@ Install all four source files from [v0.3.1](https://github.com/abruption/dots-ta The development build adds `dots-ts version` and `dots-ts version --json` to show the dots-tailscale package version separately from the Tailscale binary version. These commands are not included in v0.3.1. See [package version and file verification](docs/usage.md#package-version). +The development build also adds stable-release checks, opt-in cached notifications and verified manual updates with code-only rollback. See [update management](docs/usage.md#update-management) for requirements, recovery-terminal confirmation and the separate external SSH check. + ## Included | Path | Purpose | diff --git a/README.zh-CN.md b/README.zh-CN.md index 9d1926e..6bd3ff6 100644 --- a/README.zh-CN.md +++ b/README.zh-CN.md @@ -51,6 +51,8 @@ bash /workspace/.dots-tailscale/start.sh login 开发版新增了 `dots-ts version` 和 `dots-ts version --json`，可以分别查看 dots-tailscale 软件包版本和 Tailscale 程序版本。v0.3.1 尚不包含这些命令。详情见[软件包版本与文件校验](docs/usage.zh-CN.md#package-version)。 +开发版还新增了正式版本检查、可选的缓存通知，以及支持代码备份和回滚的手动更新。运行条件、独立恢复终端确认和单独的 SSH 验证，请参阅[更新说明](docs/usage.zh-CN.md#update-management)。 + ## 项目内容 | 路径 | 用途 | diff --git a/docs/usage.ja.md b/docs/usage.ja.md index a7e5e7f..35d6061 100644 --- a/docs/usage.ja.md +++ b/docs/usage.ja.md @@ -159,6 +159,62 @@ JSON では `package.declared_version`、`package.source_revision`、`package.in メンテナンス時は、確認済みのソース変更後に `python3 scripts/source_manifest.py` を実行し、差分を確認してから `python3 scripts/source_manifest.py --check` で検証します。CI はファイルを書き換えず、検証のみ行います。生成は確認済みのソースチェックアウトで行い、インストール先では行わないでください。リリースにはソースファイル 4 つに対応するマニフェストも含める必要があります。既存の v0.3.1 リリースには含まれていません。 + + +## リリース確認とローカル通知（開発中） + +これらのコマンドは v0.4.0 に向けて開発中です。新しいヘルパーが必要です。リリースを公開しても、既存のインストールは更新されません。リリース確認では `abruption/dots-tailscale` の安定版のうち、ソースファイル 4 つと一致する `source-manifest.json` を含むものだけを対象にします。このマニフェストを含まない古いリリース（v0.3.1 を含む）は、検証済みの更新用バンドルとして扱わず、非対応として報告します。 + +```bash +/workspace/.dots-tailscale/dots-ts update check --json +/workspace/.dots-tailscale/dots-ts update check --tag v0.4.0 --proxy http://browser-proxy:8889 --json +``` + +2つ目の例は特定の安定版を選択します。そのリリースが公開された後にのみ使用できます。`--proxy` を指定しない場合、確認では直接 HTTPS を使い、シェルのプロキシ認証情報を引き継ぎません。必要な場合は、プラットフォームのダウンロード用プロキシを明示的に指定してください。shim 用およびアプリケーション用の SOCKS/HTTP プロキシとは別の設定です。HTTPS 証明書の検証は有効のままです。GitHub トークンは不要です。 + +`update check` はインストール済みのコード、ノード state、設定、プロセスを変更しません。JSON の `status` は `update_available`、`current`、`local_newer`、`not_comparable`、`check_failed` のいずれかです。`selection` は最新バージョンを選んだか、バージョンを固定して選んだかを示します。終了コードは、比較が有効なら `0`、ローカルのインストールを比較できない場合は `1`、検索に失敗した場合は `2` です。終了コード `0` だから更新があると決めつけず、status を確認してください。リモートのリリースが見つかっても、不明、変更済み、または異なる版が混在するローカルソースが信頼済みのバージョンになるわけではありません。 + +通知は明示的に有効にした場合だけ使われ、ローカルで処理されます。更新をインストールせず、タイマーを登録せず、外部メッセージも送信しません。 + +```bash +/workspace/.dots-tailscale/dots-ts update notifications configure --enable --interval-hours 24 +/workspace/.dots-tailscale/dots-ts update notifications refresh --proxy http://browser-proxy:8889 +/workspace/.dots-tailscale/dots-ts update notifications show +/workspace/.dots-tailscale/dots-ts update notifications notify +/workspace/.dots-tailscale/dots-ts update notifications configure --disable +``` + +`refresh` は時間を制限したネットワーク確認を行い、結果をローカルに保存します。設定済みの間隔に従い、間隔が経過する前に再確認する場合は `--force` を明示的に指定します。`notify` はキャッシュだけを読み取り、有効期限を確認し、同じバージョンの通知を繰り返しません。対話型の起動が成功すると、短い時間制限内に stderr へキャッシュ済み通知を表示する場合があります。通知確認が失敗しても起動結果は変わりません。生の `cli` 転送や JSON 出力には通知文を追加しません。保存先が読み取り専用の場合や `refresh` に失敗した場合、古い結果を新しい確認の成功として扱うことはありません。記録時刻と期限切れ／エラー状態は `show` で確認してください。定期的な `refresh` には、別途明示的に設定した呼び出し元が必要です。VM が停止している間に VM 内のコマンドが実行されることはありません。 + +### 確認済みバンドルによる手動更新 + +インストール先、ランタイムソケット、Linux のプロセス情報を参照できる環境で実行してください。`update apply` には Linux、Python 3.9 以降、pidfd のサポートが必要です。開始前に、クラウドコンピューターを操作できる独立したターミナルを開いてください。`--confirm-recovery-terminal` は確認したことを記録するだけで、そのターミナルが利用可能かを検査するものではありません。トランザクションワーカーは呼び出し元のセッションから切り離されるため、SSH 経由で処理を開始できますが、SSH 接続自体は復旧手段になりません。 + +確認済みの安定版を固定して選びます。 + +```bash +/workspace/.dots-tailscale/dots-ts update apply --tag v0.4.1 --proxy http://browser-proxy:8889 --confirm-recovery-terminal --json +/workspace/.dots-tailscale/dots-ts update status --id TRANSACTION_ID --json +``` + +これは使用例であり、v0.4.1 の公開を告知するものではありません。`TRANSACTION_ID` は apply が返す ID に置き換えてください。リリースのダウンロードがブロックされている場合や、未公開候補を明示的に試す場合は、確認済みのローカルソースバンドルも使用できます。 + +```bash +/workspace/.dots-tailscale/dots-ts update apply --bundle /workspace/reviewed-bundle --confirm-recovery-terminal --json +``` + +バンドルにはソースファイル 4 つと、それらに対応する元のマニフェストが必要です。ローカル選択では開発版も指定できますが、公開済みの GitHub リリースとしては扱いません。更新ツールはファイルを置き換える前にハッシュと構文を検証します。チェックサムが一致することは選択したバンドルとの整合性を示しますが、作成者を独立に信頼できることの証明にはなりません。 + +`started` は切り離されたワーカーが処理を受け付けたことを示し、更新の成功を意味しません。再接続した後に `update status` を確認してください。最終状態は `applied`、`failed_rolled_back`、`rollback_failed_manual_required` を区別します。ローカルの準備状態とバージョン確認だけでは外部からの SSH 接続を証明できません。`applied` またはロールバック成功後に、新しい SSH 接続を厳格なホスト鍵確認付きで開き、目的の接続を別途テストしてください。 + +ワーカーは、setup と proxy apply が使うものと同じ操作ロックを保持したまま、4 つのソースファイルとそのマニフェストだけをバックアップして置き換えます。再起動するのは特定済みの shim プロセスだけです。tailscaled と sshd は動作したままです。ノード state、ログイン資格情報、秘密鍵、SSH 設定、プロキシ設定をコピーしたり初期化したりしません。同梱 Tailscale バイナリのバージョンを変更するリリースは、この更新ツールの対象外として拒否されます。識別できないプロセスを終了したり、ソケットを削除したりしません。 + +インストール済みのマニフェストが一致している必要があります。メタデータのない古いインストールでは、インストール済みの 4 ファイルすべてを、別途入手して確認したリリースと照合してください。インストール済みファイル自身から期待するチェックサムを生成しないでください。出所を確認できない場合や、異なるソースが混在している場合は手動確認のため中止します。既存インストールに対する意図的な移行でソースチェックアウトを使う場合も、更新ツールを実行する前に、そのチェックアウトが自身のマニフェストと一致している必要があります。 + +未完了のトランザクションがあると、新しい更新は開始できません。ワーカーの強制終了後や VM の入れ替え後は、独立したターミナルから状態とコードのみのバックアップを確認してください。再試行のためにジャーナルを削除したり、次の処理が安全だと決めつけたりしないでください。ロールバック後も準備状態を確認できない場合は、証拠を保持し、確認済みのバックアップから手動で復旧してください。この更新ツールは VM を自動復旧するサービスではありません。 + +トランザクションの記録は `.update-transactions/TRANSACTION_ID/` に保存されます。`journal.json` は進行段階、`stage/` は選択したバンドル、`backup/` は以前のソースのみを保持し、`supervisor/` には updater のコードのスナップショットを保存します。外部から接続を確認するまでは記録を残してください。`update status` の終了コードは、実行中または適用済みなら 0、拒否、ロールバック、中断、手動復旧が必要な場合は 1 です。実行中の終了コード 0 は完了を意味しません。 + ## v0.3.1 ヘルパーと任意のアプリケーションプロキシ diff --git a/docs/usage.ko.md b/docs/usage.ko.md index 2f86645..fd89fdf 100644 --- a/docs/usage.ko.md +++ b/docs/usage.ko.md @@ -108,6 +108,62 @@ JSON은 `package.declared_version`, `package.source_revision`, `package.integrit 유지보수 시에는 검토한 소스 변경 후 `python3 scripts/source_manifest.py`로 명세를 생성하고 차이를 확인한 뒤 `python3 scripts/source_manifest.py --check`로 검사합니다. CI는 파일을 수정하지 않고 검사만 합니다. 설치 경로가 아닌 검토된 소스 체크아웃에서만 생성하세요. 릴리스에는 소스 파일 네 개와 일치하는 명세도 함께 배포해야 합니다. 기존 v0.3.1 릴리스에는 이 명세가 없습니다. + + +## 새 릴리스 확인과 로컬 알림 (개발 중) + +v0.4.0을 위해 개발 중인 기능으로 새 도구가 설치되어 있어야 합니다. 릴리스 발행만으로 기존 설치가 바뀌지는 않습니다. `abruption/dots-tailscale`의 정식 릴리스 중 소스 파일 네 개와 일치하는 `source-manifest.json`이 있는 묶음만 확인합니다. v0.3.1처럼 명세가 없는 기존 릴리스는 검증된 업데이트 대상으로 간주하지 않고 지원되지 않는다고 표시합니다. + +```bash +/workspace/.dots-tailscale/dots-ts update check --json +/workspace/.dots-tailscale/dots-ts update check --tag v0.4.0 --proxy http://browser-proxy:8889 --json +``` + +두 번째 예시는 특정 정식 버전을 선택합니다. 실제로 해당 릴리스가 발행된 뒤에 사용할 수 있습니다. `--proxy`가 없으면 직접 HTTPS로 연결하며 셸의 프록시 자격정보를 가져오지 않습니다. 필요한 경우 플랫폼 다운로드 프록시를 직접 지정하세요. shim이나 애플리케이션 SOCKS·HTTP 프록시와는 별개입니다. HTTPS 인증서 검증은 유지하며 GitHub 토큰은 필요하지 않습니다. + +`update check`는 설치 코드, 노드 상태, 설정이나 프로세스를 바꾸지 않습니다. JSON의 `status`는 `update_available`, `current`, `local_newer`, `not_comparable`, `check_failed` 중 하나이고, `selection`은 최신 릴리스 조회인지 특정 버전 선택인지 구분합니다. 정상 비교는 종료 코드 0, 로컬 설치를 비교할 수 없으면 1, 조회 실패는 2입니다. 종료 코드 0만 보고 새 버전이 있다고 판단하지 말고 status를 읽으세요. 원격 릴리스를 찾았다고 해서 버전을 알 수 없거나 파일이 수정·혼합된 로컬 설치가 검증되는 것은 아닙니다. + +알림은 직접 켜야 하며 로컬에서만 표시합니다. 업데이트를 설치하거나 타이머를 등록하거나 외부로 메시지를 보내지 않습니다. + +```bash +/workspace/.dots-tailscale/dots-ts update notifications configure --enable --interval-hours 24 +/workspace/.dots-tailscale/dots-ts update notifications refresh --proxy http://browser-proxy:8889 +/workspace/.dots-tailscale/dots-ts update notifications show +/workspace/.dots-tailscale/dots-ts update notifications notify +/workspace/.dots-tailscale/dots-ts update notifications configure --disable +``` + +`refresh`는 제한된 시간 안에 네트워크로 조회하고 결과를 저장합니다. 설정한 간격 안에서는 유효한 결과를 재사용하며, `--force`를 지정하면 그 전에 새로 조회합니다. `notify`는 저장된 결과만 읽으며 만료된 결과와 같은 버전의 반복 안내를 걸러냅니다. 대화형 터미널에서 기동에 성공하면 짧은 제한 시간 안에 캐시된 안내를 stderr로 보여줄 수 있습니다. 알림 확인이 실패해도 기동 결과는 바뀌지 않습니다. 원시 `cli` 전달이나 JSON 출력에는 안내 문구가 섞이지 않습니다. 저장 경로가 읽기 전용이거나 갱신에 실패해도 오래된 결과를 새로 확인한 결과로 취급하지 않습니다. `show`에서 기록 시각과 stale·오류 상태를 확인하세요. 주기적으로 갱신하려면 호출 수단을 별도로 설정해야 하며, VM이 없을 때는 VM 안의 명령도 실행되지 않습니다. + +### 검증된 수동 업데이트 + +설치 경로, 런타임 소켓과 Linux 프로세스 정보가 보이는 환경에서 실행하세요. `update apply`에는 Linux, Python 3.9 이상과 pidfd 지원이 필요합니다. 먼저 클라우드 컴퓨터의 독립 터미널을 열고 제어가 되는지 확인하세요. `--confirm-recovery-terminal`은 사용자가 이를 확인했다는 뜻이며, 프로그램이 터미널의 작동 여부를 검사해 주지는 않습니다. 작업 프로세스가 호출 세션에서 분리되므로 SSH에서 시작할 수 있지만, 그 SSH 연결 자체를 복구 경로로 삼아서는 안 됩니다. + +조회한 정식 릴리스 중 특정 버전을 선택합니다. + +```bash +/workspace/.dots-tailscale/dots-ts update apply --tag v0.4.1 --proxy http://browser-proxy:8889 --confirm-recovery-terminal --json +/workspace/.dots-tailscale/dots-ts update status --id TRANSACTION_ID --json +``` + +사용 예시이며 v0.4.1이 이미 발행되었다는 뜻은 아닙니다. `TRANSACTION_ID`는 apply가 반환한 ID로 바꾸세요. 릴리스 파일 다운로드가 막혔거나 미발행 후보를 직접 시험할 때는 검토한 로컬 소스 묶음을 사용할 수 있습니다. + +```bash +/workspace/.dots-tailscale/dots-ts update apply --bundle /workspace/reviewed-bundle --confirm-recovery-terminal --json +``` + +소스 파일 네 개와 해당 파일에 일치하는 원본 명세가 필요합니다. 로컬 묶음에는 개발 버전을 사용할 수 있지만, 이를 GitHub에 발행된 릴리스로 표시하지는 않습니다. 교체 전에 해시와 문법을 검사합니다. 체크섬 일치는 선택한 묶음과 파일이 같다는 뜻이며, 작성자의 신뢰성을 별도로 증명하지는 않습니다. + +`started`는 분리된 작업 프로세스가 작업을 접수했다는 뜻으로, 업데이트 성공이 아닙니다. 재접속 후 `update status`로 확인하세요. 최종 결과는 `applied`, `failed_rolled_back`, `rollback_failed_manual_required`로 구분합니다. 로컬 준비 상태와 버전 확인만으로 외부 SSH 접속까지 확인되지는 않습니다. 적용이나 원복 성공 후에는 엄격한 호스트 키 검증을 유지한 새 SSH 연결로 실제 접속을 별도로 확인하세요. + +setup과 proxy apply가 사용하는 작업 잠금을 유지한 채 소스 파일 네 개와 명세만 백업·교체합니다. 정확히 식별한 shim 프로세스만 재시작하고 tailscaled와 sshd는 계속 실행합니다. 노드 상태, 로그인 정보, 개인키, SSH 설정과 프록시 설정은 복사하거나 초기화하지 않습니다. 내부 Tailscale 실행 파일 버전을 바꾸는 릴리스는 이 updater의 지원 범위 밖이므로 거부합니다. 정체를 알 수 없는 프로세스를 종료하거나 소켓을 지우지 않습니다. + +설치 파일과 일치하는 명세가 있어야 합니다. 메타데이터가 없는 구버전은 먼저 별도로 확보하고 검토한 릴리스와 설치 파일 네 개를 비교하세요. 설치된 파일에서 기대 체크섬을 새로 만들면 안 됩니다. 출처를 확인할 수 없거나 파일이 섞여 있으면 수동 검토를 위해 멈추세요. 의도적으로 마이그레이션할 때 사용하는 소스 체크아웃도 자체 명세와 일치해야 기존 설치를 대상으로 updater를 실행할 수 있습니다. + +이전 작업이 미완료 상태면 새 업데이트를 막습니다. 작업 프로세스가 강제 종료되거나 VM이 교체되면 독립 터미널에서 상태와 코드 백업을 확인하세요. 다시 시도하려고 기록을 삭제하거나 다음 단계가 안전하다고 가정하지 마세요. 원복 후에도 준비 상태를 확인할 수 없으면 증거를 보존하고 검증된 백업으로 수동 복구해야 합니다. 이 updater는 VM을 자동으로 복구하는 서비스가 아닙니다. + +작업 기록은 `.update-transactions/TRANSACTION_ID/`에 남습니다. `journal.json`에는 진행 단계, `stage/`에는 선택한 묶음, `backup/`에는 이전 소스 묶음만 저장하고, `supervisor/`에는 updater 코드 사본을 둡니다. 외부 연결을 확인할 때까지 기록을 보존하세요. `update status`의 종료 코드는 진행 중이거나 적용된 작업은 0, 거부·원복·중단·수동 복구 필요 결과는 1입니다. 진행 중 작업의 종료 코드 0은 완료를 뜻하지 않습니다. + ## v0.3.1 도구와 선택적 애플리케이션 프록시 diff --git a/docs/usage.md b/docs/usage.md index fb0511f..75631bf 100644 --- a/docs/usage.md +++ b/docs/usage.md @@ -159,6 +159,62 @@ JSON separates `package.declared_version`, `package.source_revision`, `package.i For maintainers: after intentional source changes, run `python3 scripts/source_manifest.py`, review the resulting diff, then run `python3 scripts/source_manifest.py --check`. CI runs the check without rewriting files. Regenerate only in the reviewed source checkout, not in an installed directory. A release must distribute the matching manifest alongside the four source assets; the existing v0.3.1 release does not contain it. + + +## Release checks and local notifications (development) + +These commands are being developed for v0.4.0. They require the new helpers; publishing a release does not update an existing installation. Release checks accept only stable releases from `abruption/dots-tailscale` with the four source files and a matching `source-manifest.json`. Older releases without this manifest, including v0.3.1, are reported as unsupported rather than treated as verified update bundles. + +```bash +/workspace/.dots-tailscale/dots-ts update check --json +/workspace/.dots-tailscale/dots-ts update check --tag v0.4.0 --proxy http://browser-proxy:8889 --json +``` + +The second example selects a specific stable version; it works only after that release exists. Without `--proxy`, the check uses direct HTTPS and does not inherit proxy credentials from the shell. Pass the platform download proxy explicitly when needed. This is separate from the shim and application SOCKS/HTTP proxies. HTTPS certificate verification remains enabled. No GitHub token is needed. + +`update check` changes no installed code, node state, configuration or processes. The JSON `status` is `update_available`, `current`, `local_newer`, `not_comparable`, or `check_failed`; `selection` identifies latest versus pinned selection. The exit code is 0 for a valid comparison, 1 when the local installation cannot be compared, and 2 for a lookup failure. Check the status rather than assuming exit 0 means an update exists. Unknown, modified or mixed local sources do not become a trusted version merely because a remote release was found. + +Notifications are opt-in and local. They do not install updates, register a timer or send external messages: + +```bash +/workspace/.dots-tailscale/dots-ts update notifications configure --enable --interval-hours 24 +/workspace/.dots-tailscale/dots-ts update notifications refresh --proxy http://browser-proxy:8889 +/workspace/.dots-tailscale/dots-ts update notifications show +/workspace/.dots-tailscale/dots-ts update notifications notify +/workspace/.dots-tailscale/dots-ts update notifications configure --disable +``` + +`refresh` performs the bounded network check and saves a local result. It reuses a fresh result within the configured interval; `--force` explicitly refreshes early. `notify` reads the cache only, respects expiry and suppresses repeated notices for the same version. Successful interactive startup can display a cached notice on stderr within a short time limit; failed notification checks never change the startup result. Raw `cli` forwarding and JSON output do not gain notice text. Read-only storage or failed refreshes do not turn old data into a new successful check. Use `show` to inspect the recorded time and stale/error state. A separate, explicitly configured caller is needed for periodic refresh, and no VM-local command runs while the VM is absent. + +### Verified manual updates + +Run from an environment where the installation, its runtime socket and Linux process information are visible. `update apply` requires Linux with Python 3.9 or newer and pidfd support. Before starting, open the cloud computer's independent terminal and make sure you can control it. `--confirm-recovery-terminal` records your acknowledgement; it cannot check that terminal for you. SSH initiation is supported because the transaction worker detaches from the calling session, but the SSH connection itself is not a recovery path. + +Choose a fixed stable release after checking it: + +```bash +/workspace/.dots-tailscale/dots-ts update apply --tag v0.4.1 --proxy http://browser-proxy:8889 --confirm-recovery-terminal --json +/workspace/.dots-tailscale/dots-ts update status --id TRANSACTION_ID --json +``` + +These are examples, not an announcement that v0.4.1 exists. Replace `TRANSACTION_ID` with the ID returned by apply. A reviewed local source bundle can be used when release downloads are blocked or when explicitly testing an unreleased candidate: + +```bash +/workspace/.dots-tailscale/dots-ts update apply --bundle /workspace/reviewed-bundle --confirm-recovery-terminal --json +``` + +The bundle must contain the four source files and their matching original manifest. Local selection can contain a development version; it is not presented as a published GitHub release. The updater validates hashes and syntax before replacing files. A matching checksum proves consistency with the selected bundle, not independent trust in its author. + +`started` means the detached worker has accepted the operation, not that the update succeeded. Poll `update status` after reconnecting. Terminal outcomes distinguish `applied`, `failed_rolled_back`, and `rollback_failed_manual_required`. Local readiness and version verification do not prove external SSH access: after `applied` or a successful rollback, open a new SSH connection with strict host-key checking and test the intended connection separately. + +The worker backs up and replaces only the four source files and their manifest, while holding the same operation lock used by setup and proxy apply. It restarts only the identified shim process; tailscaled and sshd stay running. Node state, login credentials, private keys, SSH configuration and proxy settings are not copied or reset. Releases that change the bundled Tailscale binary version are outside this updater's scope and are refused. No unknown process is killed and no socket is removed. + +A matching installed manifest is required. For older installations without metadata, first compare all four installed files against an independently obtained, reviewed release. Do not generate expected checksums from the installed files themselves. If provenance cannot be established or files are mixed, stop for manual review. A source checkout used for a deliberate migration must also match its own manifest before its updater can run against an existing installation. + +A prior incomplete transaction blocks new updates. After a killed worker or VM replacement, inspect status and the code-only backup from the independent terminal; do not delete the journal to force a retry or assume the next step is safe. If rollback cannot establish readiness, keep the evidence and recover manually from the verified backup. The updater is not an automatic VM recovery service. + +Transaction records live under `.update-transactions/TRANSACTION_ID/`: `journal.json` records the phase, `stage/` contains the selected bundle, `backup/` contains only the previous source bundle, and `supervisor/` holds the updater code snapshot. Keep these records until you have checked external connectivity. `update status` returns exit 0 for a running or applied transaction and 1 for a rejected, rolled-back, interrupted or manual-recovery result. Exit 0 for a running transaction is not completion. + ## v0.3.1 helpers and optional application proxies diff --git a/docs/usage.zh-CN.md b/docs/usage.zh-CN.md index 9ee877a..34670ff 100644 --- a/docs/usage.zh-CN.md +++ b/docs/usage.zh-CN.md @@ -159,6 +159,62 @@ JSON 将 `package.declared_version`、`package.source_revision`、`package.integ 维护时，在确认源码修改后运行 `python3 scripts/source_manifest.py`，检查生成的差异，再运行 `python3 scripts/source_manifest.py --check`。CI 只检查，不重写文件。请只在已审核的源码检出目录中生成清单，不要在安装目录中生成。发布时需要将匹配的清单与四个源文件一起提供；现有 v0.3.1 发布包不包含此清单。 + + +## 发布检查与本地通知（开发中） + +这些命令正在为 v0.4.0 开发，需要新版辅助工具；发布新版本不会自动更新已有安装。发布检查只接受来自 `abruption/dots-tailscale` 的稳定版，且必须包含四个源文件和匹配的 `source-manifest.json`。不含此清单的旧版（包括 v0.3.1）会报告为不受支持，不会被当作已验证的更新包。 + +```bash +/workspace/.dots-tailscale/dots-ts update check --json +/workspace/.dots-tailscale/dots-ts update check --tag v0.4.0 --proxy http://browser-proxy:8889 --json +``` + +第二个示例会选择指定的稳定版本；只有该版本发布后才能使用。未指定 `--proxy` 时，检查会直接通过 HTTPS 连接，不会继承 shell 中的代理凭据。需要时请显式指定平台下载代理。这与 shim 代理以及应用 SOCKS/HTTP 代理相互独立。HTTPS 证书验证始终保持启用。无需 GitHub token。 + +`update check` 不会更改已安装代码、节点状态、配置或进程。JSON 中的 `status` 可为 `update_available`、`current`、`local_newer`、`not_comparable` 或 `check_failed`；`selection` 用于区分选择最新版本还是固定版本。有效比较的退出码为 `0`，本地安装无法比较时为 `1`，查询失败时为 `2`。请根据状态判断，不要假定退出码为 `0` 就表示存在更新。即使找到了远程发布版，未知、已修改或混杂的本地源文件也不会因此成为可信版本。 + +通知为可选的本地功能，不会安装更新、注册定时器或发送外部消息： + +```bash +/workspace/.dots-tailscale/dots-ts update notifications configure --enable --interval-hours 24 +/workspace/.dots-tailscale/dots-ts update notifications refresh --proxy http://browser-proxy:8889 +/workspace/.dots-tailscale/dots-ts update notifications show +/workspace/.dots-tailscale/dots-ts update notifications notify +/workspace/.dots-tailscale/dots-ts update notifications configure --disable +``` + +`refresh` 会执行有时限的网络检查并保存本地结果。`notify` 只读取缓存，遵守过期时间，并抑制同一版本的重复通知。交互式启动成功时，可在短时限内将缓存通知输出到 `stderr`；通知检查失败不会改变启动结果。原样转发的 `cli` 输出和 JSON 输出不会增加通知文字。存储只读或刷新失败时，不会把旧数据当作一次新的成功检查。使用 `show` 查看记录时间以及结果是否过期或出错。定期刷新需要另行显式配置调用方；VM 不运行时，VM 内不会执行任何命令。默认情况下，`refresh` 遵守已配置的间隔；传入 `--force` 可提前刷新。 + +### 经验证的手动更新 + +请在能够访问安装目录、运行时套接字和 Linux 进程信息的环境中运行。`update apply` 要求使用支持 pidfd 的 Linux，并且 Python 版本为 3.9 或更高。开始前，请打开云电脑上的独立终端并确认可以操作该终端。`--confirm-recovery-terminal` 只记录你已确认；它无法替你检查该终端。支持通过 SSH 发起更新，因为事务工作进程会与调用会话脱离；但 SSH 连接本身不能作为恢复途径。 + +先检查并选定一个固定的稳定版本： + +```bash +/workspace/.dots-tailscale/dots-ts update apply --tag v0.4.1 --proxy http://browser-proxy:8889 --confirm-recovery-terminal --json +/workspace/.dots-tailscale/dots-ts update status --id TRANSACTION_ID --json +``` + +以上仅为示例，并不表示 v0.4.1 已经发布。请将 `TRANSACTION_ID` 替换为 `apply` 返回的 ID。如果发布下载受阻，或需要明确测试尚未发布的候选版本，也可以使用经过审查的本地源文件包： + +```bash +/workspace/.dots-tailscale/dots-ts update apply --bundle /workspace/reviewed-bundle --confirm-recovery-terminal --json +``` + +源文件包必须包含四个源文件和与之匹配的原始清单。本地选择可以是开发版本，但不会被标记为已发布的 GitHub 版本。更新器会在替换文件前验证哈希和语法。校验和匹配只能证明文件与所选源文件包一致，不能独立证明发布者可信。 + +`started` 表示已脱离会话运行的工作进程接受了操作，不表示更新成功。重新连接后，请轮询 `update status`。最终状态包括 `applied`、`failed_rolled_back` 和 `rollback_failed_manual_required`。本地就绪状态和版本验证不能证明外部 SSH 可用：更新成功或成功回滚后，请使用严格主机密钥验证新建 SSH 连接，并单独测试所需连接。 + +工作进程会在与 setup 和 proxy apply 共用的操作锁保护下，仅备份和替换四个源文件及其清单。它只会重启已识别的 shim 进程；tailscaled 和 sshd 会继续运行。节点状态、登录凭据、私钥、SSH 配置和代理设置不会被复制或重置。若发布版本会更改捆绑的 Tailscale 二进制版本，此更新器会拒绝操作；该类更新不在其适用范围内。更新器不会终止无法识别的进程，也不会删除套接字。 + +必须有匹配的已安装清单。对于没有元数据的旧安装，先将全部四个已安装文件与独立获取并经过审查的发布版本逐一比对。不要根据已安装文件本身生成预期校验值。如果无法确认来源，或文件版本混杂，请停止并进行人工审查。若要用源码检出执行有意的迁移，该检出也必须先与自身清单匹配，更新器才能用于现有安装。 + +之前未完成的事务会阻止新更新。工作进程被终止或 VM 被替换后，请在独立终端检查状态和仅包含代码的备份；不要为了强行重试而删除事务记录，也不要假定下一步一定安全。如果回滚无法恢复就绪状态，请保留证据，并根据已验证的备份手动恢复。此更新器不是自动 VM 恢复服务。 + +事务记录保存在 `.update-transactions/TRANSACTION_ID/` 下：`journal.json` 记录阶段，`stage/` 保存选定的文件包，`backup/` 只保存之前的源码包，`supervisor/` 保存更新器代码快照。在确认外部连接正常前，请保留这些记录。`update status` 对运行中或已应用的事务返回退出码 0；对拒绝、已回滚、中断或需要手动恢复的结果返回 1。运行中的退出码 0 不代表已经完成。 + ## v0.3.1 辅助工具与可选应用代理 diff --git a/dots-ts b/dots-ts index d6227f8..88ce29b 100755 --- a/dots-ts +++ b/dots-ts @@ -1,6 +1,13 @@ #!/usr/bin/env python3 """Local Tailscale CLI forwarding and bounded, shareable diagnostics.""" import argparse +import base64 +import contextlib +import fcntl +import ssl +import urllib.error +import urllib.parse +import urllib.request import hashlib import selectors import time @@ -131,6 +138,859 @@ def tailscale_version(directory): process.stdout.close() +# All update requests are unauthenticated and confined to this repository. +UPDATE_REPOSITORY = "abruption/dots-tailscale" +UPDATE_API = "https://api.github.com/repos/" + UPDATE_REPOSITORY + "/releases" +UPDATE_WEB = "https://github.com/" + UPDATE_REPOSITORY +UPDATE_TIMEOUT = 12 +UPDATE_ERRORS = {"invalid_version", "invalid_proxy", "invalid_url", "redirect_host", "timeout", "offline", + "tls_error", "rate_limited", "not_found", "http_error", "response_too_large", + "invalid_response", "invalid_metadata", "unsupported_release", "no_stable_release"} + + +class UpdateError(Exception): + def __init__(self, code): + self.code = code if isinstance(code, str) and code in UPDATE_ERRORS else "invalid_response" + super().__init__(self.code) + + +def stable_version(text): + if not isinstance(text, str) or not re.fullmatch(r"(?:0|[1-9][0-9]{0,8})\.(?:0|[1-9][0-9]{0,8})\.(?:0|[1-9][0-9]{0,8})", text): + raise UpdateError("invalid_version") + return tuple(int(part) for part in text.split(".")) + + +def validate_proxy(proxy): + if proxy is None: + return None + try: + parsed = urllib.parse.urlsplit(proxy) + if (len(proxy) > 512 or parsed.scheme not in ("http", "https") or not parsed.hostname + or parsed.username is not None or parsed.password is not None + or parsed.path not in ("", "/") or parsed.query or parsed.fragment + or parsed.port is None or any(ord(c) < 33 or ord(c) > 126 for c in proxy)): + raise ValueError("invalid_proxy") + except (ValueError, TypeError): + raise UpdateError("invalid_proxy") + return proxy + + +def validate_update_url(url, redirect=False): + try: + parsed = urllib.parse.urlsplit(url) + if (not isinstance(url, str) or len(url) > 8192 or parsed.scheme != "https" + or parsed.username is not None or parsed.password is not None + or parsed.port not in (None, 443) or parsed.fragment + or any(ord(c) < 33 or ord(c) > 126 for c in url)): + raise ValueError("invalid_url") + if redirect: + valid = parsed.hostname in ("release-assets.githubusercontent.com", "objects.githubusercontent.com") + else: + api_path = "/repos/" + UPDATE_REPOSITORY + "/releases/" + asset_path = "/" + UPDATE_REPOSITORY + "/releases/download/" + valid = (not parsed.query and ( + parsed.hostname == "api.github.com" and (parsed.path == api_path + "latest" + or re.fullmatch(re.escape(api_path) + r"tags/v[0-9]+\.[0-9]+\.[0-9]+", parsed.path)) + or parsed.hostname == "github.com" and re.fullmatch(re.escape(asset_path) + + r"v[0-9]+\.[0-9]+\.[0-9]+/(?:tailscale-setup\.sh|shim\.py|dots-ts|dots_proxy\.py|source-manifest\.json)", parsed.path))) + if not valid: + raise ValueError("invalid_url") + except (ValueError, TypeError): + raise UpdateError("redirect_host" if redirect else "invalid_url") + return url + + +def _network_fetch(url, proxy, limit): + """Runs in a supervised child, so DNS and slow response bodies have a total limit.""" + validate_update_url(url) + validate_proxy(proxy) + class ReleaseRedirect(urllib.request.HTTPRedirectHandler): + max_redirections = 3 + max_repeats = 1 + def redirect_request(self, request, fp, code, message, headers, newurl): + # API requests never redirect; only release asset downloads may do so. + if urllib.parse.urlsplit(url).hostname != "github.com": + raise UpdateError("redirect_host") + validate_update_url(newurl, redirect=True) + return super().redirect_request(request, fp, code, message, headers, newurl) + opener = urllib.request.build_opener(urllib.request.ProxyHandler({"http": proxy, "https": proxy} if proxy else {}), + urllib.request.HTTPSHandler(context=ssl.create_default_context()), ReleaseRedirect()) + request = urllib.request.Request(url, headers={"Accept": "application/vnd.github+json" if url.startswith(UPDATE_API) else "application/octet-stream", + "User-Agent": "dots-ts-update", "Accept-Encoding": "identity"}) + try: + with opener.open(request, timeout=UPDATE_TIMEOUT) as response: + if response.status != 200: + raise UpdateError("http_error") + length = response.headers.get("Content-Length") + if length is not None and (not length.isdigit() or int(length) > limit): + raise UpdateError("response_too_large") + raw = response.read(limit + 1) + if len(raw) > limit: + raise UpdateError("response_too_large") + return raw + except urllib.error.HTTPError as error: + code = ("rate_limited" if error.code == 429 or error.code == 403 and + (error.headers.get("X-RateLimit-Remaining") == "0" or error.headers.get("Retry-After")) + else "not_found" if error.code == 404 else "http_error") + error.close() + raise UpdateError(code) + except urllib.error.URLError as error: + if isinstance(error.reason, ssl.SSLError): + raise UpdateError("tls_error") + if isinstance(error.reason, (TimeoutError, socket.timeout)): + raise UpdateError("timeout") + raise UpdateError("offline") + except ssl.SSLError: + raise UpdateError("tls_error") + except (TimeoutError, socket.timeout): + raise UpdateError("timeout") + except OSError: + raise UpdateError("offline") + + +def fetch_bytes(url, proxy=None, limit=2097152, deadline=None): + validate_update_url(url) + validate_proxy(proxy) + remaining = (deadline if deadline is not None else time.monotonic() + UPDATE_TIMEOUT) - time.monotonic() + if remaining <= 0: + raise UpdateError("timeout") + try: + result = subprocess.run([sys.executable, str(pathlib.Path(__file__).resolve()), "_fetch-update"], + input=json.dumps({"url": url, "proxy": proxy, "limit": limit}).encode(), + stdout=subprocess.PIPE, stderr=subprocess.DEVNULL, timeout=remaining, check=False) + reply = json.loads(result.stdout) + if not isinstance(reply, dict) or result.returncode: + raise UpdateError("invalid_response") + if "error" in reply: + raise UpdateError(reply["error"]) + raw = base64.b64decode(reply["data"], validate=True) + if len(raw) > limit: + raise UpdateError("response_too_large") + return raw + except subprocess.TimeoutExpired: + raise UpdateError("timeout") + except (OSError, ValueError, TypeError, KeyError): + raise UpdateError("invalid_response") + + +def parse_release_json(raw): + def unique(pairs): + result = {} + for key, value in pairs: + if key in result: + raise ValueError("duplicate_key") + result[key] = value + return result + try: + return json.loads(raw, object_pairs_hook=unique) + except (ValueError, TypeError, UnicodeError, RecursionError): + raise UpdateError("invalid_response") + + +def release_metadata(metadata, tag=None): + if not isinstance(metadata, dict): + raise UpdateError("invalid_metadata") + version_tag = metadata.get("tag_name") + if not isinstance(version_tag, str) or not version_tag.startswith("v"): + raise UpdateError("invalid_metadata") + stable_version(version_tag[1:]) + if (tag is not None and version_tag != tag or metadata.get("draft") is not False + or metadata.get("prerelease") is not False + or metadata.get("html_url") != UPDATE_WEB + "/releases/tag/" + version_tag + or type(metadata.get("id")) is not int or metadata["id"] <= 0 + or metadata.get("url") != UPDATE_API + "/" + str(metadata["id"])): + raise UpdateError("invalid_metadata") + assets = metadata.get("assets") + if not isinstance(assets, list) or len(assets) > 100: + raise UpdateError("invalid_metadata") + selected = {} + for asset in assets: + if not isinstance(asset, dict): + raise UpdateError("invalid_metadata") + name = asset.get("name") + if name not in tuple(CORE_FILES) + (MANIFEST,): + continue + if name in selected: + raise UpdateError("invalid_metadata") + size_limit = 16384 if name == MANIFEST else 8 * 1024 * 1024 + expected = UPDATE_WEB + "/releases/download/" + version_tag + "/" + name + if (asset.get("browser_download_url") != expected or asset.get("state") != "uploaded" + or type(asset.get("size")) is not int or not 0 < asset["size"] <= size_limit): + raise UpdateError("invalid_metadata") + selected[name] = {"url": expected, "size": asset["size"]} + if set(selected) != set(CORE_FILES) | {MANIFEST}: + raise UpdateError("unsupported_release") + return {"tag": version_tag, "version": version_tag[1:], "assets": selected} + + +def download_release_asset(release, name, proxy=None, deadline=None): + if name not in tuple(CORE_FILES) + (MANIFEST,): + raise UpdateError("invalid_metadata") + stable_version(release["version"]) + if release["tag"] != "v" + release["version"]: + raise UpdateError("invalid_metadata") + asset = release["assets"][name] + expected = UPDATE_WEB + "/releases/download/" + release["tag"] + "/" + name + if asset["url"] != expected: + raise UpdateError("invalid_metadata") + raw = fetch_bytes(expected, proxy, 16384 if name == MANIFEST else 8 * 1024 * 1024, deadline) + if len(raw) != asset["size"]: + raise UpdateError("invalid_metadata") + return raw + + +def fetch_release(tag=None, proxy=None, timeout=UPDATE_TIMEOUT): + validate_proxy(proxy) + deadline = time.monotonic() + timeout + if tag is not None: + if not isinstance(tag, str) or not tag.startswith("v"): + raise UpdateError("invalid_version") + stable_version(tag[1:]) + raw = fetch_bytes(UPDATE_API + "/tags/" + tag, proxy, deadline=deadline) + release = release_metadata(parse_release_json(raw), tag) + else: + # GitHub's latest endpoint excludes draft/prerelease entries server-side. + raw = fetch_bytes(UPDATE_API + "/latest", proxy, deadline=deadline) + release = release_metadata(parse_release_json(raw)) + raw_manifest = download_release_asset(release, MANIFEST, proxy, deadline) + manifest = parse_release_json(raw_manifest) + if (not isinstance(manifest, dict) or set(manifest) != {"schema_version", "package_version", "files"} + or type(manifest["schema_version"]) is not int or manifest["schema_version"] != 1 + or manifest["package_version"] != release["version"] or not isinstance(manifest["files"], dict) + or set(manifest["files"]) != set(CORE_FILES) + or any(not isinstance(value, str) or not re.fullmatch(r"[0-9a-f]{64}", value) + for value in manifest["files"].values())): + raise UpdateError("invalid_metadata") + release["manifest"] = manifest + return release + + +def update_check(directory, tag=None, proxy=None): + package = package_report(directory) + report = {"status": "check_failed", "selection": "pinned" if tag else "latest", "package": package, + "release": None, "error": None} + try: + release = fetch_release(tag, proxy) + report["release"] = {"tag": release["tag"], "version": release["version"]} + if package["integrity"] != "match": + report["status"] = "not_comparable" + else: + try: + installed = stable_version(package["declared_version"]) + except UpdateError: + report["status"] = "not_comparable" + else: + candidate = stable_version(release["version"]) + report["status"] = "update_available" if candidate > installed else "local_newer" if candidate < installed else "current" + except UpdateError as error: + report["error"] = error.code + return report + + +def update_check_exit(report): + return 2 if report["status"] == "check_failed" else 1 if report["status"] == "not_comparable" else 0 + + +NOTICE_DIRECTORY = ".update-notifications" + + +class NoticeError(Exception): + def __init__(self, code): + self.code = code if code in ("storage_unavailable", "invalid_settings", "invalid_cache") else "storage_unavailable" + super().__init__(self.code) + + +@contextlib.contextmanager +def notice_store(directory, create=False): + """Owned private storage, nonblocking lock, and fixed file names only.""" + root = folder = lock = None + try: + root = os.open(str(directory), os.O_RDONLY | os.O_DIRECTORY | os.O_NOFOLLOW) + info = os.fstat(root) + if info.st_uid != os.getuid() or info.st_mode & 0o022: + raise NoticeError("storage_unavailable") + if create: + try: + os.mkdir(NOTICE_DIRECTORY, 0o700, dir_fd=root) + except FileExistsError: + pass + try: + folder = os.open(NOTICE_DIRECTORY, os.O_RDONLY | os.O_DIRECTORY | os.O_NOFOLLOW, dir_fd=root) + except FileNotFoundError: + yield None + return + info = os.fstat(folder) + if info.st_uid != os.getuid() or stat.S_IMODE(info.st_mode) != 0o700: + raise NoticeError("storage_unavailable") + lock = os.open(".lock", os.O_RDWR | os.O_CREAT | os.O_NOFOLLOW | os.O_NONBLOCK, 0o600, dir_fd=folder) + info = os.fstat(lock) + if (not stat.S_ISREG(info.st_mode) or info.st_uid != os.getuid() + or stat.S_IMODE(info.st_mode) != 0o600 or info.st_nlink != 1): + raise NoticeError("storage_unavailable") + fcntl.flock(lock, fcntl.LOCK_EX | fcntl.LOCK_NB) + yield folder + except OSError: + raise NoticeError("storage_unavailable") + finally: + for descriptor in (lock, folder, root): + if descriptor is not None: + os.close(descriptor) + + +def notice_read(folder, name): + if folder is None: + return None + try: + descriptor = os.open(name, os.O_RDONLY | os.O_NOFOLLOW | os.O_NONBLOCK, dir_fd=folder) + except FileNotFoundError: + return None + with os.fdopen(descriptor, "rb") as stream: + info = os.fstat(stream.fileno()) + if (not stat.S_ISREG(info.st_mode) or info.st_uid != os.getuid() + or stat.S_IMODE(info.st_mode) != 0o600 or info.st_nlink != 1): + raise NoticeError("storage_unavailable") + raw = stream.read(16385) + if len(raw) > 16384: + raise NoticeError("invalid_settings" if name == "settings.json" else "invalid_cache") + try: + return parse_release_json(raw) + except UpdateError: + raise NoticeError("invalid_settings" if name == "settings.json" else "invalid_cache") + + +def notice_write(folder, name, data): + # Check existing files before replacing them; never follow a link or share an inode. + notice_read(folder, name) + temporary = ".write-" + os.urandom(8).hex() + try: + descriptor = os.open(temporary, os.O_WRONLY | os.O_CREAT | os.O_EXCL | os.O_NOFOLLOW, 0o600, dir_fd=folder) + with os.fdopen(descriptor, "wb") as stream: + stream.write((json.dumps(data, sort_keys=True) + "\n").encode()) + stream.flush() + os.fsync(stream.fileno()) + os.replace(temporary, name, src_dir_fd=folder, dst_dir_fd=folder) + os.fsync(folder) + finally: + try: + os.unlink(temporary, dir_fd=folder) + except FileNotFoundError: + pass + + +def notice_settings(folder): + settings = notice_read(folder, "settings.json") + if settings is None: + return {"enabled": False, "interval_hours": 24} + if (not isinstance(settings, dict) or set(settings) != {"enabled", "interval_hours"} + or type(settings["enabled"]) is not bool or type(settings["interval_hours"]) is not int + or not 1 <= settings["interval_hours"] <= 720): + raise NoticeError("invalid_settings") + return settings + + +def notice_cache(folder, now): + cache = notice_read(folder, "cache.json") + if cache is None: + return {"checked_at": None, "last_attempt_at": None, "last_error": None, + "release": None, "last_notified": None} + if not isinstance(cache, dict) or set(cache) != {"checked_at", "last_attempt_at", "last_error", "release", "last_notified"}: + raise NoticeError("invalid_cache") + for name in ("checked_at", "last_attempt_at"): + value = cache[name] + if value is not None and (type(value) is not int or not 0 <= value <= now + 300): + raise NoticeError("invalid_cache") + if cache["last_error"] is not None and (not isinstance(cache["last_error"], str) or cache["last_error"] not in UPDATE_ERRORS): + raise NoticeError("invalid_cache") + release = cache["release"] + try: + if release is not None: + if not isinstance(release, dict) or set(release) != {"tag", "version"}: + raise NoticeError("invalid_cache") + stable_version(release["version"]) + if release["tag"] != "v" + release["version"] or cache["checked_at"] is None: + raise NoticeError("invalid_cache") + if cache["last_notified"] is not None: + if not isinstance(cache["last_notified"], str) or not cache["last_notified"].startswith("v"): + raise NoticeError("invalid_cache") + stable_version(cache["last_notified"][1:]) + except UpdateError: + raise NoticeError("invalid_cache") + return cache + + +def notice_summary(settings, cache, now): + return dict(settings, **cache, stale=(cache["checked_at"] is None or cache["last_error"] is not None + or now - cache["checked_at"] >= settings["interval_hours"] * 3600)) + + +def notification_configure(directory, enabled=None, interval_hours=None): + if enabled is None and interval_hours is None: + raise NoticeError("invalid_settings") + if interval_hours is not None and (type(interval_hours) is not int or not 1 <= interval_hours <= 720): + raise NoticeError("invalid_settings") + with notice_store(directory, create=True) as folder: + settings = notice_settings(folder) + if enabled is not None: + settings["enabled"] = enabled + if interval_hours is not None: + settings["interval_hours"] = interval_hours + notice_write(folder, "settings.json", settings) + return settings + + +def notification_show(directory): + with notice_store(directory) as folder: + now = int(time.time()) + return notice_summary(notice_settings(folder), notice_cache(folder, now), now) + + +def notification_refresh(directory, proxy=None, force=False): + # Explicit only; the startup hook never calls this or any network operation. + with notice_store(directory) as folder: + now = int(time.time()) + settings, cache = notice_settings(folder), notice_cache(folder, now) + report = notice_summary(settings, cache, now) + if not settings["enabled"]: + return dict(report, status="disabled") + if not force and not report["stale"]: + return dict(report, status="cached") + try: + release = fetch_release(proxy=proxy) + cache.update(checked_at=int(time.time()), last_attempt_at=int(time.time()), last_error=None, + release={"tag": release["tag"], "version": release["version"]}) + status = "refreshed" + except UpdateError as error: + cache.update(last_attempt_at=int(time.time()), last_error=error.code) + status = "refresh_failed" + notice_write(folder, "cache.json", cache) + return dict(notice_summary(settings, cache, int(time.time())), status=status) + + +def notification_notify(directory): + """Best-effort, cache-only local stderr notice; never changes a command result.""" + if not sys.stderr.isatty(): + return + try: + with notice_store(directory) as folder: + now = int(time.time()) + settings, cache = notice_settings(folder), notice_cache(folder, now) + summary = notice_summary(settings, cache, now) + if not settings["enabled"] or summary["stale"] or cache["release"] is None: + return + release = cache["release"] + if cache["last_notified"] == release["tag"]: + return + package = package_report(directory) + if package["integrity"] != "match" or stable_version(package["declared_version"]) >= stable_version(release["version"]): + return + cache["last_notified"] = release["tag"] + notice_write(folder, "cache.json", cache) + print("dots-ts: stable release " + release["tag"] + " is available; run dots-ts update check before updating.", file=sys.stderr) + except (NoticeError, UpdateError, OSError): + return + + +# Manual updates deliberately restart only the transport shim. Tailscale binaries, +# authentication state and SSH configuration are outside this transaction. +APPLY_DIRECTORY = '.update-transactions' +APPLY_TERMINAL = {'applied', 'failed_rolled_back', 'rollback_failed_manual_required', 'rejected'} + + +class ApplyError(Exception): + pass + + +def apply_private(path, create=False): + path = pathlib.Path(path) + for parent in (path,) + tuple(path.parents): + if parent.is_symlink(): + raise ApplyError('unsafe_directory') + if create: + path.mkdir(mode=0o700, exist_ok=True) + parent_fd = os.open(str(path.parent), os.O_RDONLY | os.O_DIRECTORY) + try: os.fsync(parent_fd) + finally: os.close(parent_fd) + info = path.stat() + if not stat.S_ISDIR(info.st_mode) or info.st_uid != os.getuid() or stat.S_IMODE(info.st_mode) != 0o700: + raise ApplyError('unsafe_directory') + return path + + +def apply_write(path, data, mode=0o600): + temporary = path.with_name(path.name + '.new') + fd = os.open(str(temporary), os.O_WRONLY | os.O_CREAT | os.O_EXCL | os.O_NOFOLLOW, mode) + try: + with os.fdopen(fd, 'wb') as stream: + stream.write(data); stream.flush(); os.fsync(stream.fileno()) + os.replace(temporary, path) + fd = os.open(str(path.parent), os.O_RDONLY | os.O_DIRECTORY) + try: os.fsync(fd) + finally: os.close(fd) + finally: + if temporary.exists(): temporary.unlink() + + +def apply_journal(transaction, journal, **changes): + journal.update(changes, updated_at=int(time.time())) + apply_write(transaction / 'journal.json', json.dumps(journal, sort_keys=True).encode()) + + +def apply_read_journal(transaction): + try: + value = parse_release_json(bounded_code_read(transaction, 'journal.json', 65536)) + states = APPLY_TERMINAL | {'prepared', 'running'} + if (not isinstance(value, dict) or type(value.get('schema')) is not int or value['schema'] != 1 + or not isinstance(value.get('state'), str) or value['state'] not in states): + raise ValueError() + for field in ('phase', 'boot_id', 'error'): + if field in value and (not isinstance(value[field], str) or len(value[field]) > 512): raise ValueError() + worker = value.get('worker') + if worker is not None: + if (not isinstance(worker, dict) or set(worker) != {'pid', 'ticks', 'uid', 'exe'} + or type(worker['pid']) is not int or worker['pid'] <= 0 + or type(worker['uid']) is not int or worker['uid'] != os.getuid() + or not isinstance(worker['ticks'], str) or not re.fullmatch('[0-9]{1,30}', worker['ticks']) + or not isinstance(worker['exe'], str) or not worker['exe'].startswith('/') or len(worker['exe']) > 4096): + raise ValueError() + return value + except (OSError, ValueError, TypeError, UpdateError): + raise ApplyError('invalid_journal') + + +def apply_identity(pid): + """None means definitely exited; inaccessible /proc remains an error.""" + p = pathlib.Path('/proc') / str(pid) + try: + fields = (p / 'stat').read_text().rsplit(')', 1)[1].split() + if fields[0] == 'Z': return None + return {'pid': int(pid), 'ticks': fields[19], 'uid': p.stat().st_uid, + 'exe': os.readlink(p / 'exe')} + except FileNotFoundError: + if p.exists(): raise ApplyError('process_identity_unknown') + return None + except (OSError, IndexError, ValueError): + raise ApplyError('process_identity_unknown') + + +def apply_boot(): + return pathlib.Path('/proc/sys/kernel/random/boot_id').read_text().strip() + + +def apply_status(directory, identifier): + if not re.fullmatch('[0-9a-f]{32}', identifier): raise ApplyError('invalid_transaction_id') + root = apply_private(directory / APPLY_DIRECTORY) + transaction = apply_private(root / identifier) + journal = apply_read_journal(transaction) + if journal.get('state') not in APPLY_TERMINAL: + worker = journal.get('worker') + try: + alive = journal.get('boot_id') == apply_boot() and worker and apply_identity(worker['pid']) == worker + except (OSError, ApplyError, KeyError): alive = False + if not alive: + journal = dict(journal, state='interrupted_manual_required') + return dict(journal, transaction_id=identifier, backup_directory=str(transaction / 'backup'), + external_ssh='pending', recovery='Use the independent recovery terminal; inspect this journal and code-only backup. Do not rerun an interrupted update blindly.') + + +def apply_bundle(directory, layout='source'): + report = package_report(directory, layout) + if report['integrity'] != 'match': raise ApplyError('untrusted_or_mixed_package') + payload = {name: bounded_code_read(directory, name if layout == 'source' else installed, 8 * 1024 * 1024) + for name, installed in CORE_FILES.items()} + payload[MANIFEST] = bounded_code_read(directory, MANIFEST, 16384) + # Verify the bytes actually captured, not only an earlier pathname read. + manifest = parse_release_json(payload[MANIFEST]) + if any(hashlib.sha256(payload[name]).hexdigest() != manifest['files'][name] for name in CORE_FILES): + raise ApplyError('package_changed_during_read') + return payload + + +def apply_syntax(payload): + for name in ('dots-ts', 'dots_proxy.py', 'shim.py'): + try: compile(payload[name], name, 'exec') + except (SyntaxError, ValueError): raise ApplyError('invalid_python_syntax') + try: + result = subprocess.run(['bash', '-n'], input=payload['tailscale-setup.sh'], + stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, timeout=5) + except subprocess.TimeoutExpired: raise ApplyError('shell_syntax_timeout') + if result.returncode: raise ApplyError('invalid_shell_syntax') + + +def apply_ts_version(payload): + matches = re.findall(rb'^TS_VERSION=(?:"([0-9]+\.[0-9]+\.[0-9]+)"|([0-9]+\.[0-9]+\.[0-9]+))$', payload['tailscale-setup.sh'], re.M) + if len(matches) != 1: raise ApplyError('unsupported_binary_version_declaration') + return matches[0][0] or matches[0][1] + + +def apply_components(directory, runtime, proxy): + matches = {'shim': [], 'sshd': []} + for entry in pathlib.Path('/proc').iterdir(): + if not entry.name.isdigit(): continue + try: + if entry.stat().st_uid != os.getuid(): continue + argv = (entry / 'cmdline').read_bytes().rstrip(b'\0').decode('utf8', 'surrogateescape').split('\0') + executable = os.readlink(entry / 'exe') + except FileNotFoundError: continue + except PermissionError: + # An unreadable unrelated process is not evidence for any target. + continue + kind = None + if re.fullmatch(r'python(?:3(?:\.\d+)?)?', pathlib.Path(executable).name) and script_operand(argv) == str(directory / 'shim/shim.py'): + kind = 'shim' + if pathlib.Path(executable).name == 'sshd' and ssh_config(argv) == str(directory / 'sshd/sshd_config'): + kind = 'sshd' + if kind: + identity = apply_identity(int(entry.name)) + if identity: + if kind == 'shim' and argv != [executable, str(directory / 'shim/shim.py')]: + # Interpreter argv[0] may be a PATH alias; options are unsupported. + if len(argv) != 2 or argv[1] != str(directory / 'shim/shim.py'): + raise ApplyError('unsupported_shim_arguments') + matches[kind].append(identity) + if any(len(value) != 1 for value in matches.values()): raise ApplyError('ambiguous_component_identity') + result = {key: value[0] for key, value in matches.items()} + daemon = proxy._daemon(directory, runtime) + if not daemon or not proxy._socket_owned(daemon, runtime): raise ApplyError('daemon_socket_identity_unknown') + if ([arg for arg in daemon['argv'][1:] if arg == '--statedir' or arg.startswith('--statedir=')] != ['--statedir=' + str(directory / 'state')] + or any(arg == '--state' or arg.startswith('--state=') for arg in daemon['argv'][1:])): + raise ApplyError('unexpected_daemon_state_directory') + result['_socket'] = proxy._socket_identity(runtime) + result['tailscaled'] = apply_identity(daemon['pid']) + if not result['tailscaled']: raise ApplyError('daemon_exited') + for name, port in (('shim', 18089), ('sshd', 22)): + if not proxy._listener_owned(result[name], port): raise ApplyError('listener_identity_unknown') + return result + + +def apply_public_identity(directory, runtime, timeout=3): + reply = run_cli(directory, runtime, ['status', '--json'], timeout=timeout) + if reply.returncode: raise ApplyError('backend_unavailable') + data = json.loads(reply.stdout) + if data.get('BackendState') != 'Running': raise ApplyError('backend_not_running') + own = data.get('Self') or {} + if not own.get('ID') or not own.get('PublicKey') or not own.get('TailscaleIPs'): + raise ApplyError('public_identity_unavailable') + return hashlib.sha256(json.dumps({key: own[key] for key in ('ID', 'PublicKey', 'TailscaleIPs')}, sort_keys=True).encode()).hexdigest() + + +def apply_environment(process): + if apply_identity(process['pid']) != process: raise ApplyError('shim_identity_changed') + try: + values = dict(item.split(b'=', 1) for item in pathlib.Path('/proc/{}/environ'.format(process['pid'])).read_bytes().split(b'\0') if item) + environment = {'PATH': '/usr/bin:/bin', 'PYTHONNOUSERSITE': '1', 'PYTHONDONTWRITEBYTECODE': '1'} + for key, default in (('UPSTREAM_HOST', 'browser-proxy'), ('UPSTREAM_PORT', '8889'), ('SHIM_PORT', '18089')): + environment[key] = values.get(key.encode(), default.encode()).decode('ascii') + if not re.fullmatch(r'[A-Za-z0-9._:-]{1,253}', environment['UPSTREAM_HOST']): raise ValueError() + if not 1 <= int(environment['UPSTREAM_PORT']) <= 65535 or environment['SHIM_PORT'] != '18089': raise ValueError() + except (OSError, UnicodeError, ValueError): raise ApplyError('unsupported_shim_environment') + if apply_identity(process['pid']) != process: raise ApplyError('shim_identity_changed') + return environment + + +def apply_stop(process, timeout=8): + import signal + current = apply_identity(process['pid']) + if current is None: return + if current != process: raise ApplyError('process_identity_changed') + fd = os.pidfd_open(process['pid']) + try: + if apply_identity(process['pid']) != process: raise ApplyError('process_identity_changed') + signal.pidfd_send_signal(fd, signal.SIGTERM) + deadline = time.monotonic() + timeout + while time.monotonic() < deadline: + current = apply_identity(process['pid']) + if current is None: return + if current != process: raise ApplyError('process_identity_changed') + time.sleep(.1) + raise ApplyError('component_stop_timeout') + finally: os.close(fd) + + +def apply_launch(directory, baseline, environment, log): + child = subprocess.Popen([baseline['shim']['exe'], str(directory / 'shim/shim.py')], + env=environment, stdin=subprocess.DEVNULL, stdout=log, stderr=log, + close_fds=True, start_new_session=True) + return child + + +def apply_candidate(child, baseline): + identity = apply_identity(child.pid) + if identity is None: + child.wait(timeout=1) + return {'pid': child.pid, 'ticks': 'exited', 'uid': os.getuid(), 'exe': baseline['shim']['exe']} + return identity + + +def apply_ready(directory, runtime, proxy, baseline, candidate, public, config, host_public, timeout=30): + if candidate['exe'] != baseline['shim']['exe']: raise ApplyError('candidate_executable_changed') + deadline = time.monotonic() + timeout + while time.monotonic() < deadline: + if apply_identity(candidate['pid']) != candidate: raise ApplyError('candidate_identity_changed') + if '_socket' in baseline and proxy._socket_identity(runtime) != baseline['_socket']: + raise ApplyError('daemon_socket_changed') + for name in ('tailscaled', 'sshd'): + if apply_identity(baseline[name]['pid']) != baseline[name]: raise ApplyError('preserved_process_changed') + try: + if apply_public_identity(directory, runtime, min(3, max(.01, deadline-time.monotonic()))) != public: raise ApplyError('node_identity_changed') + if proxy.inspect_config(directory, runtime) != config: raise ApplyError('proxy_configuration_changed') + if bounded_code_read(directory, 'sshd/ssh_host_ed25519_key.pub', 16384) != host_public: raise ApplyError('host_public_key_changed') + if not proxy._listener_owned(candidate, 18089): raise ApplyError('shim_listener_not_ready') + with socket.create_connection(('127.0.0.1', 22), timeout=min(2, max(.1, deadline-time.monotonic()))) as connection: + connection.settimeout(min(2, max(.01, deadline-time.monotonic()))) + if not connection.recv(256).startswith(b'SSH-'): raise ApplyError('ssh_not_ready') + return + except ApplyError as error: + if str(error) not in ('backend_not_running', 'backend_unavailable', 'public_identity_unavailable', 'shim_listener_not_ready', 'ssh_not_ready'): raise + except (OSError, subprocess.SubprocessError): pass + time.sleep(min(.2, max(0, deadline-time.monotonic()))) + raise ApplyError('readiness_timeout') + + +def apply_replace(directory, payload, transaction, journal, phase): + for source, installed in list(CORE_FILES.items()) + [(MANIFEST, MANIFEST)]: + apply_journal(transaction, journal, phase=phase, file=source) + target = directory / installed + if target.is_symlink() or not target.is_file(): raise ApplyError('unsafe_installed_file') + apply_write(target, payload[source], 0o600 if source == MANIFEST else 0o700) + apply_journal(transaction, journal, completed_file=source) + + +def apply_transaction(directory, runtime, transaction, journal, proxy): + old = apply_bundle(directory, 'installed') + candidate_payload = apply_bundle(transaction / 'stage') + apply_syntax(candidate_payload) + if apply_ts_version(old) != apply_ts_version(candidate_payload): raise ApplyError('binary_update_unsupported') + baseline = apply_components(directory, runtime, proxy) + environment = apply_environment(baseline['shim']) + public = apply_public_identity(directory, runtime) + try: host_public = bounded_code_read(directory, 'sshd/ssh_host_ed25519_key.pub', 16384) + except (OSError, ValueError): raise ApplyError('public_ssh_host_key_required') + config = proxy.inspect_config(directory, runtime) + if config.get('restart_required') or not config.get('daemon_running'): raise ApplyError('proxy_state_pending') + backup = apply_private(transaction / 'backup', create=True) + for name, data in old.items(): apply_write(backup / name, data) + # The shim is long lived; discard output rather than leave an unbounded log. + logfd = os.open(os.devnull, os.O_WRONLY) + child = None; candidate = None; stopped = False; mutation = False + try: + apply_journal(transaction, journal, phase='stop_old_shim', destructive=True, baseline=baseline) + apply_stop(baseline['shim']); stopped = True + apply_journal(transaction, journal, phase='replace_intent') + mutation = True + apply_replace(directory, candidate_payload, transaction, journal, 'replace') + apply_journal(transaction, journal, phase='launch_candidate') + child = apply_launch(directory, baseline, environment, logfd) + candidate = apply_candidate(child, baseline) + apply_journal(transaction, journal, candidate=candidate, phase='readiness') + apply_ready(directory, runtime, proxy, baseline, candidate, public, config, host_public) + if apply_bundle(directory, 'installed') != candidate_payload: raise ApplyError('installed_verification_failed') + apply_journal(transaction, journal, state='applied', phase='complete') + except Exception as failure: + apply_journal(transaction, journal, error=str(failure) if isinstance(failure, ApplyError) else 'transaction_failed', phase='rollback_intent') + try: + if child is not None: + if candidate is None: + if child.poll() is None: raise ApplyError('candidate_identity_unknown') + else: apply_stop(candidate) + child.wait(timeout=2) + if not stopped: + # Failed stop is not permission to start a duplicate or alter code. + raise ApplyError('old_shim_stop_uncertain') + if mutation: apply_replace(directory, old, transaction, journal, 'restore') + restored = apply_launch(directory, baseline, environment, logfd) + identity = apply_candidate(restored, baseline) + apply_journal(transaction, journal, restored=identity, phase='rollback_readiness') + apply_ready(directory, runtime, proxy, baseline, identity, public, config, host_public) + if apply_bundle(directory, 'installed') != old: raise ApplyError('rollback_verification_failed') + apply_journal(transaction, journal, state='failed_rolled_back', phase='complete') + except Exception: + apply_journal(transaction, journal, state='rollback_failed_manual_required', phase='manual_recovery') + finally: os.close(logfd) + + +def apply_worker(directory, runtime, transaction): + import signal + import dots_proxy as proxy + apply_private(directory); apply_private(transaction.parent); apply_private(transaction) + if transaction.parent != directory / APPLY_DIRECTORY: raise ApplyError('invalid_transaction_path') + journal = apply_read_journal(transaction) + if journal.get('state') != 'prepared' or journal.get('boot_id') != apply_boot(): raise ApplyError('invalid_worker_state') + interrupted = [] + for signum in (signal.SIGTERM, signal.SIGHUP, signal.SIGINT): + signal.signal(signum, lambda number, frame: interrupted.append(number) if not interrupted else None) + try: + with proxy._operation(directory): + apply_journal(transaction, journal, state='running', worker=apply_identity(os.getpid()), phase='preflight') + apply_transaction(directory, runtime, transaction, journal, proxy) + except Exception as error: + apply_journal(transaction, journal, state='rollback_failed_manual_required' if journal.get('destructive') else 'rejected', error=str(error) if isinstance(error, ApplyError) else 'worker_failed') + if interrupted: apply_journal(transaction, journal, interrupted_signal=interrupted[0]) + return 0 + + +def update_apply(directory, runtime, tag=None, bundle=None, confirm=False, proxy_url=None): + import signal + import uuid + import dots_proxy as proxy + if not confirm: raise ApplyError('independent_recovery_terminal_confirmation_required') + if sys.platform != 'linux' or not hasattr(os, 'pidfd_open') or not hasattr(signal, 'pidfd_send_signal'): + raise ApplyError('linux_python_pidfd_required') + if bool(tag) == bool(bundle): raise ApplyError('select_exactly_one_tag_or_bundle') + apply_private(directory) + caller = pathlib.Path(__file__).resolve().parent + caller_layout = 'source' if (caller / 'tailscale-setup.sh').is_file() else 'installed' + trusted = apply_bundle(caller, caller_layout) + with proxy._operation(directory): + baseline = apply_bundle(directory, 'installed') + root = apply_private(directory / APPLY_DIRECTORY, create=True) + for entry in root.iterdir(): + if not re.fullmatch('[0-9a-f]{32}', entry.name): raise ApplyError('unexpected_transaction_entry') + previous = apply_read_journal(apply_private(entry)) + if previous.get('state') not in ('applied', 'failed_rolled_back', 'rejected'): + raise ApplyError('prior_transaction_requires_manual_review') + if bundle: + payload = apply_bundle(pathlib.Path(bundle), 'source') + else: + release = fetch_release(tag, proxy_url) + deadline = time.monotonic() + 60 + payload = {name: download_release_asset(release, name, proxy_url, deadline) for name in CORE_FILES} + payload[MANIFEST] = json.dumps(release['manifest'], sort_keys=True).encode() + apply_syntax(payload) + if apply_ts_version(payload) != apply_ts_version(baseline): raise ApplyError('binary_update_unsupported') + identifier = uuid.uuid4().hex + transaction = apply_private(root / identifier, create=True) + journal = {'schema': 1, 'state': 'prepared', 'boot_id': apply_boot(), 'selection': 'local_bundle' if bundle else tag, + 'external_ssh': 'pending', 'checksum_trust': 'consistency_only_not_publisher_authentication'} + apply_journal(transaction, journal, phase='staging') + try: + stage = apply_private(transaction / 'stage', create=True) + for name, data in payload.items(): apply_write(stage / name, data) + apply_bundle(stage) + supervisor = apply_private(transaction / 'supervisor', create=True) + for name in ('dots-ts', 'dots_proxy.py'): apply_write(supervisor / name, trusted[name], 0o700) + logfd = os.open(str(transaction / 'worker.log'), os.O_WRONLY | os.O_CREAT | os.O_EXCL | os.O_NOFOLLOW, 0o600) + os.close(logfd) + except Exception: + apply_journal(transaction, journal, state='rejected', error='staging_failed') + raise + log_fd = os.open(str(transaction / 'worker.log'), os.O_WRONLY | os.O_APPEND | os.O_NOFOLLOW | os.O_NONBLOCK) + info = os.fstat(log_fd) + if not stat.S_ISREG(info.st_mode) or info.st_uid != os.getuid() or stat.S_IMODE(info.st_mode) != 0o600: + os.close(log_fd); raise ApplyError('unsafe_worker_log') + with os.fdopen(log_fd, 'ab') as log: + child = subprocess.Popen([sys.executable, str(supervisor / 'dots-ts'), '_update-worker', str(directory), str(runtime), str(transaction)], + stdin=subprocess.DEVNULL, stdout=log, stderr=log, close_fds=True, start_new_session=True) + deadline = time.monotonic() + 10 + while time.monotonic() < deadline: + current = apply_read_journal(transaction) + if current.get('worker') or current.get('state') in APPLY_TERMINAL: + return {'state': 'started' if current.get('worker') else current['state'], 'transaction_id': identifier, + 'external_ssh': 'pending', 'status_command': 'dots-ts update status --id ' + identifier} + if child.poll() is not None: raise ApplyError('worker_exited_inspect_transaction_' + identifier) + time.sleep(.05) + raise ApplyError('worker_acknowledgement_timeout_inspect_transaction_' + identifier) + + def directories(): directory = pathlib.Path(os.environ.get("DOTS_TS_DIR", str(pathlib.Path(__file__).resolve().parent))) if not directory.is_absolute(): @@ -488,6 +1348,20 @@ class SafeParser(argparse.ArgumentParser): def main(argv=None): argv = sys.argv[1:] if argv is None else argv try: + if argv and argv[0] == '_update-worker': + if len(argv) != 4: raise ValueError('invalid_worker_arguments') + return apply_worker(*(pathlib.Path(value) for value in argv[1:])) + if argv == ["_fetch-update"]: + try: + request = parse_release_json(sys.stdin.buffer.read(16385)) + if (not isinstance(request, dict) or set(request) != {"url", "proxy", "limit"} + or type(request["limit"]) is not int or not 1 <= request["limit"] <= 8 * 1024 * 1024): + raise UpdateError("invalid_response") + raw = _network_fetch(request["url"], request["proxy"], request["limit"]) + print(json.dumps({"data": base64.b64encode(raw).decode("ascii")})) + except UpdateError as error: + print(json.dumps({"error": error.code})) + return 0 if argv and argv[0] == "_verify-package": if len(argv) != 4 or argv[2] not in ("source", "installed"): raise ValueError("invalid_arguments") @@ -510,12 +1384,80 @@ def main(argv=None): return 1 parser = SafeParser(description="Local CLI forwarding and read-only diagnostics.") commands = parser.add_subparsers(dest="command", required=True) + update = commands.add_parser("update", help="check stable releases without changing code or services") + update_commands = update.add_subparsers(dest="update_command", required=True) + check = update_commands.add_parser("check") + check.add_argument("--tag") + check.add_argument("--proxy") + check.add_argument("--json", action="store_true") + apply = update_commands.add_parser('apply', help='verified detached manual update; requires independent recovery terminal') + selection = apply.add_mutually_exclusive_group(required=True) + selection.add_argument('--tag') + selection.add_argument('--bundle', type=pathlib.Path) + apply.add_argument('--confirm-recovery-terminal', action='store_true') + apply.add_argument('--proxy') + apply.add_argument('--json', action='store_true') + status = update_commands.add_parser('status') + status.add_argument('--id', required=True) + status.add_argument('--json', action='store_true') + notices = update_commands.add_parser("notifications", help="opt-in local cached notices; no scheduler") + notice_commands = notices.add_subparsers(dest="notice_command", required=True) + configure = notice_commands.add_parser("configure") + enabled = configure.add_mutually_exclusive_group() + enabled.add_argument("--enable", dest="enabled", action="store_const", const=True, default=None) + enabled.add_argument("--disable", dest="enabled", action="store_const", const=False) + configure.add_argument("--interval-hours", type=int) + configure.add_argument("--json", action="store_true") + show = notice_commands.add_parser("show") + show.add_argument("--json", action="store_true") + refresh = notice_commands.add_parser("refresh") + refresh.add_argument("--proxy") + refresh.add_argument("--force", action="store_true") + refresh.add_argument("--json", action="store_true") + notice_commands.add_parser("notify", help="cache-only; interactive stderr only") version = commands.add_parser("version", help="offline package integrity and local Tailscale version") version.add_argument("--json", action="store_true") diagnostic = commands.add_parser("doctor", help="bounded read-only checks; target probes are TCP only") diagnostic.add_argument("--json", action="store_true") diagnostic.add_argument("--target") options = parser.parse_args(argv) + if options.command == 'update' and options.update_command in ('apply', 'status'): + try: + report = (apply_status(directory, options.id) if options.update_command == 'status' else + update_apply(directory, runtime, options.tag, options.bundle, options.confirm_recovery_terminal, options.proxy)) + except (ApplyError, UpdateError) as error: + report = {'error': str(error)} + print(json.dumps(report, sort_keys=True)) + return int('error' in report or report.get('state') in ('rejected', 'failed_rolled_back', 'rollback_failed_manual_required', 'interrupted_manual_required')) + if options.command == "update" and options.update_command == "notifications": + if options.notice_command == "notify": + notification_notify(directory) + return 0 + try: + if options.notice_command == "configure": + report = notification_configure(directory, options.enabled, options.interval_hours) + elif options.notice_command == "refresh": + report = notification_refresh(directory, options.proxy, options.force) + else: + report = notification_show(directory) + except NoticeError as error: + report = {"error": error.code} + if options.json: + print(json.dumps(report, sort_keys=True)) + else: + print("update notifications: " + json.dumps(report, sort_keys=True)) + return int("error" in report or report.get("status") == "refresh_failed") + if options.command == "update": + report = update_check(directory, options.tag, options.proxy) + if options.json: + print(json.dumps(report, sort_keys=True)) + else: + print("update: " + report["status"]) + if report["release"]: + print("release: " + report["release"]["tag"] + " (" + report["selection"] + ")") + if report["error"]: + print("error: " + report["error"]) + return update_check_exit(report) if options.command == "version": report = {"package": package_report(directory), "tailscale": tailscale_version(directory)} if options.json: diff --git a/source-manifest.json b/source-manifest.json index 39bb1cd..aa388e9 100644 --- a/source-manifest.json +++ b/source-manifest.json @@ -1,9 +1,9 @@ { "files": { - "dots-ts": "69dde01ff73107965f2bdf0b020088386225d59e971f9fa94751b5f9bc447499", + "dots-ts": "b976a21139bec710364317da91a9ae57c6f1c9a3b3ea9b0042a7471d544a29df", "dots_proxy.py": "2b74af2e175d12300196cab008062ba1098e12bc208d617e3fe63c23c2613390", "shim.py": "c0c55c45fbd9126e860b56d0d0359c0c87f8b39ef5d4da4a9560522452ac8c65", - "tailscale-setup.sh": "58fff7a0efbfe43009c00cef3e3b831b04efd44a061ab03fd3784cbc8fc12504" + "tailscale-setup.sh": "79b703df7f1c871a5ba5e081cac19d1e810f0d4886212f461b198573af77830b" }, "package_version": "0.4.0-dev", "schema_version": 1 diff --git a/tailscale-setup.sh b/tailscale-setup.sh index 08e8411..cfa7ce5 100755 --- a/tailscale-setup.sh +++ b/tailscale-setup.sh @@ -420,7 +420,12 @@ case "$state" in printf 'authentication_command_failed: backend is running but the explicit authentication command failed\n' >&2 exit "$AUTH_COMMAND_STATUS" fi - : > "$D/authenticated"; printf 'running: Tailscale and local SSH ready\n';; + : > "$D/authenticated"; printf 'running: Tailscale and local SSH ready\n' + # Opt-in cached notice only. Never let a notice change readiness or stdout. + if [[ "$MODE" = start && -t 2 ]]; then + timeout --kill-after=1s 1s python3 "$D/dots-ts" update notifications notify >/dev/null || true + fi + ;; NeedsLogin) printf 'needs_auth: run bash %s/setup.sh login\n' "$D"; exit 10;; NeedsMachineAuth) printf 'needs_machine_auth: approve this existing device in the Tailscale admin console\n'; exit 13;; *) printf 'unavailable: BackendState=%s\n' "$state"; exit 11;; diff --git a/tests/test_update_apply.py b/tests/test_update_apply.py new file mode 100644 index 0000000..81814e1 --- /dev/null +++ b/tests/test_update_apply.py @@ -0,0 +1,287 @@ +"""Manual updater: offline fault fixtures; never touches a live installation.""" +import contextlib +import hashlib +import importlib.machinery +import importlib.util +import json +import os +from pathlib import Path +import signal +import subprocess +import sys +import tempfile +import unittest +from unittest import mock + +ROOT = Path(__file__).resolve().parents[1] +loader = importlib.machinery.SourceFileLoader('dots_ts_apply_test', str(ROOT / 'dots-ts')) +spec = importlib.util.spec_from_loader(loader.name, loader) +tool = importlib.util.module_from_spec(spec) +loader.exec_module(tool) + + +def fixture(folder, installed=False, version='0.4.0-dev', marker='new'): + files = {'tailscale-setup.sh': b'#!/bin/bash\nTS_VERSION="1.102.4"\n', + 'shim.py': ('# '+marker+'\n').encode(), 'dots-ts': b'# cli\n', 'dots_proxy.py': b'# proxy\n'} + metadata = {'schema_version': 1, 'package_version': version, + 'files': {name: hashlib.sha256(data).hexdigest() for name, data in files.items()}} + files[tool.MANIFEST] = json.dumps(metadata).encode() + for name, data in files.items(): + path = folder / (tool.CORE_FILES.get(name, name) if installed else name) + path.parent.mkdir(mode=0o700, parents=True, exist_ok=True) + path.write_bytes(data) + return files + + +class ApplyTests(unittest.TestCase): + def setUp(self): + self.temp = tempfile.TemporaryDirectory(prefix='codex-update-apply-') + self.addCleanup(self.temp.cleanup) + self.root = Path(self.temp.name).resolve() + self.directory = self.root / 'install'; self.directory.mkdir(mode=0o700) + self.transaction = self.root / 'transaction'; self.transaction.mkdir(mode=0o700) + (self.transaction / 'stage').mkdir(mode=0o700) + self.old = fixture(self.directory, True, '0.3.0', 'old') + self.new = fixture(self.transaction / 'stage') + (self.directory / 'sshd').mkdir() + (self.directory / 'sshd/ssh_host_ed25519_key.pub').write_bytes(b'public-key') + for name in ('state/tailscaled.state', 'sshd/ssh_host_ed25519_key', 'shim/ca.key', 'outbound-proxy.json'): + path = self.directory / name; path.parent.mkdir(exist_ok=True); path.write_bytes(b'DUMMY fixture only '+name.encode()) + self.protected = {name: ((self.directory / name).stat().st_ino, (self.directory / name).read_bytes()) + for name in ('state/tailscaled.state', 'sshd/ssh_host_ed25519_key', 'shim/ca.key', 'outbound-proxy.json')} + self.journal = {'schema': 1, 'state': 'running'} + self.baseline = {name: {'pid': pid, 'ticks': str(pid), 'exe': '/python', 'uid': os.getuid()} + for name, pid in (('shim', 101), ('tailscaled', 102), ('sshd', 103))} + self.candidate = dict(self.baseline['shim'], pid=201, ticks='201') + self.restored = dict(self.baseline['shim'], pid=301, ticks='301') + self.proxy = mock.Mock() + self.proxy.inspect_config.return_value = {'restart_required': False, 'daemon_running': True} + + def lifecycle(self, ready=None): + stack = contextlib.ExitStack(); self.addCleanup(stack.close) + self.stop = stack.enter_context(mock.patch.object(tool, 'apply_stop')) + self.launch = stack.enter_context(mock.patch.object(tool, 'apply_launch', side_effect=[mock.Mock(), mock.Mock()])) + self.identify = stack.enter_context(mock.patch.object(tool, 'apply_candidate', side_effect=[self.candidate, self.restored])) + self.ready = stack.enter_context(mock.patch.object(tool, 'apply_ready', side_effect=ready)) + stack.enter_context(mock.patch.object(tool, 'apply_components', return_value=self.baseline)) + stack.enter_context(mock.patch.object(tool, 'apply_environment', return_value={'UPSTREAM_HOST': 'fixture'})) + stack.enter_context(mock.patch.object(tool, 'apply_public_identity', return_value='public-hash')) + return stack + + def run_transaction(self): + tool.apply_transaction(self.directory, self.root / 'runtime', self.transaction, self.journal, self.proxy) + + def assert_protected(self): + for name, expected in self.protected.items(): + path = self.directory / name + self.assertEqual((path.stat().st_ino, path.read_bytes()), expected) + self.assertFalse((self.directory / 'start.sh').exists()) + + def test_success_code_only_and_manifest_last(self): + self.lifecycle() + self.run_transaction() + self.assertEqual(self.journal['state'], 'applied') + self.assertEqual(tool.apply_bundle(self.directory, 'installed'), self.new) + self.assertEqual(self.journal['completed_file'], tool.MANIFEST) + self.stop.assert_called_once_with(self.baseline['shim']) + self.assert_protected() + self.assertEqual(set(p.name for p in (self.transaction / 'backup').iterdir()), set(self.old)) + + def test_readiness_failure_restores_exact_old_code(self): + self.lifecycle([tool.ApplyError('readiness_timeout'), None]); self.run_transaction() + self.assertEqual(self.journal['state'], 'failed_rolled_back') + self.assertEqual(tool.apply_bundle(self.directory, 'installed'), self.old) + self.assertEqual(self.stop.call_args_list, [mock.call(self.baseline['shim']), mock.call(self.candidate)]) + self.assert_protected() + + def test_rollback_failure_is_never_success(self): + self.lifecycle([tool.ApplyError('readiness_timeout'), tool.ApplyError('readiness_timeout')]); self.run_transaction() + self.assertEqual(self.journal['state'], 'rollback_failed_manual_required') + self.assert_protected() + + def test_stop_failure_never_replaces_or_launches(self): + self.lifecycle(); self.stop.side_effect = tool.ApplyError('process_identity_unknown'); self.run_transaction() + self.assertEqual(self.journal['state'], 'rollback_failed_manual_required') + self.launch.assert_not_called(); self.assertEqual(tool.apply_bundle(self.directory, 'installed'), self.old) + + def test_fault_after_each_replacement_restores_code(self): + # Recreate independent fixture for every before/after-manifest boundary. + for failed_file in list(tool.CORE_FILES) + [tool.MANIFEST]: + with self.subTest(file=failed_file): + fixture(self.directory, True, '0.3.0', 'old') + backup = self.transaction / 'backup' + if backup.exists(): + for path in backup.iterdir(): path.unlink() + backup.rmdir() + self.journal = {'schema': 1, 'state': 'running'} + original = tool.apply_write; fired = [] + def write(path, data, mode=0o600): + original(path, data, mode) + wanted = self.directory / tool.CORE_FILES.get(failed_file, failed_file) + if path == wanted and not fired: + fired.append(True); raise OSError('simulated disk failure after rename') + with self.lifecycle(), mock.patch.object(tool, 'apply_write', side_effect=write): + self.run_transaction() + self.assertEqual(self.journal['state'], 'failed_rolled_back') + self.assertEqual(tool.apply_bundle(self.directory, 'installed'), self.old) + self.assert_protected() + + def test_missing_public_host_key_stops_before_signal(self): + self.lifecycle(); (self.directory / 'sshd/ssh_host_ed25519_key.pub').unlink() + with self.assertRaisesRegex(tool.ApplyError, 'public_ssh_host_key_required'): self.run_transaction() + self.stop.assert_not_called() + + def test_changed_tailscale_binary_version_refused(self): + self.lifecycle() + self.new['tailscale-setup.sh'] = b'TS_VERSION="1.104.0"\n' + with mock.patch.object(tool, 'apply_bundle', side_effect=[self.old, self.new]): + with self.assertRaisesRegex(tool.ApplyError, 'binary_update_unsupported'): self.run_transaction() + self.stop.assert_not_called() + + def test_legacy_mixed_and_symlink_packages_refused(self): + (self.directory / tool.MANIFEST).unlink() + with self.assertRaises(tool.ApplyError): tool.apply_bundle(self.directory, 'installed') + fixture(self.directory, True) + (self.directory / 'dots-ts').write_bytes(b'changed') + with self.assertRaises(tool.ApplyError): tool.apply_bundle(self.directory, 'installed') + fixture(self.directory, True) + (self.directory / 'dots-ts').unlink(); (self.directory / 'dots-ts').symlink_to(ROOT / 'dots-ts') + with self.assertRaises(tool.ApplyError): tool.apply_bundle(self.directory, 'installed') + + def test_syntax_checks_do_not_execute_payload(self): + payload = dict(self.new, **{'shim.py': b'raise RuntimeError("must not execute")\n'}) + tool.apply_syntax(payload) + payload['shim.py'] = b'if :' + with self.assertRaisesRegex(tool.ApplyError, 'invalid_python_syntax'): tool.apply_syntax(payload) + payload['shim.py'] = b''; payload['tailscale-setup.sh'] = b'if then' + with self.assertRaisesRegex(tool.ApplyError, 'invalid_shell_syntax'): tool.apply_syntax(payload) + + def test_atomic_writer_refuses_preexisting_temporary_symlink(self): + target = self.root / 'file'; victim = self.root / 'victim'; victim.write_bytes(b'unchanged') + (self.root / 'file.new').symlink_to(victim) + with self.assertRaises(FileExistsError): tool.apply_write(target, b'bad') + self.assertEqual(victim.read_bytes(), b'unchanged') + + def test_unknown_identity_never_signalled(self): + with mock.patch.object(tool, 'apply_identity', side_effect=tool.ApplyError('process_identity_unknown')): + with self.assertRaises(tool.ApplyError): tool.apply_stop(self.baseline['shim']) + + def test_status_dead_worker_is_interrupted_not_success(self): + root = self.directory / tool.APPLY_DIRECTORY; root.mkdir(mode=0o700) + identifier = 'a' * 32; transaction = root / identifier; transaction.mkdir(mode=0o700) + journal = {'schema': 1, 'state': 'running', 'boot_id': 'boot', 'worker': self.candidate} + tool.apply_journal(transaction, journal) + with mock.patch.object(tool, 'apply_boot', return_value='boot'), mock.patch.object(tool, 'apply_identity', return_value=None): + report = tool.apply_status(self.directory, identifier) + self.assertEqual(report['state'], 'interrupted_manual_required') + self.assertEqual(report['external_ssh'], 'pending') + self.assertEqual(report['backup_directory'], str(transaction / 'backup')) + + def test_running_status_does_not_claim_external_success(self): + root = self.directory / tool.APPLY_DIRECTORY; root.mkdir(mode=0o700) + identifier = 'a' * 32; transaction = root / identifier; transaction.mkdir(mode=0o700) + journal = {'schema': 1, 'state': 'running', 'boot_id': 'boot', 'worker': self.candidate} + tool.apply_journal(transaction, journal) + with mock.patch.object(tool, 'apply_boot', return_value='boot'), mock.patch.object(tool, 'apply_identity', return_value=self.candidate): + self.assertEqual(tool.apply_status(self.directory, identifier)['state'], 'running') + + def test_real_setup_binary_version_parses(self): + payload = {'tailscale-setup.sh': (ROOT / 'tailscale-setup.sh').read_bytes()} + self.assertEqual(tool.apply_ts_version(payload), b'1.102.4') + payload['tailscale-setup.sh'] = b'TS_VERSION="1.102.4"\n' + self.assertEqual(tool.apply_ts_version(payload), b'1.102.4') + payload['tailscale-setup.sh'] = b'TS_VERSION="1.102.4\n' + with self.assertRaises(tool.ApplyError): tool.apply_ts_version(payload) + + def test_candidate_identity_failure_never_launches_duplicate(self): + self.lifecycle() + self.identify.side_effect = tool.ApplyError('process_identity_unknown') + live_child = mock.Mock(); live_child.poll.return_value = None + self.launch.side_effect = [live_child] + self.run_transaction() + self.assertEqual(self.journal['state'], 'rollback_failed_manual_required') + self.launch.assert_called_once() + self.stop.assert_called_once_with(self.baseline['shim']) + + def test_worker_journal_failure_after_destructive_intent_is_manual(self): + transaction = self.directory / tool.APPLY_DIRECTORY / ('a' * 32) + transaction.parent.mkdir(mode=0o700); transaction.mkdir(mode=0o700) + journal = {'schema': 1, 'state': 'prepared', 'boot_id': 'boot', 'phase': 'staging'} + tool.apply_journal(transaction, journal) + def transaction_failure(directory, runtime, folder, record, proxy): + record['destructive'] = True + raise OSError('simulated journal fsync failure') + proxy = mock.Mock(); proxy._operation.return_value = contextlib.nullcontext() + with mock.patch.dict(sys.modules, {'dots_proxy': proxy}), \ + mock.patch.object(tool, 'apply_boot', return_value='boot'), \ + mock.patch.object(tool, 'apply_identity', return_value=self.candidate), \ + mock.patch.object(tool, 'apply_transaction', side_effect=transaction_failure), \ + mock.patch.object(signal, 'signal'): + tool.apply_worker(self.directory, self.root, transaction) + self.assertEqual(tool.apply_read_journal(transaction)['state'], 'rollback_failed_manual_required') + + def test_malformed_journal_is_structured_error(self): + for payload in ({'schema': 1, 'state': []}, {'schema': 1, 'state': 'running', 'worker': []}, + {'schema': 1, 'state': 'running', 'worker': {'pid': 'unsafe'}}): + with self.subTest(payload=payload): + (self.transaction / 'journal.json').write_text(json.dumps(payload)) + with self.assertRaisesRegex(tool.ApplyError, 'invalid_journal'): + tool.apply_read_journal(self.transaction) + + def test_pending_transaction_blocks_new_update_before_download(self): + root = self.directory / tool.APPLY_DIRECTORY; root.mkdir(mode=0o700) + transaction = root / ('a' * 32); transaction.mkdir(mode=0o700) + tool.apply_journal(transaction, {'schema': 1, 'state': 'prepared'}) + proxy = mock.Mock(); proxy._operation.return_value = contextlib.nullcontext() + with mock.patch.dict(sys.modules, {'dots_proxy': proxy}), \ + mock.patch.object(tool.sys, 'platform', 'linux'), \ + mock.patch.object(tool.os, 'pidfd_open', create=True), \ + mock.patch.object(signal, 'pidfd_send_signal', create=True), \ + mock.patch.object(tool, 'apply_bundle', return_value=self.old), \ + mock.patch.object(tool, 'fetch_release') as fetch: + with self.assertRaisesRegex(tool.ApplyError, 'prior_transaction_requires_manual_review'): + tool.update_apply(self.directory, self.root, tag='v0.4.0', confirm=True) + fetch.assert_not_called() + + def test_refuses_without_recovery_acknowledgement(self): + with self.assertRaisesRegex(tool.ApplyError, 'confirmation_required'): + tool.update_apply(self.directory, self.root, bundle=self.root) + + def test_readiness_waits_through_transient_backend(self): + proxy = mock.Mock(); proxy.inspect_config.return_value = {}; proxy._listener_owned.return_value = True + identities = {item['pid']: item for item in list(self.baseline.values()) + [self.candidate]} + connection = mock.MagicMock(); connection.__enter__.return_value.recv.return_value = b'SSH-2.0-fixture' + with mock.patch.object(tool, 'apply_identity', side_effect=lambda pid: identities[pid]), \ + mock.patch.object(tool, 'apply_public_identity', side_effect=[tool.ApplyError('backend_not_running'), 'same']), \ + mock.patch.object(tool.socket, 'create_connection', return_value=connection): + tool.apply_ready(self.directory, self.root, proxy, self.baseline, self.candidate, 'same', {}, b'public-key', timeout=1) + + def test_identity_change_not_retried_as_transient(self): + with mock.patch.object(tool, 'apply_identity', return_value=None): + with self.assertRaisesRegex(tool.ApplyError, 'candidate_identity_changed'): + tool.apply_ready(self.directory, self.root, self.proxy, self.baseline, self.candidate, 'same', {}, b'public-key', timeout=1) + + @unittest.skipUnless(sys.platform == 'linux' and hasattr(os, 'pidfd_open'), 'Linux pidfd integration') + def test_real_owned_child_sigterm_and_identity(self): + child = subprocess.Popen([sys.executable, '-c', 'import time; time.sleep(30)']) + try: + identity = tool.apply_identity(child.pid) + tool.apply_stop(identity, timeout=2) + self.assertEqual(child.wait(timeout=2), -signal.SIGTERM) + finally: + if child.poll() is None: child.terminate(); child.wait(timeout=2) + + def test_detached_child_and_operation_lock_primitives(self): + # Independent child proves setsid/closed pipes; no daemon or live port. + child = subprocess.Popen([sys.executable, '-c', 'import os; print(os.getpid()==os.getsid(0))'], + stdout=subprocess.PIPE, stderr=subprocess.PIPE, start_new_session=True, close_fds=True) + stdout, _ = child.communicate(timeout=3) + self.assertEqual(stdout.strip(), b'True') + import fcntl + path = self.root / 'lock' + with path.open('w') as first, path.open('w') as second: + fcntl.flock(first, fcntl.LOCK_EX | fcntl.LOCK_NB) + with self.assertRaises(BlockingIOError): fcntl.flock(second, fcntl.LOCK_EX | fcntl.LOCK_NB) + + +if __name__ == '__main__': unittest.main() diff --git a/tests/test_update_check.py b/tests/test_update_check.py new file mode 100644 index 0000000..0b6f21c --- /dev/null +++ b/tests/test_update_check.py @@ -0,0 +1,196 @@ +"""Offline fixtures for stable release selection and unauthenticated bounded retrieval.""" +import contextlib +import copy +import hashlib +import importlib.machinery +import importlib.util +import io +import json +from pathlib import Path +import ssl +import subprocess +import tempfile +import unittest +import urllib.error +from unittest import mock + +ROOT = Path(__file__).resolve().parents[1] +loader = importlib.machinery.SourceFileLoader("dots_ts_update_test", str(ROOT / "dots-ts")) +spec = importlib.util.spec_from_loader(loader.name, loader) +tool = importlib.util.module_from_spec(spec) +loader.exec_module(tool) + + +def release_fixture(version="0.4.0"): + manifest = {"schema_version": 1, "package_version": version, + "files": {name: hashlib.sha256(name.encode()).hexdigest() for name in tool.CORE_FILES}} + raw = json.dumps(manifest).encode() + metadata = {"tag_name": "v" + version, "draft": False, "prerelease": False, "id": 27, + "url": tool.UPDATE_API + "/27", "html_url": tool.UPDATE_WEB + "/releases/tag/v" + version, + "assets": [{"name": name, "state": "uploaded", "size": len(raw) if name == tool.MANIFEST else len(name), + "browser_download_url": tool.UPDATE_WEB + "/releases/download/v" + version + "/" + name} + for name in list(tool.CORE_FILES) + [tool.MANIFEST]]} + return metadata, raw + + +class UpdateCheckTests(unittest.TestCase): + def setUp(self): + self.temp = tempfile.TemporaryDirectory(prefix="update-check-") + self.addCleanup(self.temp.cleanup) + self.directory = Path(self.temp.name) + metadata, raw = release_fixture("0.3.9") + for name, installed in tool.CORE_FILES.items(): + path = self.directory / installed + path.parent.mkdir(parents=True, exist_ok=True) + path.write_bytes(name.encode()) + (self.directory / tool.MANIFEST).write_bytes(raw) + + def fetch(self, version="0.4.0", metadata=None, raw_manifest=None, tag=None): + fixture, raw = release_fixture(version) + with mock.patch.object(tool, "fetch_bytes", side_effect=[json.dumps(metadata or fixture).encode(), raw_manifest or raw]) as fetch: + result = tool.fetch_release(tag) + return result, fetch + + def test_numeric_versions_and_stable_only(self): + self.assertGreater(tool.stable_version("0.10.0"), tool.stable_version("0.9.99")) + for value in ("0.4.0-dev", "v0.4.0", "0.04.0", "0.4", "PRIVATE", None): + with self.assertRaises(tool.UpdateError): + tool.stable_version(value) + + def test_latest_and_pinned_release_share_validated_manifest(self): + result, fetch = self.fetch() + self.assertEqual(result["manifest"]["package_version"], "0.4.0") + self.assertEqual(fetch.call_args_list[0].args[0], tool.UPDATE_API + "/latest") + result, fetch = self.fetch(tag="v0.4.0") + self.assertEqual(fetch.call_args_list[0].args[0], tool.UPDATE_API + "/tags/v0.4.0") + self.assertEqual(set(result["assets"]), set(tool.CORE_FILES) | {tool.MANIFEST}) + + def test_draft_prerelease_and_wrong_repo_tag_rejected(self): + original, _ = release_fixture() + for key, value in (("draft", True), ("prerelease", True), ("draft", 0), + ("tag_name", "v0.4.0-rc1"), ("url", "https://api.github.com/repos/evil/repo/releases/27"), + ("html_url", "https://github.com/evil/repo/releases/tag/v0.4.0")): + metadata = dict(original, **{key: value}) + with self.assertRaises(tool.UpdateError): + self.fetch(metadata=metadata) + with self.assertRaises(tool.UpdateError): + self.fetch(tag="v0.5.0") + + def test_legacy_releases_are_explicitly_unsupported(self): + metadata, _ = release_fixture() + metadata["assets"].pop() + with self.assertRaises(tool.UpdateError) as error: + self.fetch(metadata=metadata) + self.assertEqual(error.exception.code, "unsupported_release") + + def test_duplicate_assets_wrong_urls_sizes_and_manifest_rejected(self): + original, raw = release_fixture() + variants = [] + metadata = copy.deepcopy(original) + metadata["assets"].append(metadata["assets"][0]); variants.append(metadata) + for key, value in (("browser_download_url", "http://github.com/unsafe"), ("size", 0), ("state", "new")): + metadata = copy.deepcopy(original); metadata["assets"][0][key] = value; variants.append(metadata) + for metadata in variants: + with self.assertRaises(tool.UpdateError): self.fetch(metadata=metadata) + manifest = json.loads(raw); manifest["files"]["state/tailscaled.state"] = "a" * 64 + bad = json.dumps(manifest).encode() + metadata = copy.deepcopy(original); metadata["assets"][-1]["size"] = len(bad) + with self.assertRaises(tool.UpdateError): self.fetch(metadata=metadata, raw_manifest=bad) + manifest = json.loads(raw); manifest["package_version"] = "0.5.0" + bad = json.dumps(manifest).encode() + with self.assertRaises(tool.UpdateError): self.fetch(raw_manifest=bad) + + def test_report_states_and_check_never_writes(self): + before = {str(path.relative_to(self.directory)): path.read_bytes() for path in self.directory.rglob("*") if path.is_file()} + for version, expected in (("0.10.0", "update_available"), ("0.3.9", "current"), ("0.3.1", "local_newer")): + release, _ = self.fetch(version) + with mock.patch.object(tool, "fetch_release", return_value=release): + report = tool.update_check(self.directory, "v" + version) + self.assertEqual(report["status"], expected) + self.assertEqual(report["selection"], "pinned") + self.assertEqual(tool.update_check_exit(report), 0) + after = {str(path.relative_to(self.directory)): path.read_bytes() for path in self.directory.rglob("*") if path.is_file()} + self.assertEqual(before, after) + (self.directory / "dots-ts").write_text("modified") + with mock.patch.object(tool, "fetch_release", return_value=release): + report = tool.update_check(self.directory) + self.assertEqual(report["status"], "not_comparable") + self.assertEqual(tool.update_check_exit(report), 1) + (self.directory / tool.MANIFEST).unlink() + with mock.patch.object(tool, "fetch_release", return_value=release): + self.assertEqual(tool.update_check(self.directory)["package"]["integrity"], "unknown") + + def test_offline_rate_tls_and_timeout_remain_sanitized(self): + for code in ("offline", "rate_limited", "tls_error", "timeout", "invalid_metadata"): + with mock.patch.object(tool, "fetch_release", side_effect=tool.UpdateError(code)): + report = tool.update_check(self.directory) + self.assertEqual(report["status"], "check_failed") + self.assertEqual(report["error"], code) + self.assertEqual(tool.update_check_exit(report), 2) + self.assertNotIn(str(self.directory), json.dumps(report)) + + def test_url_and_proxy_allowlists(self): + for url in ("http://api.github.com/repos/abruption/dots-tailscale/releases/latest", "https://api.github.com.evil/x", + tool.UPDATE_API + "/latest?token=PRIVATE", "https://user:PRIVATE@api.github.com/repos/abruption/dots-tailscale/releases/latest"): + with self.assertRaises(tool.UpdateError): tool.validate_update_url(url) + for url in ("https://evil.example/a", "http://release-assets.githubusercontent.com/a", "https://release-assets.githubusercontent.com:444/a"): + with self.assertRaises(tool.UpdateError): tool.validate_update_url(url, redirect=True) + self.assertEqual(tool.validate_proxy("http://browser-proxy:8889"), "http://browser-proxy:8889") + for proxy in ("http://user:PRIVATE@browser-proxy:8889", "socks5://localhost:1055", "http://browser-proxy:8889/?token=PRIVATE"): + with self.assertRaises(tool.UpdateError): tool.validate_proxy(proxy) + + def test_network_ignores_ambient_proxy_and_tokens(self): + response = mock.MagicMock() + response.__enter__.return_value = response + response.status = 200; response.headers = {}; response.read.return_value = b"{}" + opener = mock.Mock(); opener.open.return_value = response + with mock.patch.dict("os.environ", {"HTTPS_PROXY": "http://user:PRIVATE@bad.invalid", "GITHUB_TOKEN": "PRIVATE"}), \ + mock.patch.object(tool.urllib.request, "ProxyHandler", wraps=tool.urllib.request.ProxyHandler) as proxy, \ + mock.patch.object(tool.urllib.request, "build_opener", return_value=opener): + self.assertEqual(tool._network_fetch(tool.UPDATE_API + "/latest", None, 1024), b"{}") + self.assertEqual(proxy.call_args.args[0], {}) + request = opener.open.call_args.args[0] + self.assertNotIn("Authorization", request.headers) + self.assertNotIn("PRIVATE", str(request.headers)) + + def test_http_error_classification(self): + for status, headers, expected in ((429, {}, "rate_limited"), (403, {"X-RateLimit-Remaining": "0"}, "rate_limited"), + (404, {}, "not_found"), (503, {}, "http_error")): + opener = mock.Mock(); opener.open.side_effect = urllib.error.HTTPError("private", status, "PRIVATE", headers, None) + with mock.patch.object(tool.urllib.request, "build_opener", return_value=opener), self.assertRaises(tool.UpdateError) as error: + tool._network_fetch(tool.UPDATE_API + "/latest", None, 1024) + self.assertEqual(error.exception.code, expected) + opener = mock.Mock(); opener.open.side_effect = urllib.error.URLError(ssl.SSLError("PRIVATE")) + with mock.patch.object(tool.urllib.request, "build_opener", return_value=opener), self.assertRaises(tool.UpdateError) as error: + tool._network_fetch(tool.UPDATE_API + "/latest", None, 1024) + self.assertEqual(error.exception.code, "tls_error") + + def test_whole_request_deadline_and_response_bound(self): + with mock.patch.object(tool.subprocess, "run", side_effect=subprocess.TimeoutExpired("PRIVATE", 1)), self.assertRaises(tool.UpdateError) as error: + tool.fetch_bytes(tool.UPDATE_API + "/latest") + self.assertEqual(error.exception.code, "timeout") + response = mock.MagicMock(); response.__enter__.return_value = response + response.status = 200; response.headers = {}; response.read.return_value = b"x" * 1025 + opener = mock.Mock(); opener.open.return_value = response + with mock.patch.object(tool.urllib.request, "build_opener", return_value=opener), self.assertRaises(tool.UpdateError) as error: + tool._network_fetch(tool.UPDATE_API + "/latest", None, 1024) + self.assertEqual(error.exception.code, "response_too_large") + + def test_malformed_child_error_has_fixed_classification(self): + for value in ([], {}, None, 1): + self.assertEqual(tool.UpdateError(value).code, "invalid_response") + result = mock.Mock(returncode=0, stdout=json.dumps({"error": value}).encode()) + with mock.patch.object(tool.subprocess, "run", return_value=result), self.assertRaises(tool.UpdateError) as error: + tool.fetch_bytes(tool.UPDATE_API + "/latest") + self.assertEqual(error.exception.code, "invalid_response") + + def test_json_has_no_notice_or_private_errors(self): + output = io.StringIO() + with mock.patch.object(tool, "directories", return_value=(self.directory, self.directory)), \ + mock.patch.object(tool, "fetch_release", side_effect=tool.UpdateError("offline")), contextlib.redirect_stdout(output): + code = tool.main(["update", "check", "--json"]) + self.assertEqual(code, 2) + self.assertEqual(json.loads(output.getvalue())["error"], "offline") + + +if __name__ == "__main__": unittest.main() diff --git a/tests/test_update_notice.py b/tests/test_update_notice.py new file mode 100644 index 0000000..3960bd7 --- /dev/null +++ b/tests/test_update_notice.py @@ -0,0 +1,157 @@ +"""Opt-in cache-only notices, conservative storage, and successful-start isolation.""" +import contextlib +import io +import json +import os +from pathlib import Path +import tempfile +import unittest +from unittest import mock + +from test_update_check import tool, release_fixture + +ROOT = Path(__file__).resolve().parents[1] + + +class InteractiveOutput(io.StringIO): + def isatty(self): return True + + +class UpdateNoticeTests(unittest.TestCase): + def setUp(self): + self.temp = tempfile.TemporaryDirectory(prefix="update-notice-") + self.addCleanup(self.temp.cleanup) + self.directory = Path(self.temp.name) + _, raw = release_fixture("0.3.9") + for source, installed in tool.CORE_FILES.items(): + path = self.directory / installed + path.parent.mkdir(parents=True, exist_ok=True) + path.write_bytes(source.encode()) + (self.directory / tool.MANIFEST).write_bytes(raw) + self.release = {"tag": "v0.4.0", "version": "0.4.0"} + + def enable_and_refresh(self): + tool.notification_configure(self.directory, True, 24) + with mock.patch.object(tool, "fetch_release", return_value=self.release): + report = tool.notification_refresh(self.directory) + self.assertEqual(report["status"], "refreshed") + return report + + def notify(self, interactive=True): + output = InteractiveOutput() if interactive else io.StringIO() + with contextlib.redirect_stderr(output), mock.patch.object(tool, "fetch_release", side_effect=AssertionError("notice network")): + tool.notification_notify(self.directory) + return output.getvalue() + + def test_default_is_disabled_and_has_no_storage_side_effect(self): + report = tool.notification_show(self.directory) + self.assertFalse(report["enabled"]) + self.assertTrue(report["stale"]) + self.assertEqual(tool.notification_refresh(self.directory)["status"], "disabled") + self.assertEqual(self.notify(), "") + self.assertFalse((self.directory / tool.NOTICE_DIRECTORY).exists()) + + def test_new_release_notice_deduplication_and_disable(self): + self.enable_and_refresh() + self.assertIn("v0.4.0", self.notify()) + self.assertEqual(self.notify(), "") + self.assertEqual(tool.notification_show(self.directory)["last_notified"], "v0.4.0") + tool.notification_configure(self.directory, False) + self.assertEqual(self.notify(), "") + with mock.patch.object(tool, "fetch_release", side_effect=AssertionError("disabled network")): + self.assertEqual(tool.notification_refresh(self.directory)["status"], "disabled") + + def test_interval_expiry_suppresses_stale_notice_and_force_refresh(self): + before = self.enable_and_refresh() + with mock.patch.object(tool, "fetch_release", side_effect=AssertionError("fresh network")): + self.assertEqual(tool.notification_refresh(self.directory)["status"], "cached") + with mock.patch.object(tool.time, "time", return_value=before["checked_at"] + 24 * 3600): + self.assertTrue(tool.notification_show(self.directory)["stale"]) + self.assertEqual(self.notify(), "") + with mock.patch.object(tool, "fetch_release", return_value=self.release) as fetch: + self.assertEqual(tool.notification_refresh(self.directory, force=True)["status"], "refreshed") + fetch.assert_called_once() + + def test_failed_refresh_retains_last_success_and_marks_stale(self): + before = self.enable_and_refresh() + for code in ("offline", "rate_limited", "tls_error"): + with mock.patch.object(tool.time, "time", return_value=before["checked_at"] + 10), \ + mock.patch.object(tool, "fetch_release", side_effect=tool.UpdateError(code)): + after = tool.notification_refresh(self.directory, force=True) + self.assertEqual(after["checked_at"], before["checked_at"]) + self.assertEqual(after["release"], before["release"]) + self.assertEqual(after["last_error"], code) + self.assertTrue(after["stale"]) + self.assertEqual(after["status"], "refresh_failed") + self.assertEqual(self.notify(), "") + + def test_redirected_stderr_json_and_raw_cli_remain_clean(self): + self.enable_and_refresh() + self.assertEqual(self.notify(interactive=False), "") + self.assertIsNone(tool.notification_show(self.directory)["last_notified"]) + output = io.StringIO() + with mock.patch.object(tool, "directories", return_value=(self.directory, self.directory)), \ + mock.patch.object(tool, "notification_notify", side_effect=AssertionError("unexpected notice")), \ + contextlib.redirect_stdout(output): + self.assertEqual(tool.main(["update", "notifications", "show", "--json"]), 0) + self.assertEqual(json.loads(output.getvalue())["release"], self.release) + with mock.patch.object(tool, "directories", return_value=(self.directory, self.directory)), \ + mock.patch.object(tool, "notification_notify", side_effect=AssertionError("unexpected notice")), \ + mock.patch.object(tool.os, "execv", side_effect=RuntimeError("exec boundary")): + with self.assertRaisesRegex(RuntimeError, "exec boundary"): tool.main(["cli", "status", "--json"]) + + def test_readonly_storage_failure_is_silent_and_successful(self): + self.enable_and_refresh() + with mock.patch.object(tool, "notice_write", side_effect=OSError("PRIVATE readonly")): + self.assertEqual(self.notify(), "") + with mock.patch.object(tool, "directories", return_value=(self.directory, self.directory)), \ + mock.patch.object(tool, "notice_store", side_effect=tool.NoticeError("storage_unavailable")): + self.assertEqual(tool.main(["update", "notifications", "notify"]), 0) + self.assertIsNone(tool.notification_show(self.directory)["last_notified"]) + + def test_storage_rejects_symlinks_hardlinks_modes_and_hostile_cache(self): + self.enable_and_refresh() + folder = self.directory / tool.NOTICE_DIRECTORY + cache = folder / "cache.json" + original = cache.read_bytes() + cache.unlink(); cache.symlink_to(self.directory / tool.MANIFEST) + self.assertEqual(self.notify(), "") + cache.unlink(); cache.write_bytes(original); cache.chmod(0o600) + os.link(cache, folder / "hardlink") + self.assertEqual(self.notify(), "") + (folder / "hardlink").unlink() + cache.chmod(0o644) + self.assertEqual(self.notify(), "") + cache.chmod(0o600) + hostile = json.loads(original); hostile["last_error"] = {"PRIVATE": "TOKEN"} + cache.write_text(json.dumps(hostile)) + self.assertEqual(self.notify(), "") + cache.write_bytes(original) + folder.chmod(0o755) + self.assertEqual(self.notify(), "") + + def test_interval_validation_and_safe_json_errors(self): + for interval in (0, 721, True): + with self.assertRaises(tool.NoticeError): tool.notification_configure(self.directory, True, interval) + output = io.StringIO() + with mock.patch.object(tool, "directories", return_value=(self.directory, self.directory)), contextlib.redirect_stdout(output): + self.assertEqual(tool.main(["update", "notifications", "configure", "--enable", "--interval-hours", "0", "--json"]), 1) + self.assertEqual(json.loads(output.getvalue()), {"error": "invalid_settings"}) + + def test_unknown_or_modified_package_does_not_get_update_notice(self): + self.enable_and_refresh() + (self.directory / "dots-ts").write_text("modified source") + self.assertEqual(self.notify(), "") + (self.directory / tool.MANIFEST).unlink() + self.assertEqual(self.notify(), "") + + def test_hook_is_only_after_success_for_interactive_start_and_has_hard_timeout(self): + source = (ROOT / "tailscale-setup.sh").read_text() + hook = source[source.index("# Opt-in cached notice only."):] + self.assertIn('if [[ "$MODE" = start && -t 2 ]]', hook) + self.assertIn('timeout --kill-after=1s 1s python3 "$D/dots-ts" update notifications notify >/dev/null || true', hook) + self.assertNotIn(" refresh", hook) + self.assertLess(source.index("if [[ \"${AUTH_COMMAND_STATUS:-0}\" != 0 ]]"), source.index("# Opt-in cached notice only.")) + + +if __name__ == "__main__": unittest.main() From 05e3913a1c932182dc083c4961980de0269edcdd Mon Sep 17 00:00:00 2001 From: abruption Date: Wed, 21 Oct 2026 11:33:39 +0900 Subject: [PATCH 2/4] fix: confirm process termination through pidfd readiness --- dots-ts | 44 +++++++++++++++++++++++++++----------- source-manifest.json | 2 +- tests/test_update_apply.py | 40 +++++++++++++++++++++++++++++++++- 3 files changed, 71 insertions(+), 15 deletions(-) diff --git a/dots-ts b/dots-ts index 88ce29b..d3d1710 100755 --- a/dots-ts +++ b/dots-ts @@ -666,8 +666,18 @@ def apply_identity(pid): return {'pid': int(pid), 'ticks': fields[19], 'uid': p.stat().st_uid, 'exe': os.readlink(p / 'exe')} except FileNotFoundError: - if p.exists(): raise ApplyError('process_identity_unknown') - return None + # A process can become a zombie between stat and exe: exe disappears + # before its /proc directory. Re-read state; ENOENT alone is not exit. + try: + fields = (p / 'stat').read_text().rsplit(')', 1)[1].split() + if fields[0] == 'Z': return None + except FileNotFoundError: + try: p.stat() + except FileNotFoundError: return None + except OSError: raise ApplyError('process_identity_unknown') + except (OSError, IndexError, ValueError): + raise ApplyError('process_identity_unknown') + raise ApplyError('process_identity_unknown') except (OSError, IndexError, ValueError): raise ApplyError('process_identity_unknown') @@ -792,20 +802,26 @@ def apply_stop(process, timeout=8): current = apply_identity(process['pid']) if current is None: return if current != process: raise ApplyError('process_identity_changed') - fd = os.pidfd_open(process['pid']) + try: fd = os.pidfd_open(process['pid']) + except ProcessLookupError: return # kernel confirms absence at pidfd_open try: - if apply_identity(process['pid']) != process: raise ApplyError('process_identity_changed') - signal.pidfd_send_signal(fd, signal.SIGTERM) + current = apply_identity(process['pid']) + if current is None: return + if current != process: raise ApplyError('process_identity_changed') + try: signal.pidfd_send_signal(fd, signal.SIGTERM) + except ProcessLookupError: return + # The pidfd stays bound to this process even after PID reuse. Its + # readability is positive kernel exit evidence, without /proc races. deadline = time.monotonic() + timeout - while time.monotonic() < deadline: - current = apply_identity(process['pid']) - if current is None: return - if current != process: raise ApplyError('process_identity_changed') - time.sleep(.1) + with selectors.DefaultSelector() as selector: + selector.register(fd, selectors.EVENT_READ) + while time.monotonic() < deadline: + if selector.select(max(0, deadline - time.monotonic())): return raise ApplyError('component_stop_timeout') finally: os.close(fd) + def apply_launch(directory, baseline, environment, log): child = subprocess.Popen([baseline['shim']['exe'], str(directory / 'shim/shim.py')], env=environment, stdin=subprocess.DEVNULL, stdout=log, stderr=log, @@ -889,9 +905,11 @@ def apply_transaction(directory, runtime, transaction, journal, proxy): apply_journal(transaction, journal, error=str(failure) if isinstance(failure, ApplyError) else 'transaction_failed', phase='rollback_intent') try: if child is not None: - if candidate is None: - if child.poll() is None: raise ApplyError('candidate_identity_unknown') - else: apply_stop(candidate) + # Popen.poll uses waitpid: a reaped owned child is positive exit + # evidence even if a concurrent /proc inspection was uncertain. + if child.poll() is None: + if candidate is None: raise ApplyError('candidate_identity_unknown') + apply_stop(candidate) child.wait(timeout=2) if not stopped: # Failed stop is not permission to start a duplicate or alter code. diff --git a/source-manifest.json b/source-manifest.json index aa388e9..8afa793 100644 --- a/source-manifest.json +++ b/source-manifest.json @@ -1,6 +1,6 @@ { "files": { - "dots-ts": "b976a21139bec710364317da91a9ae57c6f1c9a3b3ea9b0042a7471d544a29df", + "dots-ts": "8b0eb5435b790cde5e2bb5acb6e9cba1a3681d57988ff2314291904fc2e3953e", "dots_proxy.py": "2b74af2e175d12300196cab008062ba1098e12bc208d617e3fe63c23c2613390", "shim.py": "c0c55c45fbd9126e860b56d0d0359c0c87f8b39ef5d4da4a9560522452ac8c65", "tailscale-setup.sh": "79b703df7f1c871a5ba5e081cac19d1e810f0d4886212f461b198573af77830b" diff --git a/tests/test_update_apply.py b/tests/test_update_apply.py index 81814e1..a978793 100644 --- a/tests/test_update_apply.py +++ b/tests/test_update_apply.py @@ -60,7 +60,7 @@ def setUp(self): def lifecycle(self, ready=None): stack = contextlib.ExitStack(); self.addCleanup(stack.close) self.stop = stack.enter_context(mock.patch.object(tool, 'apply_stop')) - self.launch = stack.enter_context(mock.patch.object(tool, 'apply_launch', side_effect=[mock.Mock(), mock.Mock()])) + self.launch = stack.enter_context(mock.patch.object(tool, 'apply_launch', side_effect=[mock.Mock(poll=mock.Mock(return_value=None)), mock.Mock(poll=mock.Mock(return_value=None))])) self.identify = stack.enter_context(mock.patch.object(tool, 'apply_candidate', side_effect=[self.candidate, self.restored])) self.ready = stack.enter_context(mock.patch.object(tool, 'apply_ready', side_effect=ready)) stack.enter_context(mock.patch.object(tool, 'apply_components', return_value=self.baseline)) @@ -185,6 +185,44 @@ def test_running_status_does_not_claim_external_success(self): with mock.patch.object(tool, 'apply_boot', return_value='boot'), mock.patch.object(tool, 'apply_identity', return_value=self.candidate): self.assertEqual(tool.apply_status(self.directory, identifier)['state'], 'running') + def test_identity_stat_to_zombie_transition_is_positive_exit(self): + running = '1 (fixture) ' + ' '.join(['S'] + ['0'] * 18 + ['42']) + zombie = '1 (fixture) ' + ' '.join(['Z'] + ['0'] * 18 + ['42']) + with mock.patch.object(Path, 'read_text', side_effect=[running, zombie]), \ + mock.patch.object(Path, 'stat', return_value=mock.Mock(st_uid=os.getuid())), \ + mock.patch.object(os, 'readlink', side_effect=FileNotFoundError()): + self.assertIsNone(tool.apply_identity(1)) + + def test_identity_missing_exe_but_running_stays_unknown(self): + running = '1 (fixture) ' + ' '.join(['S'] + ['0'] * 18 + ['42']) + with mock.patch.object(Path, 'read_text', return_value=running), \ + mock.patch.object(Path, 'stat', return_value=mock.Mock(st_uid=os.getuid())), \ + mock.patch.object(os, 'readlink', side_effect=FileNotFoundError()): + with self.assertRaisesRegex(tool.ApplyError, 'process_identity_unknown'): + tool.apply_identity(1) + + def test_stop_waits_on_pidfd_without_post_signal_proc_reads(self): + selector = mock.MagicMock() + selector.__enter__.return_value.select.return_value = [(mock.Mock(), 1)] + with mock.patch.object(tool, 'apply_identity', side_effect=[self.baseline['shim'], self.baseline['shim']]) as identity, \ + mock.patch.object(os, 'pidfd_open', return_value=77, create=True), \ + mock.patch.object(signal, 'pidfd_send_signal', create=True) as send, \ + mock.patch.object(tool.selectors, 'DefaultSelector', return_value=selector), \ + mock.patch.object(os, 'close'): + tool.apply_stop(self.baseline['shim']) + self.assertEqual(identity.call_count, 2) + send.assert_called_once_with(77, signal.SIGTERM) + selector.__enter__.return_value.register.assert_called_once_with(77, tool.selectors.EVENT_READ) + + def test_exited_owned_candidate_skips_uncertain_proc_stop(self): + self.lifecycle([tool.ApplyError('candidate_identity_changed'), None]) + dead = mock.Mock(poll=mock.Mock(return_value=1)) + self.launch.side_effect = [dead, mock.Mock()] + self.run_transaction() + self.assertEqual(self.journal['state'], 'failed_rolled_back') + self.stop.assert_called_once_with(self.baseline['shim']) + dead.wait.assert_called_once_with(timeout=2) + def test_real_setup_binary_version_parses(self): payload = {'tailscale-setup.sh': (ROOT / 'tailscale-setup.sh').read_bytes()} self.assertEqual(tool.apply_ts_version(payload), b'1.102.4') From fe436ec49a9e8f6e7e645bfb039784c482c39178 Mon Sep 17 00:00:00 2001 From: abruption Date: Wed, 21 Oct 2026 11:48:53 +0900 Subject: [PATCH 3/4] fix: harden updater rollback and structured failures after independent review --- docs/usage.ja.md | 8 ++ docs/usage.ko.md | 8 ++ docs/usage.md | 8 ++ docs/usage.zh-CN.md | 8 ++ dots-ts | 159 ++++++++++++++++++++++++++++------- source-manifest.json | 4 +- tailscale-setup.sh | 2 +- tests/test_update_apply.py | 163 ++++++++++++++++++++++++++++++++++-- tests/test_update_check.py | 11 +++ tests/test_update_notice.py | 7 +- 10 files changed, 338 insertions(+), 40 deletions(-) diff --git a/docs/usage.ja.md b/docs/usage.ja.md index 35d6061..82aff84 100644 --- a/docs/usage.ja.md +++ b/docs/usage.ja.md @@ -215,6 +215,14 @@ JSON では `package.declared_version`、`package.source_revision`、`package.in トランザクションの記録は `.update-transactions/TRANSACTION_ID/` に保存されます。`journal.json` は進行段階、`stage/` は選択したバンドル、`backup/` は以前のソースのみを保持し、`supervisor/` には updater のコードのスナップショットを保存します。外部から接続を確認するまでは記録を残してください。`update status` の終了コードは、実行中または適用済みなら 0、拒否、ロールバック、中断、手動復旧が必要な場合は 1 です。実行中の終了コード 0 は完了を意味しません。 +`applied` は、選んだコードがマニフェストと一致し、ローカルの準備状態の確認を通過したことを表します。shim の CONNECT/WebSocket 通信全体を検証した結果ではありません。再起動する shim には upstream のホスト・ポートと絶対パスの SSL 証明書設定を引き継ぎますが、その他のシェル環境やデバッグ設定は引き継ぎません。stdout/stderr は破棄するため、以前の `shim.log` への新しい出力を期待せず、トランザクションの固定エラーコードと別途行う接続確認を使ってください。 + +中断したトランザクションは、証拠を削除せずに手動復旧を完了できます。 + +1. 独立したターミナルで worker が終了したことを確実に確認します。PID が見えないことや status が不明なことだけでは、終了した証拠になりません。 +2. 必要なら確認済みのコードバンドルを復元して起動し、マニフェスト、ローカルの準備状態、新しい外部 SSH 接続を厳密なホスト鍵検証付きで確認します。古いヘルパーでは、別途確認済みのソースランナーによるマニフェスト確認が必要な場合があります。 +3. worker が動いておらず、インストール先の `.operation.lock` を取得した状態で、確認済みのトランザクションディレクトリ全体を `.update-transactions/` の外にある、自分が所有する権限 700 の `.update-recovery-archive/` へ移します。記録とバックアップは一緒に残してください。これで未完了トランザクションの検索対象から外れます。apply の制限を回避するために、実行中または未確認の記録を移してはいけません。 + ## v0.3.1 ヘルパーと任意のアプリケーションプロキシ diff --git a/docs/usage.ko.md b/docs/usage.ko.md index fd89fdf..c1a5388 100644 --- a/docs/usage.ko.md +++ b/docs/usage.ko.md @@ -164,6 +164,14 @@ setup과 proxy apply가 사용하는 작업 잠금을 유지한 채 소스 파 작업 기록은 `.update-transactions/TRANSACTION_ID/`에 남습니다. `journal.json`에는 진행 단계, `stage/`에는 선택한 묶음, `backup/`에는 이전 소스 묶음만 저장하고, `supervisor/`에는 updater 코드 사본을 둡니다. 외부 연결을 확인할 때까지 기록을 보존하세요. `update status`의 종료 코드는 진행 중이거나 적용된 작업은 0, 거부·원복·중단·수동 복구 필요 결과는 1입니다. 진행 중 작업의 종료 코드 0은 완료를 뜻하지 않습니다. +`applied`는 선택한 코드가 명세와 일치하고 로컬 준비 검사를 통과했다는 뜻입니다. shim의 실제 CONNECT·WebSocket 통신까지 시험한 결과는 아닙니다. 재시작한 shim에는 기존 upstream 호스트·포트와 절대 경로의 SSL 인증서 설정을 유지하지만, 다른 셸 환경이나 디버그 설정은 물려주지 않습니다. stdout·stderr는 버리므로 기존 `shim.log`에 새 출력이 쌓일 것으로 기대하지 말고, 작업 기록의 고정 오류 코드와 별도 연결 검사로 확인하세요. + +중단된 작업은 증거를 삭제하지 않고 다음과 같이 수동 복구를 마무리할 수 있습니다. + +1. 독립 터미널에서 작업 프로세스가 종료됐음을 확실히 확인합니다. PID에 접근할 수 없거나 status가 불확실하다는 이유만으로 종료됐다고 판단하지 마세요. +2. 필요하면 검토된 코드 묶음으로 복원해 기동하고, 명세·로컬 준비 상태·새 외부 SSH 연결을 확인합니다. 구버전 도구에서는 별도로 검토한 소스 실행기로 명세를 확인해야 할 수 있습니다. +3. 작업 프로세스가 없고 설치 경로의 `.operation.lock`을 확보한 상태에서, 검토를 마친 작업 디렉터리 전체를 `.update-transactions/` 바깥의 본인 소유·권한 700인 `.update-recovery-archive/`로 옮깁니다. 기록과 백업을 함께 보존하세요. 이렇게 보관하면 미완료 작업 검색에서 제외됩니다. apply 차단을 풀려고 실행 중이거나 검토하지 않은 작업을 옮기면 안 됩니다. + ## v0.3.1 도구와 선택적 애플리케이션 프록시 diff --git a/docs/usage.md b/docs/usage.md index 75631bf..f135b31 100644 --- a/docs/usage.md +++ b/docs/usage.md @@ -215,6 +215,14 @@ A prior incomplete transaction blocks new updates. After a killed worker or VM r Transaction records live under `.update-transactions/TRANSACTION_ID/`: `journal.json` records the phase, `stage/` contains the selected bundle, `backup/` contains only the previous source bundle, and `supervisor/` holds the updater code snapshot. Keep these records until you have checked external connectivity. `update status` returns exit 0 for a running or applied transaction and 1 for a rejected, rolled-back, interrupted or manual-recovery result. Exit 0 for a running transaction is not completion. +`applied` means the selected code matches its manifest and local readiness checks passed. It does not test the shim's end-to-end CONNECT/WebSocket path. The restarted shim keeps its upstream host/port and absolute SSL certificate-path settings; unrelated shell environment and debug settings are not inherited. Its stdout/stderr are discarded, so use the transaction's fixed error codes and separate connectivity checks rather than expecting new output in the old `shim.log`. + +After an interrupted transaction, a manual recovery can be concluded without deleting evidence: + +1. In the independent terminal, positively establish that the worker has exited. An inaccessible PID or an uncertain `status` result is not proof of exit. +2. Restore a reviewed code bundle if needed, start it and verify its manifest, local readiness and a new external strict-host-key SSH connection. A legacy helper may require a separately reviewed source runner to inspect its manifest. +3. With the installation's `.operation.lock` held and no active worker, move the entire reviewed transaction directory to an owned, mode-700 `.update-recovery-archive/` directory outside `.update-transactions/`. Preserve the journal and backups together. This manual archive removes it from the pending-update scan; never archive an active or unreviewed transaction just to unblock apply. + ## v0.3.1 helpers and optional application proxies diff --git a/docs/usage.zh-CN.md b/docs/usage.zh-CN.md index 34670ff..6e74786 100644 --- a/docs/usage.zh-CN.md +++ b/docs/usage.zh-CN.md @@ -215,6 +215,14 @@ JSON 将 `package.declared_version`、`package.source_revision`、`package.integ 事务记录保存在 `.update-transactions/TRANSACTION_ID/` 下：`journal.json` 记录阶段，`stage/` 保存选定的文件包，`backup/` 只保存之前的源码包，`supervisor/` 保存更新器代码快照。在确认外部连接正常前，请保留这些记录。`update status` 对运行中或已应用的事务返回退出码 0；对拒绝、已回滚、中断或需要手动恢复的结果返回 1。运行中的退出码 0 不代表已经完成。 +`applied` 表示所选代码与清单一致，并通过了本地就绪检查，不代表已经验证 shim 的端到端 CONNECT/WebSocket 通信。重启的 shim 保留上游主机、端口和使用绝对路径的 SSL 证书设置，但不继承其他 shell 环境或调试设置。其 stdout/stderr 会被丢弃，因此应查看事务中的固定错误码并单独检查连接，而不要期待旧 `shim.log` 出现新的输出。 + +中断后，可以在保留证据的前提下完成手动恢复： + +1. 在独立终端中明确确认 worker 已退出。PID 无法访问或 status 不确定，都不能作为退出的证据。 +2. 必要时恢复经过审核的代码包并启动，验证清单、本地就绪状态以及一次新的外部严格主机密钥 SSH 连接。旧版工具可能需要单独审核的源码运行器来检查清单。 +3. 确认没有活动 worker，并持有安装目录的 `.operation.lock` 后，将整个已审核的事务目录移动到 `.update-transactions/` 之外、属于当前用户且权限为 700 的 `.update-recovery-archive/`。日志和备份应一并保留。归档后，该事务不再阻止新更新；不要为了绕过限制而归档仍在运行或未经审核的事务。 + ## v0.3.1 辅助工具与可选应用代理 diff --git a/dots-ts b/dots-ts index d3d1710..63a215a 100755 --- a/dots-ts +++ b/dots-ts @@ -165,7 +165,7 @@ def validate_proxy(proxy): return None try: parsed = urllib.parse.urlsplit(proxy) - if (len(proxy) > 512 or parsed.scheme not in ("http", "https") or not parsed.hostname + if (len(proxy) > 512 or parsed.scheme != "http" or not parsed.hostname or parsed.username is not None or parsed.password is not None or parsed.path not in ("", "/") or parsed.query or parsed.fragment or parsed.port is None or any(ord(c) < 33 or ord(c) > 126 for c in proxy)): @@ -362,6 +362,7 @@ def fetch_release(tag=None, proxy=None, timeout=UPDATE_TIMEOUT): for value in manifest["files"].values())): raise UpdateError("invalid_metadata") release["manifest"] = manifest + release["manifest_bytes"] = raw_manifest return release @@ -586,7 +587,8 @@ def notification_notify(directory): cache["last_notified"] = release["tag"] notice_write(folder, "cache.json", cache) print("dots-ts: stable release " + release["tag"] + " is available; run dots-ts update check before updating.", file=sys.stderr) - except (NoticeError, UpdateError, OSError): + except Exception: + # A best-effort interactive notice must not print a traceback or affect start. return @@ -597,7 +599,28 @@ APPLY_TERMINAL = {'applied', 'failed_rolled_back', 'rollback_failed_manual_requi class ApplyError(Exception): - pass + def __init__(self, code, transaction_id=None): + super().__init__(code) + self.transaction_id = transaction_id + + +def apply_error_report(error): + if isinstance(error, ApplyError): code = str(error) + elif isinstance(error, UpdateError): code = error.code + else: + try: + import dots_proxy as proxy + proxy_error = isinstance(error, proxy.ProxyError) + except ImportError: proxy_error = False + if proxy_error: code = 'operation_busy_or_unsafe' + elif isinstance(error, OSError): code = 'filesystem_or_process_error' + elif isinstance(error, (ValueError, TypeError, KeyError, RecursionError)): code = 'invalid_apply_data' + else: code = 'apply_internal_error' + report = {'error': code} + identifier = getattr(error, 'transaction_id', None) + if identifier: report['transaction_id'] = identifier + return report + def apply_private(path, create=False): @@ -635,6 +658,31 @@ def apply_journal(transaction, journal, **changes): apply_write(transaction / 'journal.json', json.dumps(journal, sort_keys=True).encode()) +def apply_journal_best_effort(transaction, journal, **changes): + journal.update(changes) + try: + apply_journal(transaction, journal, **changes) + return True + except Exception: + journal['journal_persistence_failed'] = True + return False + + +def apply_outcome(transaction, journal, state, **changes): + if apply_journal_best_effort(transaction, journal, state=state, **changes): return + # One bounded static marker per outcome; never append raw exception text. + try: + fd = os.open(str(transaction / 'worker.log'), os.O_WRONLY | os.O_APPEND | os.O_CREAT | os.O_NOFOLLOW | os.O_NONBLOCK, 0o600) + try: + info = os.fstat(fd) + if (stat.S_ISREG(info.st_mode) and info.st_uid == os.getuid() + and stat.S_IMODE(info.st_mode) == 0o600 and info.st_size < 4096): + os.write(fd, ('update_outcome=' + state + ' journal_persisted=false\n').encode()) + os.fsync(fd) + finally: os.close(fd) + except Exception: pass + + def apply_read_journal(transaction): try: value = parse_release_json(bounded_code_read(transaction, 'journal.json', 65536)) @@ -691,7 +739,15 @@ def apply_status(directory, identifier): root = apply_private(directory / APPLY_DIRECTORY) transaction = apply_private(root / identifier) journal = apply_read_journal(transaction) - if journal.get('state') not in APPLY_TERMINAL: + try: + log = bounded_code_read(transaction, 'worker.log', 8192) + except FileNotFoundError: log = b'' + except (OSError, ValueError): log = b'' + markers = re.findall(rb'^update_outcome=(applied|failed_rolled_back|rollback_failed_manual_required|rejected) journal_persisted=false$', log, re.M) + if markers: + journal = dict(journal, state='interrupted_manual_required', journal_persistence_failed=True, + observed_outcome=markers[-1].decode('ascii')) + if journal.get('state') not in APPLY_TERMINAL and not markers: worker = journal.get('worker') try: alive = journal.get('boot_id') == apply_boot() and worker and apply_identity(worker['pid']) == worker @@ -790,6 +846,11 @@ def apply_environment(process): environment = {'PATH': '/usr/bin:/bin', 'PYTHONNOUSERSITE': '1', 'PYTHONDONTWRITEBYTECODE': '1'} for key, default in (('UPSTREAM_HOST', 'browser-proxy'), ('UPSTREAM_PORT', '8889'), ('SHIM_PORT', '18089')): environment[key] = values.get(key.encode(), default.encode()).decode('ascii') + for key in ('SSL_CERT_FILE', 'SSL_CERT_DIR'): + if key.encode() in values: + value = os.fsdecode(values[key.encode()]) + if not os.path.isabs(value) or len(value) > 4096 or '\x00' in value: raise ValueError() + environment[key] = value if not re.fullmatch(r'[A-Za-z0-9._:-]{1,253}', environment['UPSTREAM_HOST']): raise ValueError() if not 1 <= int(environment['UPSTREAM_PORT']) <= 65535 or environment['SHIM_PORT'] != '18089': raise ValueError() except (OSError, UnicodeError, ValueError): raise ApplyError('unsupported_shim_environment') @@ -805,7 +866,12 @@ def apply_stop(process, timeout=8): try: fd = os.pidfd_open(process['pid']) except ProcessLookupError: return # kernel confirms absence at pidfd_open try: - current = apply_identity(process['pid']) + try: current = apply_identity(process['pid']) + except ApplyError: + with selectors.DefaultSelector() as selector: + selector.register(fd, selectors.EVENT_READ) + if selector.select(0): return + raise if current is None: return if current != process: raise ApplyError('process_identity_changed') try: signal.pidfd_send_signal(fd, signal.SIGTERM) @@ -822,6 +888,25 @@ def apply_stop(process, timeout=8): +def apply_stop_child(child, timeout=8): + # Popen owns the unreaped child, whose PID cannot yet be reused. This is + # authoritative even during exit_mm -> zombie when /proc/exe disappears. + if child.poll() is None: + try: child.terminate() + except ProcessLookupError: pass + try: child.wait(timeout=timeout) + except subprocess.TimeoutExpired: raise ApplyError('owned_child_stop_timeout') + + +def apply_interpreter(process): + path = process['exe'] + try: + if path.endswith(' (deleted)') or not os.path.isabs(path): raise ValueError() + info = os.stat(path) + if not stat.S_ISREG(info.st_mode) or not os.access(path, os.X_OK): raise ValueError() + except (OSError, ValueError): raise ApplyError('shim_interpreter_unavailable') + + def apply_launch(directory, baseline, environment, log): child = subprocess.Popen([baseline['shim']['exe'], str(directory / 'shim/shim.py')], env=environment, stdin=subprocess.DEVNULL, stdout=log, stderr=log, @@ -840,6 +925,10 @@ def apply_candidate(child, baseline): def apply_ready(directory, runtime, proxy, baseline, candidate, public, config, host_public, timeout=30): if candidate['exe'] != baseline['shim']['exe']: raise ApplyError('candidate_executable_changed') deadline = time.monotonic() + timeout + def remaining(): + budget = deadline - time.monotonic() + if budget <= 0: raise ApplyError('readiness_timeout') + return budget while time.monotonic() < deadline: if apply_identity(candidate['pid']) != candidate: raise ApplyError('candidate_identity_changed') if '_socket' in baseline and proxy._socket_identity(runtime) != baseline['_socket']: @@ -847,13 +936,14 @@ def apply_ready(directory, runtime, proxy, baseline, candidate, public, config, for name in ('tailscaled', 'sshd'): if apply_identity(baseline[name]['pid']) != baseline[name]: raise ApplyError('preserved_process_changed') try: - if apply_public_identity(directory, runtime, min(3, max(.01, deadline-time.monotonic()))) != public: raise ApplyError('node_identity_changed') + if apply_public_identity(directory, runtime, min(3, remaining())) != public: raise ApplyError('node_identity_changed') if proxy.inspect_config(directory, runtime) != config: raise ApplyError('proxy_configuration_changed') if bounded_code_read(directory, 'sshd/ssh_host_ed25519_key.pub', 16384) != host_public: raise ApplyError('host_public_key_changed') if not proxy._listener_owned(candidate, 18089): raise ApplyError('shim_listener_not_ready') - with socket.create_connection(('127.0.0.1', 22), timeout=min(2, max(.1, deadline-time.monotonic()))) as connection: - connection.settimeout(min(2, max(.01, deadline-time.monotonic()))) + with socket.create_connection(('127.0.0.1', 22), timeout=min(2, remaining())) as connection: + connection.settimeout(min(2, remaining())) if not connection.recv(256).startswith(b'SSH-'): raise ApplyError('ssh_not_ready') + remaining() return except ApplyError as error: if str(error) not in ('backend_not_running', 'backend_unavailable', 'public_identity_unavailable', 'shim_listener_not_ready', 'ssh_not_ready'): raise @@ -864,11 +954,12 @@ def apply_ready(directory, runtime, proxy, baseline, candidate, public, config, def apply_replace(directory, payload, transaction, journal, phase): for source, installed in list(CORE_FILES.items()) + [(MANIFEST, MANIFEST)]: - apply_journal(transaction, journal, phase=phase, file=source) + record = apply_journal_best_effort if phase == 'restore' else apply_journal + record(transaction, journal, phase=phase, file=source) target = directory / installed if target.is_symlink() or not target.is_file(): raise ApplyError('unsafe_installed_file') apply_write(target, payload[source], 0o600 if source == MANIFEST else 0o700) - apply_journal(transaction, journal, completed_file=source) + record(transaction, journal, completed_file=source) def apply_transaction(directory, runtime, transaction, journal, proxy): @@ -877,6 +968,7 @@ def apply_transaction(directory, runtime, transaction, journal, proxy): apply_syntax(candidate_payload) if apply_ts_version(old) != apply_ts_version(candidate_payload): raise ApplyError('binary_update_unsupported') baseline = apply_components(directory, runtime, proxy) + apply_interpreter(baseline['shim']) environment = apply_environment(baseline['shim']) public = apply_public_identity(directory, runtime) try: host_public = bounded_code_read(directory, 'sshd/ssh_host_ed25519_key.pub', 16384) @@ -902,27 +994,21 @@ def apply_transaction(directory, runtime, transaction, journal, proxy): if apply_bundle(directory, 'installed') != candidate_payload: raise ApplyError('installed_verification_failed') apply_journal(transaction, journal, state='applied', phase='complete') except Exception as failure: - apply_journal(transaction, journal, error=str(failure) if isinstance(failure, ApplyError) else 'transaction_failed', phase='rollback_intent') + apply_journal_best_effort(transaction, journal, error=str(failure) if isinstance(failure, ApplyError) else 'transaction_failed', state='running', phase='rollback_intent') try: - if child is not None: - # Popen.poll uses waitpid: a reaped owned child is positive exit - # evidence even if a concurrent /proc inspection was uncertain. - if child.poll() is None: - if candidate is None: raise ApplyError('candidate_identity_unknown') - apply_stop(candidate) - child.wait(timeout=2) + if child is not None: apply_stop_child(child) if not stopped: # Failed stop is not permission to start a duplicate or alter code. raise ApplyError('old_shim_stop_uncertain') if mutation: apply_replace(directory, old, transaction, journal, 'restore') restored = apply_launch(directory, baseline, environment, logfd) identity = apply_candidate(restored, baseline) - apply_journal(transaction, journal, restored=identity, phase='rollback_readiness') + apply_journal_best_effort(transaction, journal, restored=identity, phase='rollback_readiness') apply_ready(directory, runtime, proxy, baseline, identity, public, config, host_public) if apply_bundle(directory, 'installed') != old: raise ApplyError('rollback_verification_failed') - apply_journal(transaction, journal, state='failed_rolled_back', phase='complete') + apply_outcome(transaction, journal, 'failed_rolled_back', phase='complete') except Exception: - apply_journal(transaction, journal, state='rollback_failed_manual_required', phase='manual_recovery') + apply_outcome(transaction, journal, 'rollback_failed_manual_required', phase='manual_recovery') finally: os.close(logfd) @@ -941,12 +1027,21 @@ def apply_worker(directory, runtime, transaction): apply_journal(transaction, journal, state='running', worker=apply_identity(os.getpid()), phase='preflight') apply_transaction(directory, runtime, transaction, journal, proxy) except Exception as error: - apply_journal(transaction, journal, state='rollback_failed_manual_required' if journal.get('destructive') else 'rejected', error=str(error) if isinstance(error, ApplyError) else 'worker_failed') - if interrupted: apply_journal(transaction, journal, interrupted_signal=interrupted[0]) + apply_outcome(transaction, journal, 'rollback_failed_manual_required' if journal.get('destructive') else 'rejected', error=str(error) if isinstance(error, ApplyError) else 'worker_failed') + if interrupted: apply_journal_best_effort(transaction, journal, interrupted_signal=interrupted[0]) return 0 def update_apply(directory, runtime, tag=None, bundle=None, confirm=False, proxy_url=None): + context = {} + try: + return _update_apply(directory, runtime, tag, bundle, confirm, proxy_url, context) + except Exception as error: + report = apply_error_report(error) + raise ApplyError(report['error'], context.get('transaction_id')) from None + + +def _update_apply(directory, runtime, tag=None, bundle=None, confirm=False, proxy_url=None, context=None): import signal import uuid import dots_proxy as proxy @@ -963,7 +1058,8 @@ def update_apply(directory, runtime, tag=None, bundle=None, confirm=False, proxy root = apply_private(directory / APPLY_DIRECTORY, create=True) for entry in root.iterdir(): if not re.fullmatch('[0-9a-f]{32}', entry.name): raise ApplyError('unexpected_transaction_entry') - previous = apply_read_journal(apply_private(entry)) + apply_private(entry) + previous = apply_status(directory, entry.name) if previous.get('state') not in ('applied', 'failed_rolled_back', 'rejected'): raise ApplyError('prior_transaction_requires_manual_review') if bundle: @@ -972,11 +1068,12 @@ def update_apply(directory, runtime, tag=None, bundle=None, confirm=False, proxy release = fetch_release(tag, proxy_url) deadline = time.monotonic() + 60 payload = {name: download_release_asset(release, name, proxy_url, deadline) for name in CORE_FILES} - payload[MANIFEST] = json.dumps(release['manifest'], sort_keys=True).encode() + payload[MANIFEST] = release['manifest_bytes'] apply_syntax(payload) if apply_ts_version(payload) != apply_ts_version(baseline): raise ApplyError('binary_update_unsupported') identifier = uuid.uuid4().hex transaction = apply_private(root / identifier, create=True) + context['transaction_id'] = identifier journal = {'schema': 1, 'state': 'prepared', 'boot_id': apply_boot(), 'selection': 'local_bundle' if bundle else tag, 'external_ssh': 'pending', 'checksum_trust': 'consistency_only_not_publisher_authentication'} apply_journal(transaction, journal, phase='staging') @@ -989,7 +1086,7 @@ def update_apply(directory, runtime, tag=None, bundle=None, confirm=False, proxy logfd = os.open(str(transaction / 'worker.log'), os.O_WRONLY | os.O_CREAT | os.O_EXCL | os.O_NOFOLLOW, 0o600) os.close(logfd) except Exception: - apply_journal(transaction, journal, state='rejected', error='staging_failed') + apply_journal_best_effort(transaction, journal, state='rejected', error='staging_failed') raise log_fd = os.open(str(transaction / 'worker.log'), os.O_WRONLY | os.O_APPEND | os.O_NOFOLLOW | os.O_NONBLOCK) info = os.fstat(log_fd) @@ -1002,11 +1099,11 @@ def update_apply(directory, runtime, tag=None, bundle=None, confirm=False, proxy while time.monotonic() < deadline: current = apply_read_journal(transaction) if current.get('worker') or current.get('state') in APPLY_TERMINAL: - return {'state': 'started' if current.get('worker') else current['state'], 'transaction_id': identifier, + return {'state': current['state'] if current.get('state') in APPLY_TERMINAL else 'started', 'transaction_id': identifier, 'external_ssh': 'pending', 'status_command': 'dots-ts update status --id ' + identifier} - if child.poll() is not None: raise ApplyError('worker_exited_inspect_transaction_' + identifier) + if child.poll() is not None: raise ApplyError('worker_exited_inspect_transaction') time.sleep(.05) - raise ApplyError('worker_acknowledgement_timeout_inspect_transaction_' + identifier) + raise ApplyError('worker_acknowledgement_timeout_inspect_transaction') def directories(): @@ -1443,8 +1540,8 @@ def main(argv=None): try: report = (apply_status(directory, options.id) if options.update_command == 'status' else update_apply(directory, runtime, options.tag, options.bundle, options.confirm_recovery_terminal, options.proxy)) - except (ApplyError, UpdateError) as error: - report = {'error': str(error)} + except Exception as error: + report = apply_error_report(error) print(json.dumps(report, sort_keys=True)) return int('error' in report or report.get('state') in ('rejected', 'failed_rolled_back', 'rollback_failed_manual_required', 'interrupted_manual_required')) if options.command == "update" and options.update_command == "notifications": diff --git a/source-manifest.json b/source-manifest.json index 8afa793..97f0854 100644 --- a/source-manifest.json +++ b/source-manifest.json @@ -1,9 +1,9 @@ { "files": { - "dots-ts": "8b0eb5435b790cde5e2bb5acb6e9cba1a3681d57988ff2314291904fc2e3953e", + "dots-ts": "0fca9d1075c15d802d5e0c7e524a9bd0905843ef70b62623b6478c454e2b48a5", "dots_proxy.py": "2b74af2e175d12300196cab008062ba1098e12bc208d617e3fe63c23c2613390", "shim.py": "c0c55c45fbd9126e860b56d0d0359c0c87f8b39ef5d4da4a9560522452ac8c65", - "tailscale-setup.sh": "79b703df7f1c871a5ba5e081cac19d1e810f0d4886212f461b198573af77830b" + "tailscale-setup.sh": "26781177b684286bfa0b3fdf08e913bfe0edd31efbe17f821ae59e49e44ec800" }, "package_version": "0.4.0-dev", "schema_version": 1 diff --git a/tailscale-setup.sh b/tailscale-setup.sh index cfa7ce5..d40de0f 100755 --- a/tailscale-setup.sh +++ b/tailscale-setup.sh @@ -423,7 +423,7 @@ case "$state" in : > "$D/authenticated"; printf 'running: Tailscale and local SSH ready\n' # Opt-in cached notice only. Never let a notice change readiness or stdout. if [[ "$MODE" = start && -t 2 ]]; then - timeout --kill-after=1s 1s python3 "$D/dots-ts" update notifications notify >/dev/null || true + timeout --foreground --kill-after=1s 1s python3 "$D/dots-ts" update notifications notify >/dev/null || true fi ;; NeedsLogin) printf 'needs_auth: run bash %s/setup.sh login\n' "$D"; exit 10;; diff --git a/tests/test_update_apply.py b/tests/test_update_apply.py index a978793..f03542e 100644 --- a/tests/test_update_apply.py +++ b/tests/test_update_apply.py @@ -3,6 +3,7 @@ import hashlib import importlib.machinery import importlib.util +import io import json import os from pathlib import Path @@ -10,6 +11,7 @@ import subprocess import sys import tempfile +import time import unittest from unittest import mock @@ -59,6 +61,7 @@ def setUp(self): def lifecycle(self, ready=None): stack = contextlib.ExitStack(); self.addCleanup(stack.close) + stack.enter_context(mock.patch.object(tool, 'apply_interpreter')) self.stop = stack.enter_context(mock.patch.object(tool, 'apply_stop')) self.launch = stack.enter_context(mock.patch.object(tool, 'apply_launch', side_effect=[mock.Mock(poll=mock.Mock(return_value=None)), mock.Mock(poll=mock.Mock(return_value=None))])) self.identify = stack.enter_context(mock.patch.object(tool, 'apply_candidate', side_effect=[self.candidate, self.restored])) @@ -91,7 +94,7 @@ def test_readiness_failure_restores_exact_old_code(self): self.lifecycle([tool.ApplyError('readiness_timeout'), None]); self.run_transaction() self.assertEqual(self.journal['state'], 'failed_rolled_back') self.assertEqual(tool.apply_bundle(self.directory, 'installed'), self.old) - self.assertEqual(self.stop.call_args_list, [mock.call(self.baseline['shim']), mock.call(self.candidate)]) + self.stop.assert_called_once_with(self.baseline['shim']) self.assert_protected() def test_rollback_failure_is_never_success(self): @@ -215,13 +218,14 @@ def test_stop_waits_on_pidfd_without_post_signal_proc_reads(self): selector.__enter__.return_value.register.assert_called_once_with(77, tool.selectors.EVENT_READ) def test_exited_owned_candidate_skips_uncertain_proc_stop(self): - self.lifecycle([tool.ApplyError('candidate_identity_changed'), None]) + self.lifecycle() + self.identify.side_effect = [tool.ApplyError('process_identity_unknown'), self.restored] dead = mock.Mock(poll=mock.Mock(return_value=1)) self.launch.side_effect = [dead, mock.Mock()] self.run_transaction() self.assertEqual(self.journal['state'], 'failed_rolled_back') self.stop.assert_called_once_with(self.baseline['shim']) - dead.wait.assert_called_once_with(timeout=2) + dead.wait.assert_called_once_with(timeout=8) def test_real_setup_binary_version_parses(self): payload = {'tailscale-setup.sh': (ROOT / 'tailscale-setup.sh').read_bytes()} @@ -231,15 +235,28 @@ def test_real_setup_binary_version_parses(self): payload['tailscale-setup.sh'] = b'TS_VERSION="1.102.4\n' with self.assertRaises(tool.ApplyError): tool.apply_ts_version(payload) - def test_candidate_identity_failure_never_launches_duplicate(self): + def test_owned_exiting_candidate_unknown_identity_wait_success_rolls_back(self): + self.lifecycle() + self.identify.side_effect = [tool.ApplyError('process_identity_unknown'), self.restored] + live_child = mock.Mock(); live_child.poll.return_value = None + self.launch.side_effect = [live_child, mock.Mock()] + self.run_transaction() + self.assertEqual(self.journal['state'], 'failed_rolled_back') + self.assertEqual(self.launch.call_count, 2) + live_child.terminate.assert_called_once() + live_child.wait.assert_called_once_with(timeout=8) + self.stop.assert_called_once_with(self.baseline['shim']) + + def test_owned_child_timeout_never_launches_duplicate(self): self.lifecycle() self.identify.side_effect = tool.ApplyError('process_identity_unknown') live_child = mock.Mock(); live_child.poll.return_value = None + live_child.wait.side_effect = subprocess.TimeoutExpired('fixture', 8) self.launch.side_effect = [live_child] self.run_transaction() self.assertEqual(self.journal['state'], 'rollback_failed_manual_required') self.launch.assert_called_once() - self.stop.assert_called_once_with(self.baseline['shim']) + def test_worker_journal_failure_after_destructive_intent_is_manual(self): transaction = self.directory / tool.APPLY_DIRECTORY / ('a' * 32) @@ -258,6 +275,128 @@ def transaction_failure(directory, runtime, folder, record, proxy): tool.apply_worker(self.directory, self.root, transaction) self.assertEqual(tool.apply_read_journal(transaction)['state'], 'rollback_failed_manual_required') + def test_all_rollback_journal_failures_do_not_prevent_restoration(self): + self.lifecycle([tool.ApplyError('readiness_timeout'), None]) + original = tool.apply_journal + broken = [] + def journal(folder, record, **changes): + if changes.get('phase') == 'rollback_intent': broken.append(True) + if broken: raise OSError('PRIVATE disk failure') + return original(folder, record, **changes) + with mock.patch.object(tool, 'apply_journal', side_effect=journal): self.run_transaction() + self.assertEqual(tool.apply_bundle(self.directory, 'installed'), self.old) + self.assertEqual(self.launch.call_count, 2) + self.assertEqual(self.journal['state'], 'failed_rolled_back') + self.assertTrue(self.journal['journal_persistence_failed']) + self.assertNotEqual(tool.apply_read_journal(self.transaction)['state'], 'failed_rolled_back') + self.assertEqual((self.transaction / 'worker.log').read_text(), + 'update_outcome=failed_rolled_back journal_persisted=false\n') + self.assert_protected() + + def test_journal_failure_before_mutation_still_relaunches_old_shim(self): + self.lifecycle() + original = tool.apply_journal + def journal(folder, record, **changes): + if changes.get('phase') in ('replace_intent', 'rollback_intent'): + raise OSError('PRIVATE disk failure') + return original(folder, record, **changes) + with mock.patch.object(tool, 'apply_journal', side_effect=journal): self.run_transaction() + self.assertEqual(self.journal['state'], 'failed_rolled_back') + self.launch.assert_called_once() + self.assertEqual(tool.apply_bundle(self.directory, 'installed'), self.old) + + def test_nonchild_unknown_second_identity_only_accepts_readable_pidfd(self): + for readable in (True, False): + with self.subTest(readable=readable): + selector = mock.MagicMock() + selector.__enter__.return_value.select.return_value = [(mock.Mock(), 1)] if readable else [] + with mock.patch.object(tool, 'apply_identity', side_effect=[self.baseline['shim'], tool.ApplyError('process_identity_unknown')]), \ + mock.patch.object(os, 'pidfd_open', return_value=77, create=True), \ + mock.patch.object(signal, 'pidfd_send_signal', create=True) as send, \ + mock.patch.object(tool.selectors, 'DefaultSelector', return_value=selector), \ + mock.patch.object(os, 'close'): + if readable: tool.apply_stop(self.baseline['shim']) + else: + with self.assertRaises(tool.ApplyError): tool.apply_stop(self.baseline['shim']) + send.assert_not_called() + + def test_interpreter_deleted_or_nonexecutable_refused(self): + for executable in ('/python (deleted)', str(self.directory / 'dots-ts')): + with self.assertRaisesRegex(tool.ApplyError, 'shim_interpreter_unavailable'): + tool.apply_interpreter({'exe': executable}) + tool.apply_interpreter({'exe': sys.executable}) + + def test_ssl_environment_paths_preserved_only_in_memory(self): + env = b'UPSTREAM_HOST=fixture\0SSL_CERT_FILE=/certs/bundle.pem\0SSL_CERT_DIR=/certs/ca\0HOME=/private\0TOKEN=secret\0' + with mock.patch.object(tool, 'apply_identity', return_value=self.baseline['shim']), \ + mock.patch.object(Path, 'read_bytes', return_value=env): + result = tool.apply_environment(self.baseline['shim']) + self.assertEqual(result['SSL_CERT_FILE'], '/certs/bundle.pem') + self.assertEqual(result['SSL_CERT_DIR'], '/certs/ca') + self.assertNotIn('HOME', result); self.assertNotIn('TOKEN', result) + + def test_apply_public_errors_static_and_transaction_id_retained(self): + class ProxyError(RuntimeError): pass + proxy = mock.Mock(ProxyError=ProxyError) + cases = [(ProxyError('PRIVATE lock'), 'operation_busy_or_unsafe'), + (OSError('PRIVATE filesystem'), 'filesystem_or_process_error'), + (KeyError('PRIVATE manifest'), 'invalid_apply_data')] + for error, expected in cases: + def failed(*args): + args[-1]['transaction_id'] = 'a' * 32 + raise error + with mock.patch.dict(sys.modules, {'dots_proxy': proxy}), \ + mock.patch.object(tool, '_update_apply', side_effect=failed): + with self.assertRaises(tool.ApplyError) as caught: + tool.update_apply(self.directory, self.root, confirm=True) + self.assertEqual(tool.apply_error_report(caught.exception), + {'error': expected, 'transaction_id': 'a' * 32}) + + def test_unpersisted_outcome_marker_prevents_terminal_success_claim(self): + root = self.directory / tool.APPLY_DIRECTORY; root.mkdir(mode=0o700) + identifier = 'a' * 32; transaction = root / identifier; transaction.mkdir(mode=0o700) + tool.apply_journal(transaction, {'schema': 1, 'state': 'applied'}) + (transaction / 'worker.log').write_text('update_outcome=failed_rolled_back journal_persisted=false\n') + result = tool.apply_status(self.directory, identifier) + self.assertEqual(result['state'], 'interrupted_manual_required') + self.assertEqual(result['observed_outcome'], 'failed_rolled_back') + self.assertTrue(result['journal_persistence_failed']) + + def test_public_status_oserror_is_json_without_private_detail(self): + output = io.StringIO() + with mock.patch.object(tool, 'directories', return_value=(self.directory, self.root)), \ + mock.patch.object(tool, 'apply_status', side_effect=OSError('PRIVATE path')), \ + contextlib.redirect_stdout(output): + code = tool.main(['update', 'status', '--id', 'a' * 32, '--json']) + self.assertEqual(code, 1) + self.assertEqual(json.loads(output.getvalue()), {'error': 'filesystem_or_process_error'}) + + def test_terminal_worker_ack_overrides_worker_identity(self): + proxy = mock.Mock(); proxy._operation.return_value = contextlib.nullcontext() + def spawn(argv, **kwargs): + transaction = Path(argv[-1]); record = tool.apply_read_journal(transaction) + tool.apply_journal(transaction, record, state='rejected', worker=self.candidate) + return mock.Mock() + with mock.patch.dict(sys.modules, {'dots_proxy': proxy}), \ + mock.patch.object(tool.sys, 'platform', 'linux'), \ + mock.patch.object(tool.os, 'pidfd_open', create=True), \ + mock.patch.object(signal, 'pidfd_send_signal', create=True), \ + mock.patch.object(tool, 'apply_bundle', return_value=self.old), \ + mock.patch.object(tool, 'apply_syntax'), \ + mock.patch.object(tool, 'apply_boot', return_value='boot'), \ + mock.patch.object(subprocess, 'Popen', side_effect=spawn): + result = tool.update_apply(self.directory, self.root, bundle=self.transaction / 'stage', confirm=True) + self.assertEqual(result['state'], 'rejected') + self.assertEqual(result['external_ssh'], 'pending') + + def test_real_owned_child_cleanup_during_immediate_exit(self): + child = subprocess.Popen([sys.executable, '-c', 'raise SystemExit(1)']) + try: + tool.apply_stop_child(child, timeout=2) + self.assertIsNotNone(child.returncode) + finally: + if child.poll() is None: child.terminate(); child.wait(timeout=2) + def test_malformed_journal_is_structured_error(self): for payload in ({'schema': 1, 'state': []}, {'schema': 1, 'state': 'running', 'worker': []}, {'schema': 1, 'state': 'running', 'worker': {'pid': 'unsafe'}}): @@ -299,6 +438,20 @@ def test_identity_change_not_retried_as_transient(self): with self.assertRaisesRegex(tool.ApplyError, 'candidate_identity_changed'): tool.apply_ready(self.directory, self.root, self.proxy, self.baseline, self.candidate, 'same', {}, b'public-key', timeout=1) + def test_expired_readiness_cannot_report_success(self): + identities = {item['pid']: item for item in list(self.baseline.values()) + [self.candidate]} + def slow_backend(*args): + time.sleep(.02) + return 'same' + self.proxy.inspect_config.return_value = {} + self.proxy._listener_owned.return_value = True + with mock.patch.object(tool, 'apply_identity', side_effect=lambda pid: identities[pid]), \ + mock.patch.object(tool, 'apply_public_identity', side_effect=slow_backend), \ + mock.patch.object(tool.socket, 'create_connection') as connect: + with self.assertRaisesRegex(tool.ApplyError, 'readiness_timeout'): + tool.apply_ready(self.directory, self.root, self.proxy, self.baseline, self.candidate, 'same', {}, b'public-key', timeout=.01) + connect.assert_not_called() + @unittest.skipUnless(sys.platform == 'linux' and hasattr(os, 'pidfd_open'), 'Linux pidfd integration') def test_real_owned_child_sigterm_and_identity(self): child = subprocess.Popen([sys.executable, '-c', 'import time; time.sleep(30)']) diff --git a/tests/test_update_check.py b/tests/test_update_check.py index 0b6f21c..bbf765e 100644 --- a/tests/test_update_check.py +++ b/tests/test_update_check.py @@ -65,6 +65,17 @@ def test_latest_and_pinned_release_share_validated_manifest(self): self.assertEqual(fetch.call_args_list[0].args[0], tool.UPDATE_API + "/tags/v0.4.0") self.assertEqual(set(result["assets"]), set(tool.CORE_FILES) | {tool.MANIFEST}) + def test_original_manifest_bytes_preserved_and_https_proxy_refused(self): + metadata, raw = release_fixture() + original = b"\n " + raw + b"\n" + for asset in metadata["assets"]: + if asset["name"] == tool.MANIFEST: + asset["size"] = len(original) + result, _ = self.fetch(metadata=metadata, raw_manifest=original) + self.assertEqual(result["manifest_bytes"], original) + with self.assertRaisesRegex(tool.UpdateError, "invalid_proxy"): + tool.validate_proxy("https://proxy.example:8889") + def test_draft_prerelease_and_wrong_repo_tag_rejected(self): original, _ = release_fixture() for key, value in (("draft", True), ("prerelease", True), ("draft", 0), diff --git a/tests/test_update_notice.py b/tests/test_update_notice.py index 3960bd7..6dbf01a 100644 --- a/tests/test_update_notice.py +++ b/tests/test_update_notice.py @@ -109,6 +109,11 @@ def test_readonly_storage_failure_is_silent_and_successful(self): self.assertEqual(tool.main(["update", "notifications", "notify"]), 0) self.assertIsNone(tool.notification_show(self.directory)["last_notified"]) + def test_unexpected_notice_error_is_silent(self): + self.enable_and_refresh() + with mock.patch.object(tool, "package_report", side_effect=TypeError("PRIVATE error")): + self.assertEqual(self.notify(), "") + def test_storage_rejects_symlinks_hardlinks_modes_and_hostile_cache(self): self.enable_and_refresh() folder = self.directory / tool.NOTICE_DIRECTORY @@ -149,7 +154,7 @@ def test_hook_is_only_after_success_for_interactive_start_and_has_hard_timeout(s source = (ROOT / "tailscale-setup.sh").read_text() hook = source[source.index("# Opt-in cached notice only."):] self.assertIn('if [[ "$MODE" = start && -t 2 ]]', hook) - self.assertIn('timeout --kill-after=1s 1s python3 "$D/dots-ts" update notifications notify >/dev/null || true', hook) + self.assertIn('timeout --foreground --kill-after=1s 1s python3 "$D/dots-ts" update notifications notify >/dev/null || true', hook) self.assertNotIn(" refresh", hook) self.assertLess(source.index("if [[ \"${AUTH_COMMAND_STATUS:-0}\" != 0 ]]"), source.index("# Opt-in cached notice only.")) From 818f32e6fa6313c17b5e3fed47bc07d6896f0e31 Mon Sep 17 00:00:00 2001 From: abruption Date: Wed, 21 Oct 2026 13:26:52 +0900 Subject: [PATCH 4/4] docs: record independent review and live update rollback validation --- CHANGELOG.md | 2 +- docs/usage.ja.md | 2 ++ docs/usage.ko.md | 2 ++ docs/usage.md | 2 ++ docs/usage.zh-CN.md | 2 ++ docs/validation/v0.4.0-update-trial.md | 47 ++++++++++++++++++++++++++ 6 files changed, 56 insertions(+), 1 deletion(-) create mode 100644 docs/validation/v0.4.0-update-trial.md diff --git a/CHANGELOG.md b/CHANGELOG.md index 59bf69d..882bf2c 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -11,7 +11,7 @@ - Add detached manual updates from fixed releases or explicitly reviewed local bundles, with code-only backups, a persistent transaction journal and bounded shim rollback (#30). -This development work is undergoing validation; live deployment evidence will be recorded separately. +The reviewed `0.4.0-dev` candidate passed normal update, intentional startup-failure rollback, checksum-refusal and fresh external SSH/ping checks in Dots. See the [dated validation report](docs/validation/v0.4.0-update-trial.md) for the independent review, CI evidence and limits. This is not a published v0.4.0 release. ## v0.3.1 — 2026-10-19 diff --git a/docs/usage.ja.md b/docs/usage.ja.md index 82aff84..5c867ff 100644 --- a/docs/usage.ja.md +++ b/docs/usage.ja.md @@ -223,6 +223,8 @@ JSON では `package.declared_version`、`package.source_revision`、`package.in 2. 必要なら確認済みのコードバンドルを復元して起動し、マニフェスト、ローカルの準備状態、新しい外部 SSH 接続を厳密なホスト鍵検証付きで確認します。古いヘルパーでは、別途確認済みのソースランナーによるマニフェスト確認が必要な場合があります。 3. worker が動いておらず、インストール先の `.operation.lock` を取得した状態で、確認済みのトランザクションディレクトリ全体を `.update-transactions/` の外にある、自分が所有する権限 700 の `.update-recovery-archive/` へ移します。記録とバックアップは一緒に残してください。これで未完了トランザクションの検索対象から外れます。apply の制限を回避するために、実行中または未確認の記録を移してはいけません。 +確認済みの開発候補を 2026-10-21 に Dots で試験しました。通常の適用、意図的な起動失敗からのロールバック、外部 SSH の確認結果と範囲は[更新の検証記録](validation/v0.4.0-update-trial.md)を参照してください。 + ## v0.3.1 ヘルパーと任意のアプリケーションプロキシ diff --git a/docs/usage.ko.md b/docs/usage.ko.md index c1a5388..f5daaae 100644 --- a/docs/usage.ko.md +++ b/docs/usage.ko.md @@ -172,6 +172,8 @@ setup과 proxy apply가 사용하는 작업 잠금을 유지한 채 소스 파 2. 필요하면 검토된 코드 묶음으로 복원해 기동하고, 명세·로컬 준비 상태·새 외부 SSH 연결을 확인합니다. 구버전 도구에서는 별도로 검토한 소스 실행기로 명세를 확인해야 할 수 있습니다. 3. 작업 프로세스가 없고 설치 경로의 `.operation.lock`을 확보한 상태에서, 검토를 마친 작업 디렉터리 전체를 `.update-transactions/` 바깥의 본인 소유·권한 700인 `.update-recovery-archive/`로 옮깁니다. 기록과 백업을 함께 보존하세요. 이렇게 보관하면 미완료 작업 검색에서 제외됩니다. apply 차단을 풀려고 실행 중이거나 검토하지 않은 작업을 옮기면 안 됩니다. +검토된 개발 후보를 2026-10-21에 Dots에서 시험했습니다. 정상 적용, 의도적인 실패 후 원복, 외부 SSH 확인 결과와 한계는 [날짜별 업데이트 검증 기록](validation/v0.4.0-update-trial.md)을 참고하세요. + ## v0.3.1 도구와 선택적 애플리케이션 프록시 diff --git a/docs/usage.md b/docs/usage.md index f135b31..202c301 100644 --- a/docs/usage.md +++ b/docs/usage.md @@ -223,6 +223,8 @@ After an interrupted transaction, a manual recovery can be concluded without del 2. Restore a reviewed code bundle if needed, start it and verify its manifest, local readiness and a new external strict-host-key SSH connection. A legacy helper may require a separately reviewed source runner to inspect its manifest. 3. With the installation's `.operation.lock` held and no active worker, move the entire reviewed transaction directory to an owned, mode-700 `.update-recovery-archive/` directory outside `.update-transactions/`. Preserve the journal and backups together. This manual archive removes it from the pending-update scan; never archive an active or unreviewed transaction just to unblock apply. +The reviewed development candidate was tested in Dots on 2026-10-21. See the [dated update validation report](validation/v0.4.0-update-trial.md) for normal apply, intentional failure/rollback, external SSH evidence and limits. + ## v0.3.1 helpers and optional application proxies diff --git a/docs/usage.zh-CN.md b/docs/usage.zh-CN.md index 6e74786..f434aa5 100644 --- a/docs/usage.zh-CN.md +++ b/docs/usage.zh-CN.md @@ -223,6 +223,8 @@ JSON 将 `package.declared_version`、`package.source_revision`、`package.integ 2. 必要时恢复经过审核的代码包并启动，验证清单、本地就绪状态以及一次新的外部严格主机密钥 SSH 连接。旧版工具可能需要单独审核的源码运行器来检查清单。 3. 确认没有活动 worker，并持有安装目录的 `.operation.lock` 后，将整个已审核的事务目录移动到 `.update-transactions/` 之外、属于当前用户且权限为 700 的 `.update-recovery-archive/`。日志和备份应一并保留。归档后，该事务不再阻止新更新；不要为了绕过限制而归档仍在运行或未经审核的事务。 +已于 2026-10-21 在 Dots 中测试经过审核的开发候选版本。正常应用、故意启动失败后的回滚、外部 SSH 验证及其范围，请参阅[更新验证记录](validation/v0.4.0-update-trial.md)。 + ## v0.3.1 辅助工具与可选应用代理 diff --git a/docs/validation/v0.4.0-update-trial.md b/docs/validation/v0.4.0-update-trial.md new file mode 100644 index 0000000..5fc5f15 --- /dev/null +++ b/docs/validation/v0.4.0-update-trial.md @@ -0,0 +1,47 @@ +# v0.4.0 development update validation — 2026-10-21 + +This is evidence for issues #28–#30, not a published release or an automatic VM recovery claim. + +## Candidate and review + +- Source candidate: `fe436ec49a9e8f6e7e645bfb039784c482c39178`, package `0.4.0-dev`; Tailscale remains `1.102.4`. +- Based on package metadata PR #41. The four executable sources and `source-manifest.json` were transferred as a reviewed local bundle. No test tag or public GitHub release was created. +- Claude Code **2.1.305**, requested and response-metadata-confirmed model **`claude-opus-5-5`**, reviewed a frozen source/test/document bundle with tools disabled and no session persistence. +- The first review found three required fixes: exiting-child cleanup, rollback despite journal failures, and structured error responses. These were corrected with regression tests. The follow-up review reported no remaining required fixes before the live trial. It was source review, not execution by Claude. +- Linux CI passed **256 tests on Python 3.11 and 3.13**, plus ShellCheck and manifest checks: [run 39535102142](https://github.com/abruption/dots-tailscale/actions/runs/39535102142). +- An earlier Linux test found a real `/proc/stat` versus `/proc/exe` exit race. Nonchild termination now waits on its pinned pidfd; owned candidate cleanup uses its unreaped `Popen` handle. This was not dismissed as a flaky test. + +## Live prerequisites + +The user explicitly authorized a Dots trial and had an independent cloud terminal available. Before the trial, an unrelated connection outage required a manual start; the boot ID had changed and the installation still matched v0.3.0. That incident belongs to #20 and occurred before candidate deployment. + +All four installed v0.3.0 source hashes matched independently obtained Git tag contents. Only after rechecking that match, the corresponding manifest was provisioned as an explicit legacy metadata migration. Expected hashes were not generated from arbitrary installed bytes. The source runner's candidate manifest was also verified. + +The target had Linux 6.18.55, Python 3.13.5, an executable `/usr/bin/openssl`, no leftover target `.new` files, a valid public SSH host key file, and identifiable shim/daemon/SSH processes. Separate inert single-thread and multi-thread child probes confirmed pidfd readability after SIGTERM. + +## Results + +| Check | Observed result | +|---|---| +| Normal local-bundle update | Detached `started` acknowledgment, then `applied`; installed `0.4.0-dev` sources all match the reviewed manifest | +| Intentional candidate startup failure | A separately labelled bundle with a syntax-valid, immediately exiting shim produced `failed_rolled_back` and status exit 1 | +| Exact restoration | All four source files and manifest after rollback equal the reviewed healthy candidate, not the failure fixture | +| Checksum mismatch | `untrusted_or_mixed_package`, exit 1, no new transaction and no code replacement | +| External verification | New SSH connections using strict host-key checking and DERP ping succeeded after both normal apply and rollback | +| Identity and configuration | Same public node identity, public SSH host key, tailscaled and sshd identities, and proxy configuration throughout the trial | +| Protected files | SSH private key, CA private key and authorized-keys inode/size/mtime metadata remained unchanged; their contents and node-state contents were not read or hashed for this verification | +| Live release check through explicit platform proxy | Structured `unsupported_release`, exit 2: the published v0.3.1 release lacks the new manifest, so it was correctly refused as a verified update bundle | +| Monitoring afterward | A new continuous watchdog run was started for the new boot, with 60-second sampling and separate fresh external verification | + +The normal-apply verification was recorded at **13:20:07 KST**, and the rollback verification at **13:21:51 KST**. Local transaction status intentionally continues to say `external_ssh=pending`; external success is separate evidence, not a field fabricated by the updater. + +## Limits + +- The live path used a reviewed local development bundle. Successful download and application of a future manifest-bearing GitHub release remain to be checked after such a release is published. Invalid metadata, redirects, response limits and network failures are covered by offline tests. +- No deliberate VM destruction, power loss, SIGKILL, credential reset, or unknown-process termination was performed. Interrupted journals and write failures were exercised with controlled fixtures; they require manual review in production. +- `applied` proves manifest consistency and local readiness. Fresh external SSH/ping supply additional connectivity evidence for this trial, not all applications or future environments. +- The updater handles the fixed four-source bundle and restarts only shim. It refuses Tailscale binary-version changes and unknown/mixed legacy installations. +- After a crash, inspect leftover `.new` files and incomplete transactions before a manual retry. Preserve evidence and follow the documented archive/recovery procedure; do not clear a block blindly. +- The experimental watchdog cannot start itself after VM replacement. Its new-boot run is separate from the retired prior-boot ledger; no cross-VM recovery-budget continuity is claimed. + +The healthy development candidate remained installed after the trial. No merge, release, or tag publication was performed as part of validation.