Repository navigation
feat: implement potok push/pull - #66
Merged
Merged
Conversation
Encrypt+upload and download+decrypt for a registered vault - the convergence point for everything built so far (blob/manifest endpoints, client crypto, vault indexing). push <name>: resolves (or lazily creates, on first use - nothing else creates a vault server-side yet) the server-side vault, derives the AES key from the stored passphrase and the vault's kdf_salt, scans the local directory, and for every file encrypts it and uploads the ciphertext as a content-addressed blob. Builds a manifest listing path/blob-id/size/mtime per file, encrypts it, and PUTs it with the right expected_generation (0 on a fresh vault). A 409 (someone else pushed since) is reported as "pull first, then push again" rather than silently lost. pull <name>: the mirror - fetches and decrypts the manifest, then downloads and decrypts every blob, writing it back under the registered local path with directory structure preserved. Resolves the kdf_salt gap noted after unit 7/8: vaults.kdf_salt (migration 0003), a random 16-byte value generated in store.CreateVault and returned via VaultByName/ListVaults. internal/client/transport now actually is what its doc comment always claimed - the HTTP client for the Potok API. Moved the request plumbing (Client.Request/RequestBody/RequestJSON/RequestJSONBody, ReadMessage) and the vault/blob/manifest calls (FetchVault, GetOrCreateVault, UploadBlob, DownloadBlob, CurrentManifestGeneration, FetchManifest, PutManifest) out of the commands package, where they'd been living as free functions taking serverURL/apiKey on every call. Every command that talks to the server (init, doctor, remote-list, remote-delete, push, pull) goes through transport.Client now. internal/protocol/manifest.go defines Manifest/ManifestFile, the plaintext JSON shape push and pull agree on before encryption - the server never decodes it. Tested: push_pull_test.go spins up the real HTTP handler/store/ blobstore (not mocks) and drives it through the same helpers push/ pull use - full round trip with nested paths and binary content, wrong passphrase failing cleanly, first-push auto-creating the remote vault, generation advancing across two pushes. store_test.go covers unique-per-vault salts and VaultByName returning the same salt CreateVault generated. Also manually verified against a real running potokd with two separate local configs simulating two devices sharing a passphrase - diff -r on the push/pull round trip came back identical. Known gaps, deliberately left for later: no incremental push (every file is re-encrypted and re-uploaded on every run - server-side dedup means this only costs bandwidth, not storage), pull's writes aren't atomic (unlike blobstore.Put's temp-file-then-rename), and conflict handling stops at "409 means pull first" with no merge.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Encrypt+upload and download+decrypt for a registered vault - the convergence point for everything built so far (blob/manifest endpoints, client crypto, vault indexing).
push : resolves (or lazily creates, on first use - nothing else creates a vault server-side yet) the server-side vault, derives the AES key from the stored passphrase and the vault's kdf_salt, scans the local directory, and for every file encrypts it and uploads the ciphertext as a content-addressed blob. Builds a manifest listing path/blob-id/size/mtime per file, encrypts it, and PUTs it with the right expected_generation (0 on a fresh vault). A 409 (someone else pushed since) is reported as "pull first, then push again" rather than silently lost.
pull : the mirror - fetches and decrypts the manifest, then downloads and decrypts every blob, writing it back under the registered local path with directory structure preserved.
Resolves the kdf_salt gap noted after unit 7/8: vaults.kdf_salt (migration 0003), a random 16-byte value generated in store.CreateVault and returned via VaultByName/ListVaults.
internal/client/transport now actually is what its doc comment always claimed - the HTTP client for the Potok API. Moved the request plumbing (Client.Request/RequestBody/RequestJSON/RequestJSONBody, ReadMessage) and the vault/blob/manifest calls (FetchVault, GetOrCreateVault, UploadBlob, DownloadBlob,
CurrentManifestGeneration, FetchManifest, PutManifest) out of the commands package, where they'd been living as free functions taking serverURL/apiKey on every call. Every command that talks to the server (init, doctor, remote-list, remote-delete, push, pull) goes through transport.Client now.
internal/protocol/manifest.go defines Manifest/ManifestFile, the plaintext JSON shape push and pull agree on before encryption - the server never decodes it.
Tested: push_pull_test.go spins up the real HTTP handler/store/ blobstore (not mocks) and drives it through the same helpers push/ pull use - full round trip with nested paths and binary content, wrong passphrase failing cleanly, first-push auto-creating the remote vault, generation advancing across two pushes. store_test.go covers unique-per-vault salts and VaultByName returning the same salt CreateVault generated. Also manually verified against a real running potokd with two separate local configs simulating two devices sharing a passphrase - diff -r on the push/pull round trip came back identical.
Known gaps, deliberately left for later: no incremental push (every file is re-encrypted and re-uploaded on every run - server-side dedup means this only costs bandwidth, not storage), pull's writes aren't atomic (unlike blobstore.Put's temp-file-then-rename), and conflict handling stops at "409 means pull first" with no merge.