Repository navigation
Conversation
… link closure reasons
…startup The 17 JSON files under <rnfs dir>/state/ are replaced by state/rnfs.db (the standard sqlite3 module; no new dependency). Nothing changes on the wire: request paths, messages, CIDs, DAG bytes and provider and name records are as before, and 0.1.4 and new nodes find, fetch and resolve each other's content and names. rnfs/statedb.py is the database layer: one shared connection per state directory (each store holds a counted reference), WAL mode with busy_timeout, and files created 0600. The schema is a list of steps counted by PRAGMA user_version. Startup runs PRAGMA quick_check and refuses, with what to do, a damaged database, one from a newer version, or a Python without _sqlite3. transaction(durable) is the only way to write and its lock is the innermost lock of all: FULL for state saved on each change, NORMAL for batched saves. SQLite failures raise StateDbError, an OSError. Migration: at startup each legacy file still present is read with read_json and the old shape check. Its importer checks every record as the old loader did, and inserts them in one transaction that also records the file in the migrated table. The file is then renamed <name>.migrated and the directory synced. A crash between the commit and the rename only renames at the next start, so no file is imported twice or by half. Every format written by 0.1.3, 0.1.4 and dev is accepted. Damaged files are handled as before: moved to .damaged, except an unreadable pins.json or a damaged names.json still stops startup. The upgrade is one-way. Stores keep their state in memory as the read path and write only what changed. A pin or release is one row, committed durably after blocks.sync(), and memory is rolled back if the commit fails. Cache records, provider records, publish state, the name floor, keepers and the LXMF inbox and outbox track changed keys. Every validation and cap of the JSON loaders still applies at load; malformed rows are dropped with a warning and deleted. The reconcile logic of names, follow and LXMF is unchanged. Measured: Storage.pin() takes 2.5 ms at any pin count (was 5 to 14 ms at 100 to 20,000 pins). A provider save with one change in 100k records takes 0.02 ms (was 155 ms), a cache save with one change in 10k 0.02 ms (was 13 ms). Startup with 20k pins and 100k records takes 354 ms (was 463 ms). Behaviour changes: batched saves are no longer synced at each save, so a power loss may lose the last one (never part of one). A state directory that can't be written stops startup with a clear message. An unparsable providers.json is kept as .damaged. published.json IDs must be 16 bytes. A malformed names row stops startup, as a malformed names.json did. Tests: existing tests read the database through new helpers in tests/support.py and close their stores so Windows can delete temp dirs. The damaged-state tests now cover damaged legacy files and damaged rows. New: test_statedb.py; test_migrate_*.py per store; test_upgrade_014.py, which loads tests/fixtures/rnfs-0.1.4/, a node directory written by 0.1.4's own code; its blocks are rebuilt by the test, not checked in. The smoke test now checks the database holds a pin and imported a 0.1.4 file; sqlite3 is named in the PyInstaller spec. Docs: design.md (state database, migration, lock order, State files), specification.md (storage and jobs wording), CHANGELOG.md, README.md. Test and lint caches added to .gitignore.
This commit introduces a new test suite for the deny list feature, covering various aspects such as entry parsing, local entry persistence, list management, and interactions with offline nodes. The tests ensure that denied content is correctly handled, including scenarios for denied roots, DAGs, and stored content. Additionally, the CLI commands for managing deny lists are validated against an offline node's API. This extensive testing framework aims to enhance the reliability and robustness of the deny list implementation.
This commit introduces a new test suite located in `tests/test_public.py`, which thoroughly tests the public API functionalities of both the `rnfs.Node` and `rnfs.Client` classes. The tests cover various aspects including data addition, retrieval, error handling, name management, and job processing. The suite ensures that both embedded and client-server interactions behave consistently, validating the integrity and reliability of the API.
- Introduced support for keeping connections open for one request after a proof request, allowing clients to send their requests on the same connection. - Updated the ApiHandler to manage connection states based on proof requests and added logic to handle keep-alive headers. - Modified the ApiConnection to ensure that each request can independently prove the daemon, maintaining security and connection integrity. - Implemented tests to verify the behavior of connection handling during proof requests and ensure that old tokens are not sent after a daemon restart. - Enhanced LXMF resource delivery checks to refuse multi-segment resources, preventing potential memory issues. - Updated documentation to reflect changes in announce validity and resource handling.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Release 0.1.5: state in SQLite, deny lists, debug log, public API for apps, and fixes from the pre-release security audit. See CHANGELOG.md, section 0.1.5.