Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ A terminal application for interacting with Todoist, built in Rust with a modern
## Features

- **Interactive TUI Interface** - Beautiful terminal user interface with ratatui
- **Local Data Caching** - Fast, responsive UI with in-memory SQLite storage
- **Local Data Caching** - Fast, responsive UI backed by a local SQLite cache that survives restarts
- **Smart Sync** - Automatic sync on startup and manual refresh with 'r'
- **Project Management** - Browse projects with hierarchical display
- **Task Management** - View, navigate, complete, and create tasks
Expand Down Expand Up @@ -128,8 +128,8 @@ Essential keyboard shortcuts to get started:
## How It Works

Terminalist uses a smart sync mechanism:
- **Fast Startup**: In-memory SQLite database for instant loading
- **Auto Sync**: Syncs with Todoist on startup and every 5 minutes
- **Fast Startup**: Cached data from the last run is shown instantly while the sync runs
- **Startup Sync**: Syncs with Todoist once on startup, in the background
- **Manual Sync**: Press `r` to force refresh from Todoist
- **Real-time Updates**: Create, modify, and delete tasks/projects immediately

Expand Down
5 changes: 3 additions & 2 deletions docs/ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,14 +58,15 @@ src/

### Local Storage
- Data is cached locally in a **file-backed SQLite database**
- Database is recreated from scratch on each startup by syncing with the backend
- Database persists between runs and is treated as a disposable cache; the backend stays authoritative
- The schema revision is stamped in `PRAGMA user_version`; a mismatch drops and rebuilds every table rather than migrating
- Uses Sea-ORM for type-safe database operations
- Repository pattern provides clean data access layer
- UUID-based primary keys for robust entity management

### Sync Behavior
- **First Run**: Automatically syncs all data from Todoist
- **Startup**: Loads local data instantly, then syncs in background if data is older than 5 minutes
- **Startup**: Shows the cached data immediately, then syncs in the background; a failed sync leaves the cached view in place
- **Manual Sync**: Press `r` to force refresh from Todoist API
- **Sync Indicators**: Sync progress is shown during operations

Expand Down
2 changes: 1 addition & 1 deletion docs/DEVELOPMENT.md
Original file line number Diff line number Diff line change
Expand Up @@ -72,7 +72,7 @@ This project uses the following Rust crates (see `Cargo.toml` for exact versions
- `ratatui = "0.29"` - Terminal UI framework
- `crossterm = "0.29"` - Cross-platform terminal handling
- `tokio = "1.x"` - Async runtime
- `sqlx = "0.8"` - Database toolkit with SQLite support
- `sea-orm = "1.1"` - ORM with SQLite support (via its `sqlx-sqlite` feature)
- `serde` - Serialization/deserialization
- `chrono = "0.4"` - Date and time handling
- `anyhow = "1.0"` - Error handling
Expand Down
72 changes: 70 additions & 2 deletions src/storage.rs
Original file line number Diff line number Diff line change
@@ -1,10 +1,18 @@
use anyhow::{Context, Result};
use sea_orm::{ConnectOptions, ConnectionTrait, Database, DatabaseConnection, DbBackend, Schema, Statement};
use sea_orm::{
ConnectOptions, ConnectionTrait, Database, DatabaseConnection, DbBackend, Schema, Statement, TransactionTrait,
};
use std::path::PathBuf;
use std::time::Duration;

use crate::entities::{backend, label, project, section, task, task_label};

/// Schema revision of the local cache. Bump this whenever an entity definition changes.
///
/// The database is a disposable cache, not a source of truth, so a mismatch drops every
/// table and rebuilds from scratch instead of running a migration.
const SCHEMA_VERSION: i32 = 1;

/// Local storage manager for Todoist data
pub struct LocalStorage {
pub conn: DatabaseConnection,
Expand All @@ -26,7 +34,8 @@ impl LocalStorage {
/// Initialize the local storage with the application SQLite database.
///
/// The database is retained between runs and refreshed by the sync layer. Deleting it
/// here would invalidate connections held by another running Terminalist process.
/// here would invalidate connections held by another running Terminalist process, so a
/// schema change is handled by dropping the cached tables instead. See [`SCHEMA_VERSION`].
pub async fn new(_debug_mode: bool) -> Result<Self> {
let db_path = Self::get_db_path()?;
Self::new_at(db_path).await
Expand Down Expand Up @@ -60,11 +69,70 @@ impl LocalStorage {
.await?;

let storage = LocalStorage { conn };
storage.discard_stale_schema().await?;
storage.init_schema().await?;

Ok(storage)
}

/// Drop the cached tables when the file was written by a different schema revision.
async fn discard_stale_schema(&self) -> Result<()> {
let row = self
.conn
.query_one(Statement::from_string(
DbBackend::Sqlite,
"PRAGMA user_version;".to_owned(),
))
.await?;
let version = match row {
Some(row) => row.try_get::<i32>("", "user_version")?,
None => 0,
};

if version == SCHEMA_VERSION {
return Ok(());
}

// A transaction pins one pooled connection, so the pragma below applies to the drops.
let txn = self.conn.begin().await?;

// Asking the file what it holds beats hardcoding a list: it also clears out tables from
// revisions that no longer have a matching entity.
//
// Reverse creation order drops children before parents. Dropping a parent first leaves
// the child holding a foreign key to a missing table, and its own drop then fails on
// resolving that reference. sqlite_master lists objects in creation order, and
// init_schema creates parents first, so walking it backwards is dependency order.
let tables = txn
.query_all(Statement::from_string(
DbBackend::Sqlite,
"SELECT name FROM sqlite_master WHERE type = 'table' AND name NOT LIKE 'sqlite_%' \
ORDER BY rowid DESC;"
.to_owned(),
))
.await?;

for table in tables {
let name = table.try_get::<String>("", "name")?;
txn.execute(Statement::from_string(
DbBackend::Sqlite,
format!("DROP TABLE IF EXISTS \"{name}\";"),
))
.await?;
}

// Not a bind parameter: SQLite only accepts a literal here, and the value is a constant.
txn.execute(Statement::from_string(
DbBackend::Sqlite,
format!("PRAGMA user_version = {SCHEMA_VERSION};"),
))
.await?;

txn.commit().await?;

Ok(())
}

/// Initialize database schema
async fn init_schema(&self) -> Result<()> {
let backend = self.conn.get_database_backend();
Expand Down
71 changes: 71 additions & 0 deletions tests/storage/db.rs
Original file line number Diff line number Diff line change
Expand Up @@ -29,3 +29,74 @@ async fn test_local_storage_creation() {
second.conn.close().await.expect("second connection should close");
std::fs::remove_file(db_path).expect("test database should be removed");
}

#[tokio::test]
async fn test_stale_schema_version_rebuilds_cache() {
let db_path = std::env::temp_dir().join(format!("terminalist-schema-{}.db", uuid::Uuid::new_v4()));

let storage = LocalStorage::new_at(db_path.clone())
.await
.expect("LocalStorage should be created successfully");
storage
.conn
.execute(Statement::from_string(
DbBackend::Sqlite,
"INSERT INTO backends \
(uuid, backend_type, name, is_enabled, credentials, settings) \
VALUES ('00000000-0000-0000-0000-000000000002', 'test', 'Test', 1, '{}', '{}')"
.to_owned(),
))
.await
.expect("cached row should be inserted");
// A table from a revision that no longer has a matching entity.
storage
.conn
.execute(Statement::from_string(
DbBackend::Sqlite,
"CREATE TABLE retired_entity (uuid TEXT PRIMARY KEY);".to_owned(),
))
.await
.expect("legacy table should be created");
// Pretend the file was written by an older revision of the entities.
storage
.conn
.execute(Statement::from_string(
DbBackend::Sqlite,
"PRAGMA user_version = 0;".to_owned(),
))
.await
.expect("user_version should be writable");
storage.conn.close().await.expect("connection should close");

let reopened = LocalStorage::new_at(db_path.clone())
.await
.expect("stale cache should be rebuilt, not rejected");
let count = reopened
.conn
.query_one(Statement::from_string(
DbBackend::Sqlite,
"SELECT COUNT(*) AS count FROM backends;".to_owned(),
))
.await
.expect("backends table should exist after the rebuild")
.expect("count query should return a row")
.try_get::<i64>("", "count")
.expect("count should be readable");
assert_eq!(count, 0, "a stale cache must be dropped, not kept");

let leftovers = reopened
.conn
.query_all(Statement::from_string(
DbBackend::Sqlite,
"SELECT name FROM sqlite_master WHERE type = 'table' AND name = 'retired_entity';".to_owned(),
))
.await
.expect("sqlite_master should be queryable");
assert!(
leftovers.is_empty(),
"tables without a matching entity must be dropped too"
);

reopened.conn.close().await.expect("connection should close");
std::fs::remove_file(db_path).expect("test database should be removed");
}
Loading