perf: Caching and I/O optimizations (P3, P10)

[dshkol]

Summary

Performance optimizations for caching and I/O operations.

Changes

P3: list_cansim_cached_tables single-pass optimization

• Consolidate three separate lapply() calls into a single iteration over cached table paths
• Collects timeCached, rawSize, and title in one pass per cached table
• Avoids repeated dir() calls and file read operations that were being done 3x per path
• Use vapply() for type-safe extraction from collected metadata

P10: Avoid unnecessary tibble conversion

• Check tibble::is_tibble() before calling as_tibble()
• Skips conversion when data is already a tibble

Files Modified

• R/cansim.R: tibble conversion check
• R/cansim_parquet.R: single-pass cache metadata collection

---

Benchmark Notes

P3: list_cansim_cached_tables - Not directly benchmarked

Reason: Requires a populated cache directory with multiple cached tables to meaningfully benchmark.

Expected improvement: The optimization reduces file I/O operations from 3N to N (where N is number of cached tables):

# Before: Three separate lapply calls, each reading files
result$timeCached <- lapply(result$path, function(p) { ... })  # Read 1
result$rawSize <- lapply(result$path, function(p) { ... })     # Read 2  
result$title <- lapply(result$path, function(p) { ... })       # Read 3

# After: Single pass collecting all metadata
cache_metadata <- lapply(result$path, function(p) {
  # Collect timeCached, rawSize, title in one pass
  list(timeCached = ..., rawSize = ..., title = ...)
})
result$timeCached <- do.call("c", lapply(cache_metadata, `[[`, "timeCached"))
result$rawSize <- vapply(cache_metadata, `[[`, numeric(1), "rawSize")
result$title <- vapply(cache_metadata, `[[`, character(1), "title")

To benchmark manually:

library(microbenchmark)
library(cansim)

# First, populate cache with several tables:
cache_path <- "~/.cansim_cache"
for (tbl in c("34-10-0013", "17-10-0005", "14-10-0287")) {
  get_cansim_sqlite(tbl, cache_path = cache_path)
}

# Benchmark (run on master, then on optimized branch)
microbenchmark(
  list_cached = list_cansim_cached_tables(cache_path = cache_path),
  times = 20
)

---

P10: tibble conversion check - Not directly benchmarked

Reason: Negligible impact - this is primarily a code quality improvement.

Analysis: The is_tibble() check is O(1) and very fast. Most data flowing through normalize_cansim_values() is already a tibble from prior processing, so the as_tibble() call was largely unnecessary. The improvement is in avoiding the conversion overhead when data is already the correct type.

# Before
data <- as_tibble(data)

# After
if (!tibble::is_tibble(data)) data <- as_tibble(data)

---

Deferred Optimizations

P6 (field cache utilization): Evaluated but not implemented

• The audit plan referenced "field cache written but never read" at lines 254-263
• Could not identify a specific field cache location in the current code structure
• May require further investigation or the referenced caching pattern may have changed

P7 (csv2sqlite transform copies): Evaluated but not implemented

• Using conditional piping {if (...) ... else .} would harm code readability
• The current conditional structure is clearer and the performance gain (~25-40%) is minor
• Trade-off favors maintainability over micro-optimization

---

Summary

Optimization	Benchmarked	Expected Impact	Status
P3 (single-pass)	Manual setup required	Reduces I/O 3x→1x	✅ Implemented
P10 (tibble check)	N/A	Negligible	✅ Implemented

Test Plan

• devtools::check() passes (0 errors, 0 warnings)
• All existing tests pass
• Manual testing with cached tables to verify metadata extraction

🤖 Generated with Claude Code