Skip to content

perf: optimize TTL expiration with runtime.nanotime() and consolidated type assertions#21

Open
analytically wants to merge 2 commits intogo-pkgz:masterfrom
analytically:feature/nanotime
Open

perf: optimize TTL expiration with runtime.nanotime() and consolidated type assertions#21
analytically wants to merge 2 commits intogo-pkgz:masterfrom
analytically:feature/nanotime

Conversation

@analytically
Copy link
Copy Markdown
Contributor

@analytically analytically commented Oct 1, 2025

Replace time.Time with int64 nanotime values for expiration tracking, simplifying TTL checks and reducing memory footprint. 19-30% faster.

Performance improvements (benchstat with n=3 on Apple M1):

  • LRU_Rand_NoExpire: 219.6ns → 173.9ns (-20.8%)
  • LRU_Freq_NoExpire: 218.5ns → 176.2ns (-19.4%)
  • LRU_Rand_WithExpire: 251.8ns → 175.4ns (-30.3%)
  • LRU_Freq_WithExpire: 250.1ns → 176.7ns (-29.3%)

Changes:

  • Use runtime.nanotime() via //go:linkname for fastest monotonic time
  • Store expiration as int64 nanoseconds instead of time.Time
  • Consolidate type assertions (2+ calls → 1 per operation)
  • Replace time.After() with direct integer comparison
  • Remove GetExpiration()

Benefits:

  • Performance: 20-30% faster across all operations
  • Memory: 16 bytes saved per entry (67% reduction in time storage)
  • Correctness: Immune to system clock adjustments (NTP, DST, manual changes)
  • Throughput: Increased from ~4.6M to ~5.7M ops/sec per goroutine

Note: runtime.nanotime() is an internal Go API that may change in future
versions. Trade-off accepted for significant performance and correctness benefits.

…d type assertions

Replace time.Time with int64 nanotime values for expiration tracking,
simplifying TTL checks and reducing memory footprint. 19-30% faster.

Signed-off-by: Mathias Bogaert <mathias.bogaert@gmail.com>
@analytically analytically requested a review from umputun as a code owner October 1, 2025 17:34
@analytically
Copy link
Copy Markdown
Contributor Author

@paskal could you review?

@umputun umputun requested review from Copilot and paskal October 1, 2025 17:50
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR optimizes cache performance by replacing time.Time with int64 nanotime values for TTL expiration tracking, achieving 19-30% performance improvements across all operations.

Key changes:

  • Use runtime.nanotime() via //go:linkname for fastest monotonic time access
  • Store expiration as int64 nanoseconds instead of time.Time structures
  • Consolidate type assertions to reduce redundant operations

Reviewed Changes

Copilot reviewed 4 out of 5 changed files in this pull request and generated 2 comments.

File Description
v3/cache.go Core optimization replacing time.Time with int64 nanotime, consolidating type assertions, and updating all expiration logic
v3/options.go Convert TTL parameter from time.Duration to int64 nanoseconds
v3/go.mod Update Go version to 1.25 and remove unused dependency
v3/cache_test.go Remove unused import and test function for removed dependency

Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

Signed-off-by: Mathias Bogaert <mathias.bogaert@gmail.com>
Copy link
Copy Markdown
Collaborator

@paskal paskal left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the effort, but I don't think we should merge this PR. Here are the concerns:

  1. //go:linkname to internal runtime functionruntime.nanotime() is an unexported internal function. Using //go:linkname to access it is fragile and explicitly discouraged by the Go team, who have been actively locking down linkname access. This could break on any Go version upgrade without warning.

  2. Breaking API changes — The PR removes GetExpiration(key K) (time.Time, bool) from the public interface and drops simplelru.LRUCache interface compliance (along with the hashicorp/golang-lru/v2 dependency). Users relying on these would break.

  3. Incompatible with testing/synctest — Go 1.24 introduced testing/synctest which provides a fake clock that intercepts time.Now() automatically. This allows deterministic testing of TTL behaviour without any library changes. Switching to runtime.nanotime() would make the cache immune to this fake clock, preventing users (and our own tests) from using this facility. See issue #24 for context.

  4. Performance vs maintenance trade-off — While the benchmark improvements (20-30%) are real, the absolute times (~175-250ns per operation) are already very fast. For a caching library, the overhead of time.Now() vs runtime.nanotime() is typically negligible compared to the actual workload being cached. The performance gain doesn't justify the maintenance risk and API breakage.

We're going to rewrite our tests to use testing/synctest instead, which demonstrates that time.Now() is the right approach — it works with Go's testing infrastructure out of the box.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants