Clarify cache strategy

[stephen-dixon]

For general discussion and tracking

Cache Issues / Design Questions

• cache_enabled currently controls several different behaviours at once. Example: enabling it may enable mapping-result caching, data-source result caching, and the optional RamCache, even though these serve
  different purposes.

• Mapping-result cache keys are too coarse on develop. Current behaviour can cache by map_path only, so time_slice[0]/data and time_slice[1]/data may both resolve to time_slice[#]/data and risk reusing the
  wrong fully sliced result.

• Data-source cache admission on develop is based on static mapping-definition reference counts, not runtime request frequency. Example: one mapping time_slice[#]/data requested for 100 indices may never
  cache the shared raw UDA fetch because only one DataSourceMapping object exists.

• Data-source cache state is static/global on develop. This can leak results between mapper instances or tests if two data sources use the same name and rendered args.

• Data-source result caching and mapping-result caching need different policies. Example: raw UDA data should be reused across many slices, while fully sliced mapping results should only be reused for exact
  same request identity.

• cache_size currently only applies to RamCache, not mapping or data-source result caches. Mapping/data-source caches can grow without an entry or memory limit.

• RamCache has unclear ownership and purpose. It appears to be a centrally managed generic resource cache for data sources, but data sources can still implement their own caches and key namespacing is
  convention-only.

• Current cache sizing is entry-count oriented, but cached arrays vary widely in memory cost. Example: one equilibrium array may be much larger than hundreds of scalar mapping results.

• Eviction policy is incomplete or inconsistent. On develop, RamCache::drop_entries() is empty; on the cache branch it is LRU by entry count, while mapping/data-source caches still lack bounded eviction.

• Cache keys need to be explicit about request identity. Relevant dimensions include experiment, group, partition, resolved map path, indices, runtime attributes, requested rank/type, data-source instance,
  and rendered request args.

• Cache behaviour for slices should be documented and tested. Expected behaviour: fetch raw data once for identical data-source args, then apply different slices after the raw-cache lookup.

• There is no clear cache observability API. Developers cannot easily inspect hit/miss counts, entries, bytes used, or whether a cache layer is active.

Initial Design Proposals

• Split cache configuration by layer. Example:

  [caches.mapping]
  enabled = true

  [caches.data_source]
  enabled = true

  [caches.resource]
  enabled = false

• Make mapping-result cache keys include full request identity. At minimum: experiment, group, partition attributes, resolved map_path, indices, runtime attributes, requested data type, and rank.

• Make data-source result cache keys include data-source identity and rendered request args. Example key: {data_source_pointer, data_source_name, rendered_args}.

• Cache raw data-source results before slice/scale/offset, and apply slice/scale/offset after lookup. This supports UDA-style patterns where one large raw fetch backs many indexed slice requests.

• Replace static data-source cache state with handler-owned cache state. This prevents cross-mapper/test pollution and gives MappingHandler::reset() a clear cache-clearing responsibility.

• Remove data-source cache admission based on static reference counts. Prefer admission = "always" for data-source results, because runtime reuse matters more than how many mapping definitions reference the
  same args.

• Keep mapping-result cache admission conservative. Example:

  [caches.mapping]
  admission = "reference_count"
  reference_threshold = 2

  or disable it by default until exact key semantics and memory limits are robust.

• Add byte-aware cache limits using TypedDataArray::bytes(). Example:

  [[nodiscard]] size_t bytes() const {
  return size() * data_type_size(data_type());
  }

• Support both entry and memory limits. Example:

  [caches.data_source]
  max_entries = 100
  max_bytes = 2147483648
  eviction = "lru"

• Use LRU eviction consistently for bounded caches. This should apply to data-source result cache, mapping-result cache, and any retained resource cache.

• Rename or replace RamCache with an explicit ResourceCache if the team wants a shared data-source resource cache. Example:

  resource_cache.get_or_create(
  CacheNamespace{"UDA"},
  "client:plugin-gateway:56565",
  estimated_bytes,
  [] { return std::make_shared(...); }
  );

• Define ResourceCache ownership semantics clearly. libtokamap owns lifetime, limits, eviction, and stats; data sources own key construction, value type, and validity rules.

• Require namespaced resource-cache keys. Example: UDA/client/plugin-gateway:56565 rather than plain host:port, to avoid collisions between unrelated data sources.

• Add cache stats/introspection. Example:

  mapper.cache_info()

  returning entries, bytes, hits, misses, evictions, and enabled/disabled state per cache layer.

• Add tests for common cache patterns. Cover exact repeat requests, different indices, different runtime attrs, sibling mappings sharing raw args, cache disabled, cross-mapper isolation, and LRU eviction.

• Preserve backward compatibility for current flat config keys as aliases. Example: cache_enabled remains a top-level master switch, while new [caches.*] sections provide detailed policy.