Cache store: slim the in-memory index (≈144 → ≈60-70 B per record) #866

New issue

Closed

opened 2026-10-19 09:55:27 +00:00 by sky_line · 1 comment

sky_line commented

2026-10-19 09:55:27 +00:00

Owner

Follow-up to #715. The store's index is std::unordered_map<Key, Entry> across 64 shards (llcachestore.cpp ~278-302). With 1.22M records (956k of them mesh parts), that's about 144 B/record ≈ 170 MB RAM (measured with the same layout). The old caches held about 6 MB in RAM.

The index also loads at startup, and that time grows with it: 7.8 ms at 33k records, 338 ms at 1.22M.

Where the bytes go

Key: 24 B.
Entry: 96 B.
- 10×U32 location / size / time fields;
- meta[32] inline on every entry, which only GPU-ready records use (27 B);
- PendingPtr (shared_ptr, 16 B), almost always null.
Node (next pointer + cached hash) and buckets: about 24 B.

Plan

Move meta out of Entry into a side table (per shard, keyed by slot), present only for kinds that write meta.
Move pending into a small per-shard map of in-flight writes.
Replace the node-based map with a flat open-addressed table, without per-node allocation or a cached hash. Pack the time fields if possible (e.g. 16-bit day/epoch-relative stamps where precision allows).
Report index RAM (bytes, records, bytes/record) in statsLine() / CacheMetrics, and measure snapshot load time against record count.

No on-disk format change is needed for 1-3 (snapshot format may change; a snapshot mismatch already falls back to a scan).

Optional, larger: one index record per mesh with a part table instead of one per part (about 70% fewer records). That needs a mesh layout change and lazy rewrite like the per-level texture ticket.

Acceptance

Index ≤ 70 B/record measured at 1.2M records.
Startup index load time is lower at the same record count.
All llcachestore tests pass. No behaviour change in eviction, compaction, read-only mode or overflow.

Follow-up to #715. The store's index is `std::unordered_map<Key, Entry>` across 64 shards (`llcachestore.cpp` ~278-302). With 1.22M records (956k of them mesh parts), that's about **144 B/record ≈ 170 MB RAM** (measured with the same layout). The old caches held about 6 MB in RAM. The index also loads at startup, and that time grows with it: 7.8 ms at 33k records, **338 ms at 1.22M**. ## Where the bytes go - Key: 24 B. - Entry: 96 B. - 10×U32 location / size / time fields; - `meta[32]` inline on every entry, which only GPU-ready records use (27 B); - `PendingPtr` (`shared_ptr`, 16 B), almost always null. - Node (next pointer + cached hash) and buckets: about 24 B. ## Plan 1. Move `meta` out of `Entry` into a side table (per shard, keyed by slot), present only for kinds that write meta. 2. Move `pending` into a small per-shard map of in-flight writes. 3. Replace the node-based map with a flat open-addressed table, without per-node allocation or a cached hash. Pack the time fields if possible (e.g. 16-bit day/epoch-relative stamps where precision allows). 4. Report index RAM (bytes, records, bytes/record) in `statsLine()` / CacheMetrics, and measure snapshot load time against record count. No on-disk format change is needed for 1-3 (snapshot format may change; a snapshot mismatch already falls back to a scan). Optional, larger: one index record per mesh with a part table instead of one per part (about 70% fewer records). That needs a mesh layout change and lazy rewrite like the per-level texture ticket. ## Acceptance - Index ≤ 70 B/record measured at 1.2M records. - Startup index load time is lower at the same record count. - All llcachestore tests pass. No behaviour change in eviction, compaction, read-only mode or overflow.

sky_line added the

perf

label

2026-10-19 09:55:27 +00:00

sky_line added this to the Slipstream project

2026-10-19 09:55:27 +00:00

sky_line referenced this issue from a commit

2026-10-19 11:20:27 +00:00

Cache store: flat in-memory index, ~153 -> ~67 B per record (#866)

sky_line referenced this issue from a commit

2026-10-19 11:20:27 +00:00

Cache metrics: store index RAM (index_bytes, index_records) in the store KV line and JSON (#866)