[Epic] Cache rewrite: segment store, compression, tiered texture caching, disk-aware sizing #715

Closed
opened 2026-10-12 11:17:23 +00:00 by sky_line · 4 comments
Owner

Cache rewrite. Replace the file-per-item caches with one budgeted, compressed, crash-safe store and a policy for what is worth caching.

Parts

  • #708 Segment store (append-only segments, index, crash safety, no global lock, multi-instance)
  • #709 SSD wear protection (compaction policy, daily write cap, drive-type behaviour, write stats)
  • #710 zstd for GPU-ready textures
  • #705 Texture tiers: one budget, J2C as the source, GPU-ready admitted on reuse, Normal / Aggressive caching strategy, J2C evicted first when GPU-ready holds full size
  • #711 Default sizes from free disk space, explicit sizes kept, disk-full guard (free - 2 GB)
  • #712 Hard-disk locality (region-ordered writes, hot-set rewrites)
  • #713 Object cache zstd
  • #714 Migration + inspect/dump tool

Related: #706 (texture RAM copies: boosted breakdown, face-less fix), #707 (RAM copies only while a feature holds a lease; depends on #705).

Principles

  • J2C (the server's format) stays the source of truth; GPU-ready is a curated hot tier.
  • Keep everything while space is free; evict by priority (duplicates first, then low reuse) when it isn't.
  • Never wipe the whole cache; a bad record is dropped on its own.
  • No more SSD writes than today; better locality for hard disks.
  • Never crash on a full disk; leave 2 GB free.

Order

Segment store + wear policy -> migration -> zstd -> texture tiers (#705) -> sizes -> HDD locality. Object cache zstd is independent.

**Cache rewrite.** Replace the file-per-item caches with one budgeted, compressed, crash-safe store and a policy for what is worth caching. ## Parts - [ ] #708 Segment store (append-only segments, index, crash safety, no global lock, multi-instance) - [ ] #709 SSD wear protection (compaction policy, daily write cap, drive-type behaviour, write stats) - [ ] #710 zstd for GPU-ready textures - [ ] #705 Texture tiers: one budget, J2C as the source, GPU-ready admitted on reuse, Normal / Aggressive caching strategy, J2C evicted first when GPU-ready holds full size - [ ] #711 Default sizes from free disk space, explicit sizes kept, disk-full guard (free - 2 GB) - [ ] #712 Hard-disk locality (region-ordered writes, hot-set rewrites) - [ ] #713 Object cache zstd - [ ] #714 Migration + inspect/dump tool **Related:** #706 (texture RAM copies: boosted breakdown, face-less fix), #707 (RAM copies only while a feature holds a lease; depends on #705). ## Principles - J2C (the server's format) stays the source of truth; GPU-ready is a curated hot tier. - Keep everything while space is free; evict by priority (duplicates first, then low reuse) when it isn't. - Never wipe the whole cache; a bad record is dropped on its own. - No more SSD writes than today; better locality for hard disks. - Never crash on a full disk; leave 2 GB free. ## Order Segment store + wear policy -> migration -> zstd -> texture tiers (#705) -> sizes -> HDD locality. Object cache zstd is independent.
sky_line added this to the Slipstream project 2026-10-12 11:17:23 +00:00
Author
Owner

Step 1 done: audit of today's disk caches + the interface the segment store must implement. (Code: main at 3df839bf30.)

Today's caches

Cache Owner Layout Key Budget (default) Threads
J2C textures LLTextureCache (lltexturecache.cpp) texture.entries + texture.cache (first 600 B of each texture) + [0-f]/<uuid>.texture bodies UUID ~10.6 GB (65% of CacheSize 16 GB), 1M entries own worker thread, 4 mutexes
GPU-ready LLGpuTexCache (llgputexcache.cpp) gputex/<uuid>_<w>x<h>_e_l_v[_mask].gtx UUID + mask 3 GB 2-thread pool, merged by poll()
Assets + meshes + upload scratch LLDiskCache / LLFileSystem (llfilesystem/) cache/sl_cache_<uuid>_0.asset, one flat folder UUID only (type ignored) ~5.7 GB (35%) no lock; main, mesh threads, General queue
Decoded sounds LLSoundCache sounds/<uuid>.dsf UUID 1 GB main
Cluster LOD (#744, new) LLCleanClusterCache clusterlod/<uuid>_f<face>_b<ver>.clod mesh UUID + face 2 GB 1-thread pool
Object cache LLVOCache objectcache/object.cache + objects_<x>_<y>.slc / _extras.slec region 128 regions 1 I/O thread

GPU-ready and cluster LOD already have the store's client shape (in-RAM index, async verified reads, results merged on one thread). The hard consumers are LLFileSystem (seek / append / read-write, ~70 call sites, the mesh repository patches a sparse file in place, downloads write a temp UUID then rename) and LLTextureCache (600-byte header split, a full-prefix rewrite per discard step, reads at an offset).

Interface (sketch; full version in the local research note)

namespace LLCache {
enum class Kind : U8 { J2C, GPUTEX, ASSET, MESH, CLUSTER, SCRATCH };
struct Key { LLUUID id; Kind kind; U32 variant; };   // variant: mask bit, mesh part, face
struct RecordInfo { Key key; U64 stored_bytes, raw_bytes; U32 kind_version; U8 codec;
                    U32 last_access, prev_session_access; U16 reuse_count; U8 meta[32]; };
struct Codec { enum Id : U8 { IDENTITY, ZSTD };       // bytes in -> bytes out, any backend
               static bool encode(Id, int level, const U8*, size_t, std::vector<U8>&);
               static bool decode(Id, const U8*, size_t, U8*, size_t raw); };
struct KindPolicy { Codec::Id codec; int level; U32 format_version; bool ranged_reads; BudgetGroupId group; };

class Store {
  static bool open(const Config&);                 // lock file; 2nd instance = read-only + private overflow
  static void setKindPolicy(Kind, const KindPolicy&);
  static bool lookup(const Key&, RecordInfo* = nullptr);   // index only, no I/O, any thread
  static void touch(const Key&);                   // RAM only, saved with the index snapshot
  static Ticket<ReadResult> read(const Key&, Priority, Completion = {});
  static Ticket<ReadResult> readRange(const Key&, U64 off, U64 len, Priority, Completion = {});
  static ReadResult readSync(const Key&);  static ReadResult readRangeSync(const Key&, U64, U64);
  static bool write(const Key&, WriteSource, const U8* meta, U8 meta_len, U32 flags, Priority,
                    Completion = {}, Ticket<Status>* done = nullptr);
      // flags: SUPERSEDE_PENDING, SKIP_IF_NOT_LARGER (J2C), PIN_SESSION (asset delivery)
      // WriteSource: owned bytes, or a borrowed span + release() (mapped Vulkan staging buffer)
  static std::unique_ptr<Staged> beginStaged(Kind);  // append()... commit(final_key) | abort()
  static bool remove(const Key&);  static void removeAll(Kind);
  static BudgetGroupId addBudgetGroup(const char*, U64);  static void setBudget(BudgetGroupId, U64);
  static void setEvictionPolicy(BudgetGroupId, std::shared_ptr<EvictionPolicy>);  // #705
  static void setWearPolicy(const WearPolicy&);    // #709 / #712
  static void setDiskReserve(U64);                 // #711
  static Stats stats(/*Kind | group*/);  static void forEach(Kind, ...);  // #714
};
}

Ticket can be polled (fetch workers that park) or completed on a WorkQueue the caller picks plus an optional wake callback. The store never chooses a thread.

Mapping (phase 1 = adapters, call sites unchanged; phase 2 = call sites move)

Today Kind / key Codec Phase-1 adapter
LLGpuTexCache GPUTEX (uuid, mask) zstd same static API over lookup/read/write; Info = record meta
LLTextureCache J2C (uuid) none one record = whole prefix (no 600 B split); readRangeSync(offset,size); writes supersede
LLFileSystem READ / exists / size ASSET (uuid) none pread at record offset + position (seek is free); Ogg decode works unchanged
LLFileSystem WRITE / APPEND / rename ASSET none per-UUID staging table that outlives the objects (the transfer / xfer code opens one APPEND object per packet); exists / size / READ see staged bytes; rename(new_id) commits one record, WRITE restarts, remove aborts, session end commits
Mesh READ_WRITE + setSize MESH (uuid, part) none not emulated: ~14 sites in llmeshrepository.cpp move to per-part records (header, LOD0-3, skin, physics) in phase 1
LLDiskCache::purge / purge thread -- -- removed (the store evicts on write)
LLCleanClusterCache CLUSTER (uuid, face) none optional, after assets; pages = range reads
LLSoundCache, LLVOCache stay files VOCache: zstd via Codec (#713) --

Findings worth knowing

  • A second viewer instance runs the 15-minute asset purge over the shared folder and writes assets unprotected (the purge thread starts outside the read-only check, llappviewer.cpp:9156). The other caches go read-only.
  • The asset cache key ignores the asset type. The land auction snapshot writes a TGA and a J2C under the same UUID, so the J2C overwrites the TGA (llfloaterauction.cpp:206/216). Phase 1 keeps UUID-only keys (type in meta) to preserve behaviour.
  • Assets reach their consumers through the cache: the fetch writes the file and the callback reads it back. So the store needs read-your-writes for queued records and must not evict a record written this session before it is read.
  • zstd is not a dependency yet (3p manifest has zlib-ng only).
  • J2C refinement already rewrites the whole prefix on every discard step today, so append-only costs no extra bytes there.

Open question for the owner

The budget model. #708 says one store with one budget, #711 lists Textures / Assets / Sounds separately, and #705 makes GPU-ready a capped share inside Textures. Proposal: one store with budget groups TEXTURES {J2C, GPU-ready share}, ASSETS {assets, mesh} and CLUSTER, and sounds keep their own budget outside the store.

Build the store core (segments, record format, sharded index + snapshot, one writer thread, tail recovery, lock file / overflow, codec shim, stats) as a standalone library with #708's tests (crash at each write step, corrupt / truncated records, disk full, two instances). Then move GPU-ready onto it behind its current API. It has whole-record semantics, our code on both ends, the only compressing kind (so it proves the codec), and a failure there just falls back to J2C. Then J2C (adds range reads and supersede), then assets + the mesh parts (the 265k-file folder; it waits only so the core is proven before it carries asset delivery). Add the zstd 3p package in parallel.

**Step 1 done: audit of today's disk caches + the interface the segment store must implement.** (Code: main at 3df839bf30.) ### Today's caches | Cache | Owner | Layout | Key | Budget (default) | Threads | |---|---|---|---|---|---| | J2C textures | `LLTextureCache` (lltexturecache.cpp) | `texture.entries` + `texture.cache` (first 600 B of each texture) + `[0-f]/<uuid>.texture` bodies | UUID | ~10.6 GB (65% of `CacheSize` 16 GB), 1M entries | own worker thread, 4 mutexes | | GPU-ready | `LLGpuTexCache` (llgputexcache.cpp) | `gputex/<uuid>_<w>x<h>_e_l_v[_mask].gtx` | UUID + mask | 3 GB | 2-thread pool, merged by `poll()` | | Assets + meshes + upload scratch | `LLDiskCache` / `LLFileSystem` (llfilesystem/) | `cache/sl_cache_<uuid>_0.asset`, one flat folder | UUID only (type ignored) | ~5.7 GB (35%) | no lock; main, mesh threads, General queue | | Decoded sounds | `LLSoundCache` | `sounds/<uuid>.dsf` | UUID | 1 GB | main | | Cluster LOD (#744, new) | `LLCleanClusterCache` | `clusterlod/<uuid>_f<face>_b<ver>.clod` | mesh UUID + face | 2 GB | 1-thread pool | | Object cache | `LLVOCache` | `objectcache/object.cache` + `objects_<x>_<y>.slc` / `_extras.slec` | region | 128 regions | 1 I/O thread | GPU-ready and cluster LOD already have the store's client shape (in-RAM index, async verified reads, results merged on one thread). The hard consumers are `LLFileSystem` (seek / append / read-write, ~70 call sites, the mesh repository patches a sparse file in place, downloads write a temp UUID then rename) and `LLTextureCache` (600-byte header split, a full-prefix rewrite per discard step, reads at an offset). ### Interface (sketch; full version in the local research note) ```cpp namespace LLCache { enum class Kind : U8 { J2C, GPUTEX, ASSET, MESH, CLUSTER, SCRATCH }; struct Key { LLUUID id; Kind kind; U32 variant; }; // variant: mask bit, mesh part, face struct RecordInfo { Key key; U64 stored_bytes, raw_bytes; U32 kind_version; U8 codec; U32 last_access, prev_session_access; U16 reuse_count; U8 meta[32]; }; struct Codec { enum Id : U8 { IDENTITY, ZSTD }; // bytes in -> bytes out, any backend static bool encode(Id, int level, const U8*, size_t, std::vector<U8>&); static bool decode(Id, const U8*, size_t, U8*, size_t raw); }; struct KindPolicy { Codec::Id codec; int level; U32 format_version; bool ranged_reads; BudgetGroupId group; }; class Store { static bool open(const Config&); // lock file; 2nd instance = read-only + private overflow static void setKindPolicy(Kind, const KindPolicy&); static bool lookup(const Key&, RecordInfo* = nullptr); // index only, no I/O, any thread static void touch(const Key&); // RAM only, saved with the index snapshot static Ticket<ReadResult> read(const Key&, Priority, Completion = {}); static Ticket<ReadResult> readRange(const Key&, U64 off, U64 len, Priority, Completion = {}); static ReadResult readSync(const Key&); static ReadResult readRangeSync(const Key&, U64, U64); static bool write(const Key&, WriteSource, const U8* meta, U8 meta_len, U32 flags, Priority, Completion = {}, Ticket<Status>* done = nullptr); // flags: SUPERSEDE_PENDING, SKIP_IF_NOT_LARGER (J2C), PIN_SESSION (asset delivery) // WriteSource: owned bytes, or a borrowed span + release() (mapped Vulkan staging buffer) static std::unique_ptr<Staged> beginStaged(Kind); // append()... commit(final_key) | abort() static bool remove(const Key&); static void removeAll(Kind); static BudgetGroupId addBudgetGroup(const char*, U64); static void setBudget(BudgetGroupId, U64); static void setEvictionPolicy(BudgetGroupId, std::shared_ptr<EvictionPolicy>); // #705 static void setWearPolicy(const WearPolicy&); // #709 / #712 static void setDiskReserve(U64); // #711 static Stats stats(/*Kind | group*/); static void forEach(Kind, ...); // #714 }; } ``` `Ticket` can be polled (fetch workers that park) or completed on a WorkQueue the caller picks plus an optional wake callback. The store never chooses a thread. ### Mapping (phase 1 = adapters, call sites unchanged; phase 2 = call sites move) | Today | Kind / key | Codec | Phase-1 adapter | |---|---|---|---| | `LLGpuTexCache` | GPUTEX (uuid, mask) | zstd | same static API over `lookup/read/write`; `Info` = record meta | | `LLTextureCache` | J2C (uuid) | none | one record = whole prefix (no 600 B split); `readRangeSync(offset,size)`; writes supersede | | `LLFileSystem` READ / exists / size | ASSET (uuid) | none | pread at record offset + position (seek is free); Ogg decode works unchanged | | `LLFileSystem` WRITE / APPEND / rename | ASSET | none | per-UUID staging table that outlives the objects (the transfer / xfer code opens one APPEND object per packet); exists / size / READ see staged bytes; `rename(new_id)` commits one record, WRITE restarts, `remove` aborts, session end commits | | Mesh READ_WRITE + setSize | MESH (uuid, part) | none | **not emulated**: ~14 sites in llmeshrepository.cpp move to per-part records (header, LOD0-3, skin, physics) in phase 1 | | `LLDiskCache::purge` / purge thread | -- | -- | removed (the store evicts on write) | | `LLCleanClusterCache` | CLUSTER (uuid, face) | none | optional, after assets; pages = range reads | | `LLSoundCache`, `LLVOCache` | stay files | VOCache: zstd via `Codec` (#713) | -- | ### Findings worth knowing - A second viewer instance runs the 15-minute asset purge over the shared folder and writes assets unprotected (the purge thread starts outside the read-only check, llappviewer.cpp:9156). The other caches go read-only. - The asset cache key ignores the asset type. The land auction snapshot writes a TGA and a J2C under the same UUID, so the J2C overwrites the TGA (llfloaterauction.cpp:206/216). Phase 1 keeps UUID-only keys (type in meta) to preserve behaviour. - Assets reach their consumers through the cache: the fetch writes the file and the callback reads it back. So the store needs read-your-writes for queued records and must not evict a record written this session before it is read. - zstd is not a dependency yet (3p manifest has zlib-ng only). - J2C refinement already rewrites the whole prefix on every discard step today, so append-only costs no extra bytes there. ### Open question for the owner The budget model. #708 says one store with one budget, #711 lists Textures / Assets / Sounds separately, and #705 makes GPU-ready a capped share inside Textures. Proposal: one store with budget groups TEXTURES {J2C, GPU-ready share}, ASSETS {assets, mesh} and CLUSTER, and sounds keep their own budget outside the store. ### Recommended first implementation step Build the store core (segments, record format, sharded index + snapshot, one writer thread, tail recovery, lock file / overflow, codec shim, stats) as a standalone library with #708's tests (crash at each write step, corrupt / truncated records, disk full, two instances). Then move **GPU-ready** onto it behind its current API. It has whole-record semantics, our code on both ends, the only compressing kind (so it proves the codec), and a failure there just falls back to J2C. Then J2C (adds range reads and supersede), then assets + the mesh parts (the 265k-file folder; it waits only so the core is proven before it carries asset delivery). Add the zstd 3p package in parallel.
Author
Owner

Decided 2026-10-18: region-aware placement inside the segment store

The store stays one append-only segment store with one index (#708). On top of it, content is placed by region, so a revisit reads a few contiguous runs instead of thousands of scattered records, and the store can prefetch a region on arrival. The details are split across the tickets:

  • #708: placement hints on writes, per-region runs in the single append stream, region manifests.
  • #712: prefetch on arrival (teleport, crossing, neighbour connect while heading there), and the HDD-only hot-set rewrite.
  • #705: region-recency eviction (records age out with the regions that use them; shared assets survive).

Interface additions (both optional; the drop-in adapters work without them, they just get no locality):

  • write(..., PlacementHint{region_id | SHARED})
  • prefetchRegion(region_id, priority), and cancelPrefetch(region_id)

Bail-out clause (owner): if testing shows placement is too complex to manage reliably, fall back towards plain size-budgeted caches. Placement must therefore stay removable: hints are advisory, a missing or wrong hint only costs locality and never correctness, and manifests are a separate record kind that can be dropped without touching the data records.

## Decided 2026-10-18: region-aware placement inside the segment store The store stays one append-only segment store with one index (#708). On top of it, content is **placed by region**, so a revisit reads a few contiguous runs instead of thousands of scattered records, and the store can **prefetch a region on arrival**. The details are split across the tickets: - **#708:** placement hints on writes, per-region **runs** in the single append stream, **region manifests**. - **#712:** **prefetch on arrival** (teleport, crossing, neighbour connect while heading there), and the HDD-only hot-set rewrite. - **#705:** **region-recency eviction** (records age out with the regions that use them; shared assets survive). **Interface additions** (both optional; the drop-in adapters work without them, they just get no locality): - `write(..., PlacementHint{region_id | SHARED})` - `prefetchRegion(region_id, priority)`, and `cancelPrefetch(region_id)` **Bail-out clause (owner):** if testing shows placement is too complex to manage reliably, fall back towards plain size-budgeted caches. Placement must therefore stay **removable**: hints are advisory, a missing or wrong hint only costs locality and never correctness, and manifests are a separate record kind that can be dropped without touching the data records.
Author
Owner

Decisions (owner, 2026-10-18)

  1. Budget groups: one store with budget groups.

    • Textures: J2C plus a GPU-ready share capped at about 30% (#705).
    • Assets: assets plus mesh.
    • Cluster LOD pages.

    Sounds keep their own small cache outside the store. Preferences can still show separate sizes per group.

  2. Cluster LOD page cache joins the store as its own budget group. It migrates after the GPU-ready textures.

  3. Second running viewer: read-only for now. The second instance opens the store read-only and writes new records to a small private overflow (required, because assets reach their consumers through the cache). Separately, the current asset cache's purge must not run in a second instance; it runs today regardless of the read-only check.

First implementation step: the store core as a standalone library (indra/llcachestore) with #708's tests, including the #709 wear policy, region-run placement and the zstd codec layer (LLCacheCodec, already merged). Viewer integration follows, GPU-ready textures first.

## Decisions (owner, 2026-10-18) 1. **Budget groups:** one store with budget groups. - **Textures:** J2C plus a GPU-ready share capped at about 30% (#705). - **Assets:** assets plus mesh. - **Cluster LOD pages.** Sounds keep their own small cache outside the store. Preferences can still show separate sizes per group. 2. **Cluster LOD page cache joins the store** as its own budget group. It migrates after the GPU-ready textures. 3. **Second running viewer: read-only for now.** The second instance opens the store read-only and writes new records to a small private overflow (required, because assets reach their consumers through the cache). Separately, the current asset cache's purge must not run in a second instance; it runs today regardless of the read-only check. **First implementation step:** the store core as a standalone library (`indra/llcachestore`) with #708's tests, including the #709 wear policy, region-run placement and the zstd codec layer (`LLCacheCodec`, already merged). Viewer integration follows, GPU-ready textures first.
Author
Owner

Shipped in 1.4.0 Beta 1. Closing; reopen if it misbehaves.

Shipped in 1.4.0 Beta 1. Closing; reopen if it misbehaves.
sky_line 2026-10-20 15:03:20 +00:00
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
sky_line/Slipstream#715
No description provided.