sky_line/Slipstream

Fork 1

Epic: networking -- own the asset fetch path, modern curl + OpenSSL for everything else #484

New issue

Closed

opened 2026-10-06 12:18:09 +00:00 by sky_line · 2 comments

sky_line commented

2026-10-06 12:18:09 +00:00

Owner

Plan:

#391: measure first what pipelining buys today (cold cache, pipelining on vs off within LL's connection caps). See the analysis there, including the CDN probe and the TPV-policy constraint.
#485: our own asset fetcher, a pipelined HTTP/1.1 client for CDN texture and mesh GETs (only if the measurement says pipelining matters).
#392 + #391: OpenSSL 3, then curl 8.x for everything else (login, region caps, inventory, web, chat image previews, voice, update check).
- OpenSSL 3 is low-risk for our own code: X509, EVP and ChaCha20 saved-login encryption are unchanged. Blowfish is only used by the dead llmail feature (needs the legacy provider, or drop it).
- The work is mostly rebuilding and repackaging the dependent libraries.
Afterwards: remove the dead pipelining paths from llcorehttp.

Rules:

Stay within LL's shipped connection caps (TPV policy 1.a / 2.f).
Treat all network input as hostile.
No dependency on LL changing the CDN.

SL's own viewer still ships curl 7.54.1 (2017) and OpenSSL 1.1.1 (end of life). Upgrading curl naively loses HTTP/1.1 pipelining (removed in curl 7.65), which our texture and mesh fetches use, and the asset CDN offers no HTTP/2 to replace it. LL changes slowly, so we don't wait on them. **Plan:** - [ ] #391: **measure first** what pipelining buys today (cold cache, pipelining on vs off within LL's connection caps). See the analysis there, including the CDN probe and the TPV-policy constraint. - [ ] #485: **our own asset fetcher**, a pipelined HTTP/1.1 client for CDN texture and mesh GETs (only if the measurement says pipelining matters). - [ ] #392 + #391: **OpenSSL 3, then curl 8.x** for everything else (login, region caps, inventory, web, chat image previews, voice, update check). - OpenSSL 3 is low-risk for our own code: X509, EVP and ChaCha20 saved-login encryption are unchanged. Blowfish is only used by the dead llmail feature (needs the legacy provider, or drop it). - The work is mostly rebuilding and repackaging the dependent libraries. - [ ] Afterwards: remove the dead pipelining paths from `llcorehttp`. **Rules:** - Stay within LL's shipped connection caps (TPV policy 1.a / 2.f). - Treat all network input as hostile. - No dependency on LL changing the CDN.

sky_line added the

epic

perf

platform

labels

2026-10-06 12:18:09 +00:00

sky_line referenced this issue

2026-10-06 12:18:09 +00:00

Our own asset fetcher: pipelined HTTP/1.1 client for texture/mesh GETs to the asset CDN #485

sky_line commented

2026-10-06 12:39:15 +00:00

Author

Owner

Measured: HttpPipelining on vs off, cold cache (2026-10-06)

Setup: the AssetFetch timeline log (e991adb381), real caches moved aside, default connection counts (texture 8, mesh2 8, well within LL's caps).

Light region (on first, then off):

	Mesh done	Textures drained	Queue < 10	Texture GETs
ON	+12.1 s	+40.6 s	+44.7 s	6,435
OFF	+8.0 s	+36.3 s	+40.4 s	6,420

Busy region (off first, then on; similar camera turn in both):

	Mesh main load	Textures drained	Queue empty	Texture GETs	Mesh
OFF	+26 s	+86.5 s	+90.6 s	19,313	20,174 GETs, 243 MB
ON	+32.5 s	+79.4 s	+83.5 s	19,489	20,070 GETs, 239 MB

Conclusion:

Pipelining's effect is about ±8% and roughly a wash. Whichever run went second was faster in both pairs, which suggests Akamai edge-cache warm-up and is about the same size as the effect itself.
No retries or errors in any run.
Throughput was about 200–245 texture GETs/s either way: limited by our own fetch and decode scheduling, not by pipelining.

Decision:

Upgrading to curl 8.x (no pipelining) is safe for asset loading at the current connection counts. No cap increase is needed, so no TPV-policy concern.
#485 (our own pipelined fetcher) is no longer needed for throughput; it's optional for scheduling only.
The next win is our fetch and decode pipeline.

## Measured: HttpPipelining on vs off, cold cache (2026-10-06) Setup: the AssetFetch timeline log (e991adb381), real caches moved aside, default connection counts (texture 8, mesh2 8, well within LL's caps). **Light region** (on first, then off): | | Mesh done | Textures drained | Queue < 10 | Texture GETs | |---|---|---|---|---| | ON | +12.1 s | +40.6 s | +44.7 s | 6,435 | | OFF | **+8.0 s** | **+36.3 s** | **+40.4 s** | 6,420 | **Busy region** (off first, then on; similar camera turn in both): | | Mesh main load | Textures drained | Queue empty | Texture GETs | Mesh | |---|---|---|---|---|---| | OFF | **+26 s** | +86.5 s | +90.6 s | 19,313 | 20,174 GETs, 243 MB | | ON | +32.5 s | **+79.4 s** | **+83.5 s** | 19,489 | 20,070 GETs, 239 MB | **Conclusion:** - Pipelining's effect is about ±8% and roughly a wash. Whichever run went second was faster in both pairs, which suggests Akamai edge-cache warm-up and is about the same size as the effect itself. - No retries or errors in any run. - Throughput was about 200–245 texture GETs/s either way: limited by our own fetch and decode scheduling, not by pipelining. **Decision:** - Upgrading to curl 8.x (no pipelining) is safe for asset loading at the current connection counts. No cap increase is needed, so no TPV-policy concern. - #485 (our own pipelined fetcher) is no longer needed for throughput; it's optional for scheduling only. - The next win is our fetch and decode pipeline.

sky_line referenced this issue from a commit

2026-10-06 13:14:08 +00:00

AssetFetch timeline log: texture/mesh fetch progress after login + settle time (#484)

~~sky_line referenced this issue 2026-10-06 14:15:23 +00:00~~

Explore a "Classic" UI mode (2007-style SL layout, like Firestorm): chosen at startup, live switch as a bonus #489

sky_line added this to the Slipstream project

2026-10-06 15:04:14 +00:00

sky_line commented

2026-10-09 08:53:45 +00:00

Author

Owner

Done. curl 8.x (#391) and OpenSSL 3 (#392) are in; asset fetch runs under our own scheduling on top of llcorehttp (#485, Beta 7) instead of a custom pipelined client, since pipelining measured as a wash and the CDN is HTTP/1.1-only. Pipelining is off (CURLPIPE_NOTHING) and the old pipeline-depth path is gone. Connection counts stay within LL's shipped caps.

sky_line closed this issue

2026-10-09 08:53:46 +00:00

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

sky_line/Slipstream#484

No description provided.

Rows
Columns