Implement capability HTTP and downloads (#54)
All checks were successful
Native code generation / deterministic (push) Successful in 11m59s
Imaging and meshing gate / native (push) Successful in 3m55s
JPEG 2000 feature / linux (push) Successful in 2m26s
Native Rust workspace compile / compile (push) Successful in 3m58s
Skia feature / linux (push) Successful in 31m44s
All checks were successful
Native code generation / deterministic (push) Successful in 11m59s
Imaging and meshing gate / native (push) Successful in 3m55s
JPEG 2000 feature / linux (push) Successful in 2m26s
Native Rust workspace compile / compile (push) Successful in 3m58s
Skia feature / linux (push) Successful in 31m44s
This commit is contained in:
85
docs/caps-http.md
Normal file
85
docs/caps-http.md
Normal file
@@ -0,0 +1,85 @@
|
||||
# Capability HTTP, rate limiting, and downloads
|
||||
|
||||
`HttpCapsClient`, `CapsRateLimiter`, and `Http::DownloadManager` are native Rust
|
||||
implementations of the corresponding LibreMetaverse classes. They do not call
|
||||
.NET code or use a helper process. A normal `GridClient` owns a `reqwest`
|
||||
client using rustls, its category limiter, and their shutdown order. Tests and
|
||||
embedders can inject either a `reqwest::Client` or an executor-neutral
|
||||
`HttpMessageHandler`, plus a `TimeProvider` for deterministic token-bucket time.
|
||||
|
||||
## HTTP policy
|
||||
|
||||
Only absolute `http` and `https` request URIs are accepted. Response headers
|
||||
are stored as text without interpreting `Location` or `Content-Location`, so a
|
||||
successful response containing an opaque `slcaps://` value remains usable.
|
||||
Production redirects are followed only while the next URI is HTTP(S) and the
|
||||
configured redirect count has not been exhausted.
|
||||
|
||||
The default policy bounds request bodies at 32 MiB, wire response bodies at
|
||||
64 MiB, decompressed bodies at 128 MiB, redirects at 10, and concurrent HTTP
|
||||
requests at `Settings::max_http_connections()` (32). Declared and observed
|
||||
response lengths are checked. Gzip and zlib/raw-deflate decoding reads through
|
||||
a limiting adapter, preventing a small compressed response from expanding
|
||||
beyond policy. Unsupported content encodings fail closed. Upload and download
|
||||
progress follows the mapped C# `ProgressReport`, including two-decimal percent
|
||||
values when a total length is known.
|
||||
|
||||
`CapsHttpLimits`, `HttpCapsClient::with_reqwest_client`, and
|
||||
`HttpCapsClient::with_handler_policy` are the explicit injection seams. Fake
|
||||
handlers receive the exact method, URI, content type, and request bytes and can
|
||||
return deterministic status, headers, and bytes without external network
|
||||
access. Grid-client shutdown cancels in-flight operations before disposing the
|
||||
rate limiter.
|
||||
|
||||
## Rate categories
|
||||
|
||||
Each category owns a bounded, oldest-first token bucket. A full waiting queue
|
||||
returns a non-acquired lease; the HTTP pipeline then proceeds, matching the C#
|
||||
handler's overload behavior rather than dropping a request.
|
||||
|
||||
| Category | Burst | Tokens/second | Queue |
|
||||
| --- | ---: | ---: | ---: |
|
||||
| Default | 20 | 10 | 30 |
|
||||
| RenderMaterials | 4 | 2 | 20 |
|
||||
| AssetFetch | 24 | 12 | 60 |
|
||||
| AssetUpload | 4 | 2 | 10 |
|
||||
| Inventory | 6 | 3 | 20 |
|
||||
| EventQueue | 3 | 2 | 3 |
|
||||
| DisplayName | 5 | 2 | 15 |
|
||||
| Voice | 10 | 5 | 20 |
|
||||
|
||||
`RegisterCapUri` applies the complete case-insensitive cap-name mapping from the
|
||||
reference. Unregistered URIs use `Default`; texture/mesh, upload, inventory,
|
||||
event queue, display-name/profile, material, and voice endpoints remain
|
||||
isolated from one another. Cancellation removes queued tickets, and disposal
|
||||
wakes all waiters immediately.
|
||||
|
||||
## Download behavior
|
||||
|
||||
The manager owns a 256-entry dispatch queue and permits 1 through 32 active
|
||||
downloads (8 by default). Absolute URIs are canonicalized before deduplication.
|
||||
All subscribers to the same active URI share one HTTP request, receive progress,
|
||||
and complete with cloned response bytes. Cancellation by any subscriber cancels
|
||||
that shared operation, as in the reference. Caller-provided
|
||||
`TaskCompletionSource` values are completed by the fire-and-forget API.
|
||||
|
||||
HTTP 401, 403, 404, and 410 responses are permanent. Other unsuccessful
|
||||
statuses and transport failures retry up to the request's configured retry
|
||||
count with bounded backoff. Disposal cancels active and queued work, wakes every
|
||||
waiter, joins the dispatcher, and prevents new requests.
|
||||
|
||||
Errors expose only typed categories and `Debug` output substitutes a redacted
|
||||
marker for capability URIs. No request URL, query token, body, or response body
|
||||
is logged.
|
||||
|
||||
Run the deterministic issue gate with:
|
||||
|
||||
```sh
|
||||
cargo test -p libremetaverse --test caps_http
|
||||
cargo test -p libremetaverse-compat-tests --test network_semantics queue_download
|
||||
cargo test -p libremetaverse-compat-tests --test network_semantics put_response_with_non_http_location
|
||||
```
|
||||
|
||||
The first suite uses injected handlers for all policy and concurrency cases and
|
||||
a loopback-only fake HTTP server for the actual reqwest redirect/stream path. It
|
||||
requires no live grid or Internet service.
|
||||
Reference in New Issue
Block a user