diff --git a/docs/design/grpc-contract.md b/docs/design/grpc-contract.md index f54ca17b..4f28b38d 100644 --- a/docs/design/grpc-contract.md +++ b/docs/design/grpc-contract.md @@ -62,9 +62,9 @@ service InferenceCache { - **ReplicaStats** `{ string replica_id = 1; int64 cache_memory_bytes = 2; float hit_rate = 3; float pressure = 4; string client_version = 5; int64 t2_hit_tokens = 6; int64 t2_query_tokens = 7; }` `client_version` is an opaque version string identifying the client-side cache library the reporting replica is linked against (e.g. an LMCache client release). It carries no semver semantics on the wire — producers populate it, the server accepts it, no semver parsing or ordering is performed at this layer. Empty / unset is allowed and means "unknown"; older producers that don't fill the field MUST keep being accepted (additive, v1alpha1-compatible). The field reserves wire space for the producer half of an end-to-end client/server version-skew detection surface; the consumer half (server-side storage, per-CacheBackend exposure, the operator-visible status condition that closes the silent client/server mismatch class of bug) lands in a follow-up change and is intentionally out of scope for this contract update. `t2_hit_tokens` / `t2_query_tokens` are cumulative token counters for the **tier-2 (external offload, e.g. LMCache) cache**, sourced from the engine's `vllm:external_prefix_cache_{hits,queries}_total`. The server derives a presence-aware tier-2 hit-rate per CacheBackend (`status.indexParticipation.t2HitRate`): nil until the tier is exercised (`t2_query_tokens > 0`), then the query-weighted `hits/queries` ratio — so a value of `0` means tier-2 is configured but serving no reloads (a silently-degraded offload tier), distinct from "not yet used". Both default to 0 for producers that don't populate them (additive, v1alpha1-compatible). -- **CacheEvent** `{ Type type; string replica_id; string model_id; string tenant_id; bytes prefix_hash; int64 timestamp_us; string adapter_id = 7; }` +- **CacheEvent** `{ Type type; string replica_id; string model_id; string tenant_id; bytes prefix_hash; int64 timestamp_us; string adapter_id = 7; bool adapter_scoped = 8; }` Type ∈ `PREFIX_ADDED | PREFIX_EVICTED | REPLICA_UPDATED | ALL_CLEARED` - `adapter_id` narrows a `PREFIX_EVICTED` to one adapter partition; empty removes the prefix across **every** partition (the conservative legacy behavior — removal is soft state). Ignored by `ALL_CLEARED` (a flush clears the replica across adapters) and `REPLICA_UPDATED` (liveness is adapter-independent). + `adapter_id` narrows a `PREFIX_EVICTED` to one adapter partition **when `adapter_scoped` is set** (so `""` targets the base-model partition, not a sweep) **or when `adapter_id` is non-empty** (a pre-`adapter_scoped` producer naming a LoRA adapter — honored so a mixed-version fleet doesn't regress its narrowing). Only an empty `adapter_id` with `adapter_scoped` unset removes the prefix across **every** partition (the conservative legacy sweep — removal is soft state). Ignored by `ALL_CLEARED` (a flush clears the replica across adapters) and `REPLICA_UPDATED` (liveness is adapter-independent). - **Metric** `{ string name; string type; map labels; double value; int64 timestamp_us; }` (Prometheus `inferencecache_*`, tech spec §4.3) - **StreamEventsRequest** `{ string model_id; string tenant_id; repeated CacheEvent.Type types; }` - **StreamMetricsRequest** `{ repeated string names; }` @@ -268,7 +268,7 @@ The fix partitions the **index**, not the hash: - **The fingerprint construction is unchanged.** Adapter identity is never mixed into the hash, so `pkg/fingerprint`, its golden vectors, and every cross-language implementation (Go / Python / Rust) stay byte-for-byte as they were. The hash still answers "what content is this?"; the partition answers "whose KV is it?". - **Matching inside a partition is unchanged.** Exact-match and the block-chain longest-leading-run rule behave identically; the partition only bounds which entries are candidates. - `LookupRouteResponse.adapter_id` (5) echoes the partition the index was consulted in, so a caller can confirm the round trip. It is empty on the fail-open envelopes returned *before* the index is consulted (`TIMEOUT`, tokenizer fail-open, `POLICY_REQUIRES_CHAIN`). -- `CacheEvent.adapter_id` (7) narrows a `PREFIX_EVICTED` to one partition. Without it, one adapter's GPU eviction would drop the identical hash under *every* adapter — discarding hints that are still valid. Empty keeps the original cross-partition sweep. +- `CacheEvent.adapter_id` (7) narrows a `PREFIX_EVICTED` to one partition, with the new `adapter_scoped` (8) flag distinguishing a base-model eviction from a legacy sweep. The removal scopes to `adapter_id` when `adapter_scoped` is set (so `adapter_id=""` drops only the base partition, no longer discarding another adapter's still-valid hint for the identical hash) **or** when `adapter_id` is non-empty (a pre-`adapter_scoped` producer that named a LoRA adapter — honored so a mixed-version fleet doesn't regress a `sql-lora` eviction into a cross-partition sweep). Only an empty `adapter_id` with `adapter_scoped` unset keeps the conservative cross-partition sweep. A separate bool rather than making `adapter_id` `optional`, which the wire-compat gate (`buf breaking`) rejects as a cardinality change — the bool is purely additive. **Producer / consumer must agree, exactly as they already do for `hash_scheme`.** A lookup that sets `adapter_id` will not match entries ingested without one, and vice versa. Empty/unset is the **default partition**, and this is exactly the pre-adapter behavior **for base-model / non-LoRA traffic**: an engine serving no LoRA emits a nil `lora_id`, so ingest and lookup both land under `""` and are completely unaffected — no regression, no configuration required. **This is not a "single LoRA adapter just works" claim.** Only a *nil* `lora_id` uses `""`; *any* non-nil `lora_id` — including a deployment with exactly one adapter — must resolve to a **configured** `--lora-adapter-names` identity, otherwise it is dropped at ingest (**fail-closed**) rather than landing under `""` or a replica-local alias. So even one LoRA adapter requires the ingest identifier (the subscriber's resolved `adapter_id`) to match the gateway's query `adapter_id` end-to-end, or every lookup silently misses — the same agreement `hash_scheme` already demands. diff --git a/docs/reference-stack/scripts/default_install_smoke.sh b/docs/reference-stack/scripts/default_install_smoke.sh index 38e35f1c..71e7a633 100755 --- a/docs/reference-stack/scripts/default_install_smoke.sh +++ b/docs/reference-stack/scripts/default_install_smoke.sh @@ -1196,6 +1196,25 @@ grpcurl_report_cache_state() { 2>>"$err_file" } +grpcurl_publish_event() { + local payload="$1" + local err_file="${2:-$LOG_DIR/grpcurl-publish-event.err}" + # Default install is plaintext (see grpcurl_lookup_route). + if printf '%s\n' "$payload" | grpcurl -plaintext -max-time 5 -d @ \ + "localhost:$GRPC_LOCAL_PORT" \ + inferencecache.v1alpha1.InferenceCache/PublishEvent \ + 2>"$err_file"; then + return 0 + fi + log "reflection PublishEvent probe failed; falling back to proto-file probe" >&2 + printf '%s\n' "$payload" | grpcurl -plaintext -max-time 5 \ + -import-path proto -proto inferencecache/v1alpha1/inferencecache.proto \ + -d @ \ + "localhost:$GRPC_LOCAL_PORT" \ + inferencecache.v1alpha1.InferenceCache/PublishEvent \ + 2>>"$err_file" +} + has_reason_code() { local resp="$1" local want="$2" @@ -1472,6 +1491,86 @@ assert_adapter_no_alias "$adapter_none_payload" "$LOG_DIR/grpcurl-adapter-none.e "LookupRoute without adapter_id matched an adapter-scoped prefix — adapter-scoped ingest must stay out of the default partition" log "adapter partition enforced end-to-end: same adapter_id → PREFIX_MATCH; a different adapter_id and an absent adapter_id each returned the fail-open AFFINITY_HINT (RPC succeeded, off the prefix-match path) on the identical prefix hash" +# --- adapter-scoped eviction probe (adapter_scoped) ------------------------ +# +# A base-model PREFIX_EVICTED (adapter_id="" WITH adapter_scoped) must drop ONLY +# the base partition and leave a co-resident LoRA hint for the SAME token hash +# intact — the over-sweep the adapter_scoped flag fixes. Driven end-to-end with no +# engine traffic: seed the identical prefix hash under BOTH the base ("") and an +# "evict-lora" partition on one replica, PublishEvent a base-scoped eviction, then +# assert via LookupRoute that the base partition stopped matching while the LoRA +# partition still returns PREFIX_MATCH. +log "seeding base + LoRA partitions and asserting a base-scoped PREFIX_EVICTED spares the LoRA hint" +evict_replica="evict-smoke-replica" +evict_hash_b64="ZXZpY3QtcHJlZml4" # base64("evict-prefix"); tokenCount=64 clears both policy gates + +evict_base_seed="{\"replicaId\":\"$evict_replica\",\"modelId\":\"$adapter_model\",\"tenantId\":\"$POLICY_SMOKE_NS\",\"hashScheme\":\"vllm\",\"prefixes\":[{\"prefixHash\":\"$evict_hash_b64\",\"tokenCount\":64}]}" +evict_lora_seed="{\"replicaId\":\"$evict_replica\",\"modelId\":\"$adapter_model\",\"tenantId\":\"$POLICY_SMOKE_NS\",\"hashScheme\":\"vllm\",\"prefixes\":[{\"prefixHash\":\"$evict_hash_b64\",\"tokenCount\":64,\"adapterId\":\"evict-lora\"}]}" +for seed in "$evict_base_seed" "$evict_lora_seed"; do + grpcurl_report_cache_state "$seed" "$LOG_DIR/grpcurl-evict-seed.err" >/dev/null || { + cat "$LOG_DIR/grpcurl-evict-seed.err" >&2 || true + fail "grpcurl ReportCacheState did not accept an eviction-probe seed" + } +done + +evict_base_lookup="{\"modelId\":\"$adapter_model\",\"tenantId\":\"$POLICY_SMOKE_NS\",\"hashScheme\":\"vllm\",\"prefixHash\":\"$evict_hash_b64\",\"prefixTokenCount\":64}" +evict_lora_lookup="{\"modelId\":\"$adapter_model\",\"tenantId\":\"$POLICY_SMOKE_NS\",\"hashScheme\":\"vllm\",\"prefixHash\":\"$evict_hash_b64\",\"prefixTokenCount\":64,\"adapterId\":\"evict-lora\"}" + +# Precondition: both partitions match before the eviction. Wait for the seeds to +# apply (evict-lora), then confirm base is present too — else "base gone" below +# would pass vacuously. +deadline=$(($(date +%s) + POLICY_PUSH_TIMEOUT)) +evict_lora_pre="" +until has_reason_code "$evict_lora_pre" "PREFIX_MATCH"; do + evict_lora_pre="$(grpcurl_lookup_route "$evict_lora_lookup" "$LOG_DIR/grpcurl-evict-pre-lora.err")" || true + has_reason_code "$evict_lora_pre" "PREFIX_MATCH" && break + if [ "$(date +%s)" -ge "$deadline" ]; then + echo "evict-lora precondition LookupRoute (want PREFIX_MATCH):" >&2; echo "$evict_lora_pre" >&2 + fail "eviction-probe precondition: the evict-lora partition never matched within ${POLICY_PUSH_TIMEOUT}s" + fi + sleep 2 +done +evict_base_pre="$(grpcurl_lookup_route "$evict_base_lookup" "$LOG_DIR/grpcurl-evict-pre-base.err")" || true +has_reason_code "$evict_base_pre" "PREFIX_MATCH" || { + echo "base precondition LookupRoute (want PREFIX_MATCH):" >&2; echo "$evict_base_pre" >&2 + fail "eviction-probe precondition: the base partition did not match before the eviction" +} + +# Base-scoped eviction: adapter_scoped=true with an empty adapter_id targets the +# base partition ONLY (without the flag an empty adapter_id sweeps every partition). +evict_event="{\"type\":\"PREFIX_EVICTED\",\"replicaId\":\"$evict_replica\",\"modelId\":\"$adapter_model\",\"tenantId\":\"$POLICY_SMOKE_NS\",\"prefixHash\":\"$evict_hash_b64\",\"adapterScoped\":true}" +evict_ack="$(grpcurl_publish_event "$evict_event" "$LOG_DIR/grpcurl-evict-publish.err")" || { + cat "$LOG_DIR/grpcurl-evict-publish.err" >&2 || true + fail "grpcurl PublishEvent did not accept the base-scoped PREFIX_EVICTED" +} +log "PublishEvent (base-scoped eviction) response: $evict_ack" + +# The base partition must stop matching (proves the eviction applied); once it +# has, the LoRA partition must STILL match (proves it was not over-swept). +deadline=$(($(date +%s) + POLICY_PUSH_TIMEOUT)) +evict_base_post="" +while :; do + evict_base_post="$(grpcurl_lookup_route "$evict_base_lookup" "$LOG_DIR/grpcurl-evict-post-base.err")" || true + if [ -n "$evict_base_post" ] && ! has_reason_code "$evict_base_post" "PREFIX_MATCH"; then + break + fi + if [ "$(date +%s)" -ge "$deadline" ]; then + echo "base LookupRoute after eviction (want NOT PREFIX_MATCH):" >&2; echo "$evict_base_post" >&2 + fail "the base partition was still PREFIX_MATCH ${POLICY_PUSH_TIMEOUT}s after a base-scoped PREFIX_EVICTED — the eviction did not apply" + fi + sleep 2 +done + +evict_lora_post="$(grpcurl_lookup_route "$evict_lora_lookup" "$LOG_DIR/grpcurl-evict-post-lora.err")" || { + cat "$LOG_DIR/grpcurl-evict-post-lora.err" >&2 || true + fail "grpcurl LookupRoute (evict-lora, post-eviction) failed — a transport error must not mask the over-sweep check" +} +has_reason_code "$evict_lora_post" "PREFIX_MATCH" || { + echo "evict-lora LookupRoute after the base eviction (want PREFIX_MATCH):" >&2; echo "$evict_lora_post" >&2 + fail "a base-scoped PREFIX_EVICTED also swept the evict-lora partition — the adapter_scoped over-sweep fix regressed" +} +log "adapter_scoped eviction enforced end-to-end: a base-scoped PREFIX_EVICTED dropped the base partition while the co-resident LoRA hint for the identical hash survived" + # --- routingFloorScore end-to-end probe ------------------------------------ # Proves the new field flows CR → controller flatten → /policy push → server # resolver → buildLookupResponse downgrade. The same 64-token prefix that diff --git a/pkg/adapters/engine/mapper.go b/pkg/adapters/engine/mapper.go index c4c6e565..77ea8c86 100644 --- a/pkg/adapters/engine/mapper.go +++ b/pkg/adapters/engine/mapper.go @@ -89,16 +89,20 @@ func (c Config) ClearedEvent(tsSeconds float64) *icpb.CacheEvent { // EvictedEvent builds one PREFIX_EVICTED CacheEvent for an already-derived prefix // hash (our content fingerprint, the index key to drop) in a specific adapter // partition. The subscriber maps an evicted engine block hash to both via -// positionalIndex.Removed. An empty adapterID is the default partition; the -// server then falls back to its cross-partition (legacy) removal. +// positionalIndex.Removed. adapter_scoped is always set, so the server drops the +// prefix from exactly the adapterID partition (even "" for the base model) — a +// base-model eviction never sweeps live LoRA hints for the same token hash. +// (Only a producer that leaves adapter_scoped false AND adapter_id empty gets the +// conservative cross-partition legacy sweep; a non-empty adapter_id still narrows.) func (c Config) EvictedEvent(prefixHash []byte, adapterID string, tsSeconds float64) *icpb.CacheEvent { return &icpb.CacheEvent{ - Type: icpb.CacheEvent_PREFIX_EVICTED, - ReplicaId: c.ReplicaID, - ModelId: c.ModelID, - TenantId: c.TenantID, - PrefixHash: prefixHash, - AdapterId: adapterID, - TimestampUs: microsFromSeconds(tsSeconds), + Type: icpb.CacheEvent_PREFIX_EVICTED, + ReplicaId: c.ReplicaID, + ModelId: c.ModelID, + TenantId: c.TenantID, + PrefixHash: prefixHash, + AdapterId: adapterID, + AdapterScoped: true, + TimestampUs: microsFromSeconds(tsSeconds), } } diff --git a/pkg/adapters/engine/mapper_test.go b/pkg/adapters/engine/mapper_test.go index f87faba8..c3c46581 100644 --- a/pkg/adapters/engine/mapper_test.go +++ b/pkg/adapters/engine/mapper_test.go @@ -22,8 +22,14 @@ func TestEvictedEvent(t *testing.T) { if !bytes.Equal(e.PrefixHash, []byte{0x01, 0x02}) { t.Errorf("prefix hash = %x, want 0102", e.PrefixHash) } - if e.AdapterId != "" { - t.Errorf("adapter id = %q, want \"\" (no adapter)", e.AdapterId) + // A base-model eviction carries adapter_id="" but marks it authoritative + // (adapter_scoped), so the server drops only the base ("") partition instead + // of sweeping every adapter. + if got := e.GetAdapterId(); got != "" { + t.Errorf("adapter id = %q, want \"\" (base partition)", got) + } + if !e.GetAdapterScoped() { + t.Error("adapter_scoped = false; a base eviction must mark adapter_id authoritative to scope removal to the base partition") } } @@ -31,8 +37,8 @@ func TestEvictedEvent(t *testing.T) { // prefix only there — the same hash can be live under another adapter. func TestEvictedEventCarriesAdapter(t *testing.T) { e := testConfig().EvictedEvent([]byte{0x01}, "sql-lora", 1.0) - if e.AdapterId != "sql-lora" { - t.Errorf("adapter id = %q, want sql-lora", e.AdapterId) + if e.GetAdapterId() != "sql-lora" || !e.GetAdapterScoped() { + t.Errorf("adapter id = %q scoped = %v, want sql-lora / true", e.GetAdapterId(), e.GetAdapterScoped()) } } diff --git a/pkg/index/adapter_partition_test.go b/pkg/index/adapter_partition_test.go index bfdb7b51..6d463a3f 100644 --- a/pkg/index/adapter_partition_test.go +++ b/pkg/index/adapter_partition_test.go @@ -188,7 +188,7 @@ func TestAdapterPartitionEvictionIsScopedToItsAdapter(t *testing.T) { idx.ApplyEvent(Event{ Type: EventPrefixEvicted, ReplicaID: "replica-0", Model: adapterModel, Tenant: adapterTenant, - PrefixHash: hash("same-tokens"), Adapter: "sql-lora", + PrefixHash: hash("same-tokens"), Adapter: "sql-lora", AdapterScoped: true, }) if got := lookupUnder(idx, "sql-lora", "same-tokens"); len(got) != 0 { @@ -199,6 +199,51 @@ func TestAdapterPartitionEvictionIsScopedToItsAdapter(t *testing.T) { } } +// A base-model eviction (Adapter "" WITH AdapterScoped) drops ONLY the base +// partition and must not sweep a live LoRA hint for the same token hash — the +// over-sweep that the adapter_scoped flag exists to fix. +func TestAdapterPartitionBaseEvictionScopedSparesLoRA(t *testing.T) { + idx := New() + ingestUnder(idx, "replica-0", "", "same-tokens") + ingestUnder(idx, "replica-0", "sql-lora", "same-tokens") + + idx.ApplyEvent(Event{ + Type: EventPrefixEvicted, ReplicaID: "replica-0", + Model: adapterModel, Tenant: adapterTenant, + PrefixHash: hash("same-tokens"), Adapter: "", AdapterScoped: true, + }) + + if got := lookupUnder(idx, "", "same-tokens"); len(got) != 0 { + t.Errorf("base entry survived its own eviction: %+v", got) + } + if got := lookupUnder(idx, "sql-lora", "same-tokens"); len(got) != 1 { + t.Errorf("sql-lora entry = %+v, want it untouched by the base-model eviction", got) + } +} + +// Upgrade compatibility: a pre-adapter_scoped producer (new server, not-yet- +// upgraded subscriber) sends a non-empty adapter_id but leaves adapter_scoped +// false. Its LoRA eviction must still narrow to that adapter, not regress into a +// cross-partition sweep that wipes another adapter's live hint. +func TestAdapterPartitionEvictionNonEmptyAdapterScopesWithoutFlag(t *testing.T) { + idx := New() + ingestUnder(idx, "replica-0", "sql-lora", "same-tokens") + ingestUnder(idx, "replica-0", "chat-lora", "same-tokens") + + idx.ApplyEvent(Event{ + Type: EventPrefixEvicted, ReplicaID: "replica-0", + Model: adapterModel, Tenant: adapterTenant, + PrefixHash: hash("same-tokens"), Adapter: "sql-lora", // AdapterScoped left false (pre-upgrade producer) + }) + + if got := lookupUnder(idx, "sql-lora", "same-tokens"); len(got) != 0 { + t.Errorf("sql-lora entry survived its own eviction: %+v", got) + } + if got := lookupUnder(idx, "chat-lora", "same-tokens"); len(got) != 1 { + t.Errorf("chat-lora entry = %+v, want it untouched — a non-empty adapter_id must narrow even without adapter_scoped", got) + } +} + // An eviction with NO adapter keeps the original conservative behavior: it // sweeps every partition. That is what a pre-adapter producer emits, and for // such a producer all entries live in the "" partition anyway — so the diff --git a/pkg/index/index.go b/pkg/index/index.go index e227cc7c..fd5d20ce 100644 --- a/pkg/index/index.go +++ b/pkg/index/index.go @@ -243,14 +243,18 @@ type Event struct { Model string Tenant string PrefixHash []byte - // Adapter narrows a PREFIX_EVICTED removal to one adapter partition. Empty - // removes the prefix from EVERY adapter partition — the conservative legacy - // behavior, and identical to the pre-adapter code for producers that never - // set it (all of whose entries live in the "" partition anyway). Ignored by - // ALL_CLEARED (a flush clears the replica across adapters) and by + // Adapter narrows a PREFIX_EVICTED removal to one adapter partition. The removal + // scopes to Adapter when AdapterScoped is set (the authoritative signal — so + // Adapter="" targets the base partition, not a sweep) OR when Adapter is + // non-empty (a pre-adapter_scoped producer naming a LoRA adapter; honored so a + // mixed-version fleet doesn't regress its narrowing into a sweep). It sweeps + // EVERY adapter partition only when Adapter is empty AND AdapterScoped is unset + // — the conservative legacy behavior for a no-adapter producer. Both are ignored + // by ALL_CLEARED (a flush clears the replica across adapters) and by // REPLICA_UPDATED (liveness is adapter-independent). - Adapter string - Timestamp time.Time + Adapter string + AdapterScoped bool + Timestamp time.Time } // LookupRequest asks which replicas hold a given prefix, within a hash scheme. @@ -898,22 +902,23 @@ func (i *Index) ApplyEvent(ev Event) { // adapters at once on the same replica; dropping every partition for one // adapter's GPU eviction would throw away hints that are still valid. // - // An empty ev.Adapter falls back to the original cross-partition sweep. - // That is exact for a pre-adapter producer (all its entries are in the "" - // partition anyway), but an adapter-aware producer ALSO emits "" for a - // genuine base-model eviction — and the sweep then drops live LoRA- - // partition hints for the same prefix hash too. That is conservative soft - // state (at worst a cache miss, re-added by the authoritative - // ReportCacheState path) and a strict non-regression (pre-partition, base - // and LoRA shared one partition, so a base eviction was already total). - // Making "" mean the base partition only, without breaking the legacy - // wildcard, needs explicit adapter_id presence on the event — a tracked - // follow-up. + // The removal narrows to ev.Adapter unless it genuinely cannot tell which + // partition to target. It narrows when EITHER: + // - the producer set adapter_scoped — the new authoritative signal, so + // adapter_id="" means the base partition, not a sweep; OR + // - adapter_id is non-empty — a pre-adapter_scoped producer that named a + // LoRA adapter. Honoring that keeps its narrowing on a mixed-version + // fleet (new server + not-yet-upgraded subscriber) rather than + // regressing a "sql-lora" eviction into a cross-partition sweep. + // It sweeps EVERY partition only when adapter_id is empty AND adapter_scoped + // is unset — a legacy no-adapter producer, whose entries all live in the "" + // partition anyway, so the sweep is exact for it. + scoped := ev.AdapterScoped || ev.Adapter != "" for key, replicas := range i.prefixes { if key.tenant != ev.Tenant || key.model != ev.Model || key.prefixHash != hash { continue } - if ev.Adapter != "" && key.adapter != ev.Adapter { + if scoped && key.adapter != ev.Adapter { continue } i.removeReplicaLocked(key, replicas, ev.ReplicaID) diff --git a/pkg/server/inferencecache_service.go b/pkg/server/inferencecache_service.go index a283b49c..8269b005 100644 --- a/pkg/server/inferencecache_service.go +++ b/pkg/server/inferencecache_service.go @@ -1034,10 +1034,14 @@ func (s *inferenceCacheService) PublishEvent(_ context.Context, ev *icpb.CacheEv Model: ev.GetModelId(), Tenant: ev.GetTenantId(), PrefixHash: ev.GetPrefixHash(), - // Narrows a PREFIX_EVICTED to one adapter partition; empty keeps the - // conservative cross-partition removal (legacy producers). - Adapter: ev.GetAdapterId(), - Timestamp: microsToTime(ev.GetTimestampUs()), + // Narrows a PREFIX_EVICTED to one adapter partition when the producer + // marked adapter_id authoritative (adapter_scoped) — including "" for a + // base-model eviction. Absent adapter_scoped still narrows on a non-empty + // adapter_id (pre-flag producers); only an empty adapter_id with + // adapter_scoped unset keeps the conservative cross-partition removal. + Adapter: ev.GetAdapterId(), + AdapterScoped: ev.GetAdapterScoped(), + Timestamp: microsToTime(ev.GetTimestampUs()), }) } return &icpb.Ack{Accepted: true}, nil diff --git a/pkg/server/proto/inferencecache/v1alpha1/inferencecache.pb.go b/pkg/server/proto/inferencecache/v1alpha1/inferencecache.pb.go index c4f330e7..9acd6fa7 100644 --- a/pkg/server/proto/inferencecache/v1alpha1/inferencecache.pb.go +++ b/pkg/server/proto/inferencecache/v1alpha1/inferencecache.pb.go @@ -1307,15 +1307,25 @@ type CacheEvent struct { TenantId string `protobuf:"bytes,4,opt,name=tenant_id,json=tenantId,proto3" json:"tenant_id,omitempty"` PrefixHash []byte `protobuf:"bytes,5,opt,name=prefix_hash,json=prefixHash,proto3" json:"prefix_hash,omitempty"` TimestampUs int64 `protobuf:"varint,6,opt,name=timestamp_us,json=timestampUs,proto3" json:"timestamp_us,omitempty"` - // Adapter partition a PREFIX_EVICTED removal targets. Set, the server drops - // the prefix only from that partition; empty, it drops the prefix across - // EVERY adapter partition — the conservative legacy behavior (removal is - // soft state: an over-eager drop costs a cache miss, never a wrong answer), - // and identical to today for producers that never populate the field. - // Ignored by ALL_CLEARED (a cache flush clears the replica across every - // adapter) and by REPLICA_UPDATED (liveness is adapter-independent). - // Additive, v1alpha1-compatible. - AdapterId string `protobuf:"bytes,7,opt,name=adapter_id,json=adapterId,proto3" json:"adapter_id,omitempty"` + // Adapter partition a PREFIX_EVICTED removal targets. Honored when + // adapter_scoped is set, OR when non-empty (a pre-adapter_scoped producer that + // named a LoRA adapter). "" is the base-model / non-LoRA partition. Ignored by + // ALL_CLEARED (a cache flush clears the replica across every adapter) and by + // REPLICA_UPDATED (liveness is adapter-independent). Additive, + // v1alpha1-compatible. + AdapterId string `protobuf:"bytes,7,opt,name=adapter_id,json=adapterId,proto3" json:"adapter_id,omitempty"` + // adapter_scoped marks adapter_id as authoritative for a PREFIX_EVICTED so that + // adapter_id="" targets the base-model partition instead of sweeping every + // partition — the signal that lets a base-model eviction avoid discarding live + // LoRA hints for the same token-derived hash. An adapter-aware producer always + // sets it. The server ALSO narrows on a non-empty adapter_id when this is false, + // so a pre-adapter_scoped producer's LoRA eviction keeps narrowing on a + // mixed-version fleet; only an empty adapter_id with adapter_scoped=false takes + // the conservative cross-partition sweep. Removal is soft state: an over-eager + // drop costs a cache miss, never a wrong answer. A new bool rather than making + // adapter_id `optional`, which the wire-compat gate rejects as a cardinality + // change. Additive; ignored outside PREFIX_EVICTED. + AdapterScoped bool `protobuf:"varint,8,opt,name=adapter_scoped,json=adapterScoped,proto3" json:"adapter_scoped,omitempty"` unknownFields protoimpl.UnknownFields sizeCache protoimpl.SizeCache } @@ -1399,6 +1409,13 @@ func (x *CacheEvent) GetAdapterId() string { return "" } +func (x *CacheEvent) GetAdapterScoped() bool { + if x != nil { + return x.AdapterScoped + } + return false +} + type StreamEventsRequest struct { state protoimpl.MessageState `protogen:"open.v1"` ModelId string `protobuf:"bytes,1,opt,name=model_id,json=modelId,proto3" json:"model_id,omitempty"` @@ -1824,7 +1841,7 @@ var file_inferencecache_v1alpha1_inferencecache_proto_rawDesc = string([]byte{ 0x20, 0x01, 0x28, 0x0b, 0x32, 0x25, 0x2e, 0x69, 0x6e, 0x66, 0x65, 0x72, 0x65, 0x6e, 0x63, 0x65, 0x63, 0x61, 0x63, 0x68, 0x65, 0x2e, 0x76, 0x31, 0x61, 0x6c, 0x70, 0x68, 0x61, 0x31, 0x2e, 0x43, 0x61, 0x63, 0x68, 0x65, 0x53, 0x75, 0x6d, 0x6d, 0x61, 0x72, 0x79, 0x52, 0x07, 0x73, 0x75, 0x6d, - 0x6d, 0x61, 0x72, 0x79, 0x22, 0xee, 0x02, 0x0a, 0x0a, 0x43, 0x61, 0x63, 0x68, 0x65, 0x45, 0x76, + 0x6d, 0x61, 0x72, 0x79, 0x22, 0x95, 0x03, 0x0a, 0x0a, 0x43, 0x61, 0x63, 0x68, 0x65, 0x45, 0x76, 0x65, 0x6e, 0x74, 0x12, 0x3c, 0x0a, 0x04, 0x74, 0x79, 0x70, 0x65, 0x18, 0x01, 0x20, 0x01, 0x28, 0x0e, 0x32, 0x28, 0x2e, 0x69, 0x6e, 0x66, 0x65, 0x72, 0x65, 0x6e, 0x63, 0x65, 0x63, 0x61, 0x63, 0x68, 0x65, 0x2e, 0x76, 0x31, 0x61, 0x6c, 0x70, 0x68, 0x61, 0x31, 0x2e, 0x43, 0x61, 0x63, 0x68, @@ -1840,111 +1857,114 @@ var file_inferencecache_v1alpha1_inferencecache_proto_rawDesc = string([]byte{ 0x65, 0x73, 0x74, 0x61, 0x6d, 0x70, 0x5f, 0x75, 0x73, 0x18, 0x06, 0x20, 0x01, 0x28, 0x03, 0x52, 0x0b, 0x74, 0x69, 0x6d, 0x65, 0x73, 0x74, 0x61, 0x6d, 0x70, 0x55, 0x73, 0x12, 0x1d, 0x0a, 0x0a, 0x61, 0x64, 0x61, 0x70, 0x74, 0x65, 0x72, 0x5f, 0x69, 0x64, 0x18, 0x07, 0x20, 0x01, 0x28, 0x09, - 0x52, 0x09, 0x61, 0x64, 0x61, 0x70, 0x74, 0x65, 0x72, 0x49, 0x64, 0x22, 0x68, 0x0a, 0x04, 0x54, - 0x79, 0x70, 0x65, 0x12, 0x14, 0x0a, 0x10, 0x54, 0x59, 0x50, 0x45, 0x5f, 0x55, 0x4e, 0x53, 0x50, - 0x45, 0x43, 0x49, 0x46, 0x49, 0x45, 0x44, 0x10, 0x00, 0x12, 0x10, 0x0a, 0x0c, 0x50, 0x52, 0x45, - 0x46, 0x49, 0x58, 0x5f, 0x41, 0x44, 0x44, 0x45, 0x44, 0x10, 0x01, 0x12, 0x12, 0x0a, 0x0e, 0x50, - 0x52, 0x45, 0x46, 0x49, 0x58, 0x5f, 0x45, 0x56, 0x49, 0x43, 0x54, 0x45, 0x44, 0x10, 0x02, 0x12, - 0x13, 0x0a, 0x0f, 0x52, 0x45, 0x50, 0x4c, 0x49, 0x43, 0x41, 0x5f, 0x55, 0x50, 0x44, 0x41, 0x54, - 0x45, 0x44, 0x10, 0x03, 0x12, 0x0f, 0x0a, 0x0b, 0x41, 0x4c, 0x4c, 0x5f, 0x43, 0x4c, 0x45, 0x41, - 0x52, 0x45, 0x44, 0x10, 0x04, 0x22, 0x8d, 0x01, 0x0a, 0x13, 0x53, 0x74, 0x72, 0x65, 0x61, 0x6d, - 0x45, 0x76, 0x65, 0x6e, 0x74, 0x73, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x12, 0x19, 0x0a, - 0x08, 0x6d, 0x6f, 0x64, 0x65, 0x6c, 0x5f, 0x69, 0x64, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, 0x52, - 0x07, 0x6d, 0x6f, 0x64, 0x65, 0x6c, 0x49, 0x64, 0x12, 0x1b, 0x0a, 0x09, 0x74, 0x65, 0x6e, 0x61, - 0x6e, 0x74, 0x5f, 0x69, 0x64, 0x18, 0x02, 0x20, 0x01, 0x28, 0x09, 0x52, 0x08, 0x74, 0x65, 0x6e, - 0x61, 0x6e, 0x74, 0x49, 0x64, 0x12, 0x3e, 0x0a, 0x05, 0x74, 0x79, 0x70, 0x65, 0x73, 0x18, 0x03, - 0x20, 0x03, 0x28, 0x0e, 0x32, 0x28, 0x2e, 0x69, 0x6e, 0x66, 0x65, 0x72, 0x65, 0x6e, 0x63, 0x65, - 0x63, 0x61, 0x63, 0x68, 0x65, 0x2e, 0x76, 0x31, 0x61, 0x6c, 0x70, 0x68, 0x61, 0x31, 0x2e, 0x43, - 0x61, 0x63, 0x68, 0x65, 0x45, 0x76, 0x65, 0x6e, 0x74, 0x2e, 0x54, 0x79, 0x70, 0x65, 0x52, 0x05, - 0x74, 0x79, 0x70, 0x65, 0x73, 0x22, 0x2c, 0x0a, 0x14, 0x53, 0x74, 0x72, 0x65, 0x61, 0x6d, 0x4d, - 0x65, 0x74, 0x72, 0x69, 0x63, 0x73, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x12, 0x14, 0x0a, - 0x05, 0x6e, 0x61, 0x6d, 0x65, 0x73, 0x18, 0x01, 0x20, 0x03, 0x28, 0x09, 0x52, 0x05, 0x6e, 0x61, - 0x6d, 0x65, 0x73, 0x22, 0xe9, 0x01, 0x0a, 0x06, 0x4d, 0x65, 0x74, 0x72, 0x69, 0x63, 0x12, 0x12, - 0x0a, 0x04, 0x6e, 0x61, 0x6d, 0x65, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, 0x52, 0x04, 0x6e, 0x61, - 0x6d, 0x65, 0x12, 0x12, 0x0a, 0x04, 0x74, 0x79, 0x70, 0x65, 0x18, 0x02, 0x20, 0x01, 0x28, 0x09, - 0x52, 0x04, 0x74, 0x79, 0x70, 0x65, 0x12, 0x43, 0x0a, 0x06, 0x6c, 0x61, 0x62, 0x65, 0x6c, 0x73, - 0x18, 0x03, 0x20, 0x03, 0x28, 0x0b, 0x32, 0x2b, 0x2e, 0x69, 0x6e, 0x66, 0x65, 0x72, 0x65, 0x6e, - 0x63, 0x65, 0x63, 0x61, 0x63, 0x68, 0x65, 0x2e, 0x76, 0x31, 0x61, 0x6c, 0x70, 0x68, 0x61, 0x31, - 0x2e, 0x4d, 0x65, 0x74, 0x72, 0x69, 0x63, 0x2e, 0x4c, 0x61, 0x62, 0x65, 0x6c, 0x73, 0x45, 0x6e, - 0x74, 0x72, 0x79, 0x52, 0x06, 0x6c, 0x61, 0x62, 0x65, 0x6c, 0x73, 0x12, 0x14, 0x0a, 0x05, 0x76, - 0x61, 0x6c, 0x75, 0x65, 0x18, 0x04, 0x20, 0x01, 0x28, 0x01, 0x52, 0x05, 0x76, 0x61, 0x6c, 0x75, - 0x65, 0x12, 0x21, 0x0a, 0x0c, 0x74, 0x69, 0x6d, 0x65, 0x73, 0x74, 0x61, 0x6d, 0x70, 0x5f, 0x75, - 0x73, 0x18, 0x05, 0x20, 0x01, 0x28, 0x03, 0x52, 0x0b, 0x74, 0x69, 0x6d, 0x65, 0x73, 0x74, 0x61, - 0x6d, 0x70, 0x55, 0x73, 0x1a, 0x39, 0x0a, 0x0b, 0x4c, 0x61, 0x62, 0x65, 0x6c, 0x73, 0x45, 0x6e, - 0x74, 0x72, 0x79, 0x12, 0x10, 0x0a, 0x03, 0x6b, 0x65, 0x79, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, - 0x52, 0x03, 0x6b, 0x65, 0x79, 0x12, 0x14, 0x0a, 0x05, 0x76, 0x61, 0x6c, 0x75, 0x65, 0x18, 0x02, - 0x20, 0x01, 0x28, 0x09, 0x52, 0x05, 0x76, 0x61, 0x6c, 0x75, 0x65, 0x3a, 0x02, 0x38, 0x01, 0x22, - 0x42, 0x0a, 0x03, 0x41, 0x63, 0x6b, 0x12, 0x1a, 0x0a, 0x08, 0x61, 0x63, 0x63, 0x65, 0x70, 0x74, - 0x65, 0x64, 0x18, 0x01, 0x20, 0x01, 0x28, 0x08, 0x52, 0x08, 0x61, 0x63, 0x63, 0x65, 0x70, 0x74, - 0x65, 0x64, 0x12, 0x1f, 0x0a, 0x0b, 0x72, 0x65, 0x61, 0x73, 0x6f, 0x6e, 0x5f, 0x63, 0x6f, 0x64, - 0x65, 0x18, 0x02, 0x20, 0x01, 0x28, 0x09, 0x52, 0x0a, 0x72, 0x65, 0x61, 0x73, 0x6f, 0x6e, 0x43, - 0x6f, 0x64, 0x65, 0x2a, 0x60, 0x0a, 0x09, 0x43, 0x61, 0x63, 0x68, 0x65, 0x54, 0x69, 0x65, 0x72, - 0x12, 0x1a, 0x0a, 0x16, 0x43, 0x41, 0x43, 0x48, 0x45, 0x5f, 0x54, 0x49, 0x45, 0x52, 0x5f, 0x55, - 0x4e, 0x53, 0x50, 0x45, 0x43, 0x49, 0x46, 0x49, 0x45, 0x44, 0x10, 0x00, 0x12, 0x11, 0x0a, 0x0d, - 0x43, 0x41, 0x43, 0x48, 0x45, 0x5f, 0x54, 0x49, 0x45, 0x52, 0x5f, 0x54, 0x31, 0x10, 0x01, 0x12, - 0x11, 0x0a, 0x0d, 0x43, 0x41, 0x43, 0x48, 0x45, 0x5f, 0x54, 0x49, 0x45, 0x52, 0x5f, 0x54, 0x32, - 0x10, 0x02, 0x12, 0x11, 0x0a, 0x0d, 0x43, 0x41, 0x43, 0x48, 0x45, 0x5f, 0x54, 0x49, 0x45, 0x52, - 0x5f, 0x54, 0x33, 0x10, 0x03, 0x32, 0xcc, 0x06, 0x0a, 0x0e, 0x49, 0x6e, 0x66, 0x65, 0x72, 0x65, - 0x6e, 0x63, 0x65, 0x43, 0x61, 0x63, 0x68, 0x65, 0x12, 0x71, 0x0a, 0x0e, 0x52, 0x65, 0x6e, 0x64, - 0x65, 0x72, 0x54, 0x65, 0x6d, 0x70, 0x6c, 0x61, 0x74, 0x65, 0x12, 0x2e, 0x2e, 0x69, 0x6e, 0x66, - 0x65, 0x72, 0x65, 0x6e, 0x63, 0x65, 0x63, 0x61, 0x63, 0x68, 0x65, 0x2e, 0x76, 0x31, 0x61, 0x6c, - 0x70, 0x68, 0x61, 0x31, 0x2e, 0x52, 0x65, 0x6e, 0x64, 0x65, 0x72, 0x54, 0x65, 0x6d, 0x70, 0x6c, - 0x61, 0x74, 0x65, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x2f, 0x2e, 0x69, 0x6e, 0x66, - 0x65, 0x72, 0x65, 0x6e, 0x63, 0x65, 0x63, 0x61, 0x63, 0x68, 0x65, 0x2e, 0x76, 0x31, 0x61, 0x6c, - 0x70, 0x68, 0x61, 0x31, 0x2e, 0x52, 0x65, 0x6e, 0x64, 0x65, 0x72, 0x54, 0x65, 0x6d, 0x70, 0x6c, - 0x61, 0x74, 0x65, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x68, 0x0a, 0x0b, 0x4c, - 0x6f, 0x6f, 0x6b, 0x75, 0x70, 0x52, 0x6f, 0x75, 0x74, 0x65, 0x12, 0x2b, 0x2e, 0x69, 0x6e, 0x66, - 0x65, 0x72, 0x65, 0x6e, 0x63, 0x65, 0x63, 0x61, 0x63, 0x68, 0x65, 0x2e, 0x76, 0x31, 0x61, 0x6c, - 0x70, 0x68, 0x61, 0x31, 0x2e, 0x4c, 0x6f, 0x6f, 0x6b, 0x75, 0x70, 0x52, 0x6f, 0x75, 0x74, 0x65, - 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x2c, 0x2e, 0x69, 0x6e, 0x66, 0x65, 0x72, 0x65, - 0x6e, 0x63, 0x65, 0x63, 0x61, 0x63, 0x68, 0x65, 0x2e, 0x76, 0x31, 0x61, 0x6c, 0x70, 0x68, 0x61, - 0x31, 0x2e, 0x4c, 0x6f, 0x6f, 0x6b, 0x75, 0x70, 0x52, 0x6f, 0x75, 0x74, 0x65, 0x52, 0x65, 0x73, - 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x6e, 0x0a, 0x0d, 0x4c, 0x6f, 0x6f, 0x6b, 0x75, 0x70, 0x50, - 0x44, 0x52, 0x6f, 0x75, 0x74, 0x65, 0x12, 0x2d, 0x2e, 0x69, 0x6e, 0x66, 0x65, 0x72, 0x65, 0x6e, - 0x63, 0x65, 0x63, 0x61, 0x63, 0x68, 0x65, 0x2e, 0x76, 0x31, 0x61, 0x6c, 0x70, 0x68, 0x61, 0x31, - 0x2e, 0x4c, 0x6f, 0x6f, 0x6b, 0x75, 0x70, 0x50, 0x44, 0x52, 0x6f, 0x75, 0x74, 0x65, 0x52, 0x65, - 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x2e, 0x2e, 0x69, 0x6e, 0x66, 0x65, 0x72, 0x65, 0x6e, 0x63, - 0x65, 0x63, 0x61, 0x63, 0x68, 0x65, 0x2e, 0x76, 0x31, 0x61, 0x6c, 0x70, 0x68, 0x61, 0x31, 0x2e, - 0x4c, 0x6f, 0x6f, 0x6b, 0x75, 0x70, 0x50, 0x44, 0x52, 0x6f, 0x75, 0x74, 0x65, 0x52, 0x65, 0x73, - 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x6e, 0x0a, 0x0d, 0x47, 0x65, 0x74, 0x43, 0x61, 0x63, 0x68, - 0x65, 0x53, 0x74, 0x61, 0x74, 0x65, 0x12, 0x2d, 0x2e, 0x69, 0x6e, 0x66, 0x65, 0x72, 0x65, 0x6e, - 0x63, 0x65, 0x63, 0x61, 0x63, 0x68, 0x65, 0x2e, 0x76, 0x31, 0x61, 0x6c, 0x70, 0x68, 0x61, 0x31, - 0x2e, 0x47, 0x65, 0x74, 0x43, 0x61, 0x63, 0x68, 0x65, 0x53, 0x74, 0x61, 0x74, 0x65, 0x52, 0x65, - 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x2e, 0x2e, 0x69, 0x6e, 0x66, 0x65, 0x72, 0x65, 0x6e, 0x63, - 0x65, 0x63, 0x61, 0x63, 0x68, 0x65, 0x2e, 0x76, 0x31, 0x61, 0x6c, 0x70, 0x68, 0x61, 0x31, 0x2e, - 0x47, 0x65, 0x74, 0x43, 0x61, 0x63, 0x68, 0x65, 0x53, 0x74, 0x61, 0x74, 0x65, 0x52, 0x65, 0x73, - 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x5d, 0x0a, 0x10, 0x52, 0x65, 0x70, 0x6f, 0x72, 0x74, 0x43, - 0x61, 0x63, 0x68, 0x65, 0x53, 0x74, 0x61, 0x74, 0x65, 0x12, 0x29, 0x2e, 0x69, 0x6e, 0x66, 0x65, - 0x72, 0x65, 0x6e, 0x63, 0x65, 0x63, 0x61, 0x63, 0x68, 0x65, 0x2e, 0x76, 0x31, 0x61, 0x6c, 0x70, - 0x68, 0x61, 0x31, 0x2e, 0x43, 0x61, 0x63, 0x68, 0x65, 0x53, 0x74, 0x61, 0x74, 0x65, 0x55, 0x70, - 0x64, 0x61, 0x74, 0x65, 0x1a, 0x1c, 0x2e, 0x69, 0x6e, 0x66, 0x65, 0x72, 0x65, 0x6e, 0x63, 0x65, - 0x63, 0x61, 0x63, 0x68, 0x65, 0x2e, 0x76, 0x31, 0x61, 0x6c, 0x70, 0x68, 0x61, 0x31, 0x2e, 0x41, - 0x63, 0x6b, 0x28, 0x01, 0x12, 0x51, 0x0a, 0x0c, 0x50, 0x75, 0x62, 0x6c, 0x69, 0x73, 0x68, 0x45, - 0x76, 0x65, 0x6e, 0x74, 0x12, 0x23, 0x2e, 0x69, 0x6e, 0x66, 0x65, 0x72, 0x65, 0x6e, 0x63, 0x65, - 0x63, 0x61, 0x63, 0x68, 0x65, 0x2e, 0x76, 0x31, 0x61, 0x6c, 0x70, 0x68, 0x61, 0x31, 0x2e, 0x43, - 0x61, 0x63, 0x68, 0x65, 0x45, 0x76, 0x65, 0x6e, 0x74, 0x1a, 0x1c, 0x2e, 0x69, 0x6e, 0x66, 0x65, - 0x72, 0x65, 0x6e, 0x63, 0x65, 0x63, 0x61, 0x63, 0x68, 0x65, 0x2e, 0x76, 0x31, 0x61, 0x6c, 0x70, - 0x68, 0x61, 0x31, 0x2e, 0x41, 0x63, 0x6b, 0x12, 0x68, 0x0a, 0x11, 0x53, 0x74, 0x72, 0x65, 0x61, - 0x6d, 0x43, 0x61, 0x63, 0x68, 0x65, 0x45, 0x76, 0x65, 0x6e, 0x74, 0x73, 0x12, 0x2c, 0x2e, 0x69, + 0x52, 0x09, 0x61, 0x64, 0x61, 0x70, 0x74, 0x65, 0x72, 0x49, 0x64, 0x12, 0x25, 0x0a, 0x0e, 0x61, + 0x64, 0x61, 0x70, 0x74, 0x65, 0x72, 0x5f, 0x73, 0x63, 0x6f, 0x70, 0x65, 0x64, 0x18, 0x08, 0x20, + 0x01, 0x28, 0x08, 0x52, 0x0d, 0x61, 0x64, 0x61, 0x70, 0x74, 0x65, 0x72, 0x53, 0x63, 0x6f, 0x70, + 0x65, 0x64, 0x22, 0x68, 0x0a, 0x04, 0x54, 0x79, 0x70, 0x65, 0x12, 0x14, 0x0a, 0x10, 0x54, 0x59, + 0x50, 0x45, 0x5f, 0x55, 0x4e, 0x53, 0x50, 0x45, 0x43, 0x49, 0x46, 0x49, 0x45, 0x44, 0x10, 0x00, + 0x12, 0x10, 0x0a, 0x0c, 0x50, 0x52, 0x45, 0x46, 0x49, 0x58, 0x5f, 0x41, 0x44, 0x44, 0x45, 0x44, + 0x10, 0x01, 0x12, 0x12, 0x0a, 0x0e, 0x50, 0x52, 0x45, 0x46, 0x49, 0x58, 0x5f, 0x45, 0x56, 0x49, + 0x43, 0x54, 0x45, 0x44, 0x10, 0x02, 0x12, 0x13, 0x0a, 0x0f, 0x52, 0x45, 0x50, 0x4c, 0x49, 0x43, + 0x41, 0x5f, 0x55, 0x50, 0x44, 0x41, 0x54, 0x45, 0x44, 0x10, 0x03, 0x12, 0x0f, 0x0a, 0x0b, 0x41, + 0x4c, 0x4c, 0x5f, 0x43, 0x4c, 0x45, 0x41, 0x52, 0x45, 0x44, 0x10, 0x04, 0x22, 0x8d, 0x01, 0x0a, + 0x13, 0x53, 0x74, 0x72, 0x65, 0x61, 0x6d, 0x45, 0x76, 0x65, 0x6e, 0x74, 0x73, 0x52, 0x65, 0x71, + 0x75, 0x65, 0x73, 0x74, 0x12, 0x19, 0x0a, 0x08, 0x6d, 0x6f, 0x64, 0x65, 0x6c, 0x5f, 0x69, 0x64, + 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, 0x52, 0x07, 0x6d, 0x6f, 0x64, 0x65, 0x6c, 0x49, 0x64, 0x12, + 0x1b, 0x0a, 0x09, 0x74, 0x65, 0x6e, 0x61, 0x6e, 0x74, 0x5f, 0x69, 0x64, 0x18, 0x02, 0x20, 0x01, + 0x28, 0x09, 0x52, 0x08, 0x74, 0x65, 0x6e, 0x61, 0x6e, 0x74, 0x49, 0x64, 0x12, 0x3e, 0x0a, 0x05, + 0x74, 0x79, 0x70, 0x65, 0x73, 0x18, 0x03, 0x20, 0x03, 0x28, 0x0e, 0x32, 0x28, 0x2e, 0x69, 0x6e, + 0x66, 0x65, 0x72, 0x65, 0x6e, 0x63, 0x65, 0x63, 0x61, 0x63, 0x68, 0x65, 0x2e, 0x76, 0x31, 0x61, + 0x6c, 0x70, 0x68, 0x61, 0x31, 0x2e, 0x43, 0x61, 0x63, 0x68, 0x65, 0x45, 0x76, 0x65, 0x6e, 0x74, + 0x2e, 0x54, 0x79, 0x70, 0x65, 0x52, 0x05, 0x74, 0x79, 0x70, 0x65, 0x73, 0x22, 0x2c, 0x0a, 0x14, + 0x53, 0x74, 0x72, 0x65, 0x61, 0x6d, 0x4d, 0x65, 0x74, 0x72, 0x69, 0x63, 0x73, 0x52, 0x65, 0x71, + 0x75, 0x65, 0x73, 0x74, 0x12, 0x14, 0x0a, 0x05, 0x6e, 0x61, 0x6d, 0x65, 0x73, 0x18, 0x01, 0x20, + 0x03, 0x28, 0x09, 0x52, 0x05, 0x6e, 0x61, 0x6d, 0x65, 0x73, 0x22, 0xe9, 0x01, 0x0a, 0x06, 0x4d, + 0x65, 0x74, 0x72, 0x69, 0x63, 0x12, 0x12, 0x0a, 0x04, 0x6e, 0x61, 0x6d, 0x65, 0x18, 0x01, 0x20, + 0x01, 0x28, 0x09, 0x52, 0x04, 0x6e, 0x61, 0x6d, 0x65, 0x12, 0x12, 0x0a, 0x04, 0x74, 0x79, 0x70, + 0x65, 0x18, 0x02, 0x20, 0x01, 0x28, 0x09, 0x52, 0x04, 0x74, 0x79, 0x70, 0x65, 0x12, 0x43, 0x0a, + 0x06, 0x6c, 0x61, 0x62, 0x65, 0x6c, 0x73, 0x18, 0x03, 0x20, 0x03, 0x28, 0x0b, 0x32, 0x2b, 0x2e, + 0x69, 0x6e, 0x66, 0x65, 0x72, 0x65, 0x6e, 0x63, 0x65, 0x63, 0x61, 0x63, 0x68, 0x65, 0x2e, 0x76, + 0x31, 0x61, 0x6c, 0x70, 0x68, 0x61, 0x31, 0x2e, 0x4d, 0x65, 0x74, 0x72, 0x69, 0x63, 0x2e, 0x4c, + 0x61, 0x62, 0x65, 0x6c, 0x73, 0x45, 0x6e, 0x74, 0x72, 0x79, 0x52, 0x06, 0x6c, 0x61, 0x62, 0x65, + 0x6c, 0x73, 0x12, 0x14, 0x0a, 0x05, 0x76, 0x61, 0x6c, 0x75, 0x65, 0x18, 0x04, 0x20, 0x01, 0x28, + 0x01, 0x52, 0x05, 0x76, 0x61, 0x6c, 0x75, 0x65, 0x12, 0x21, 0x0a, 0x0c, 0x74, 0x69, 0x6d, 0x65, + 0x73, 0x74, 0x61, 0x6d, 0x70, 0x5f, 0x75, 0x73, 0x18, 0x05, 0x20, 0x01, 0x28, 0x03, 0x52, 0x0b, + 0x74, 0x69, 0x6d, 0x65, 0x73, 0x74, 0x61, 0x6d, 0x70, 0x55, 0x73, 0x1a, 0x39, 0x0a, 0x0b, 0x4c, + 0x61, 0x62, 0x65, 0x6c, 0x73, 0x45, 0x6e, 0x74, 0x72, 0x79, 0x12, 0x10, 0x0a, 0x03, 0x6b, 0x65, + 0x79, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, 0x52, 0x03, 0x6b, 0x65, 0x79, 0x12, 0x14, 0x0a, 0x05, + 0x76, 0x61, 0x6c, 0x75, 0x65, 0x18, 0x02, 0x20, 0x01, 0x28, 0x09, 0x52, 0x05, 0x76, 0x61, 0x6c, + 0x75, 0x65, 0x3a, 0x02, 0x38, 0x01, 0x22, 0x42, 0x0a, 0x03, 0x41, 0x63, 0x6b, 0x12, 0x1a, 0x0a, + 0x08, 0x61, 0x63, 0x63, 0x65, 0x70, 0x74, 0x65, 0x64, 0x18, 0x01, 0x20, 0x01, 0x28, 0x08, 0x52, + 0x08, 0x61, 0x63, 0x63, 0x65, 0x70, 0x74, 0x65, 0x64, 0x12, 0x1f, 0x0a, 0x0b, 0x72, 0x65, 0x61, + 0x73, 0x6f, 0x6e, 0x5f, 0x63, 0x6f, 0x64, 0x65, 0x18, 0x02, 0x20, 0x01, 0x28, 0x09, 0x52, 0x0a, + 0x72, 0x65, 0x61, 0x73, 0x6f, 0x6e, 0x43, 0x6f, 0x64, 0x65, 0x2a, 0x60, 0x0a, 0x09, 0x43, 0x61, + 0x63, 0x68, 0x65, 0x54, 0x69, 0x65, 0x72, 0x12, 0x1a, 0x0a, 0x16, 0x43, 0x41, 0x43, 0x48, 0x45, + 0x5f, 0x54, 0x49, 0x45, 0x52, 0x5f, 0x55, 0x4e, 0x53, 0x50, 0x45, 0x43, 0x49, 0x46, 0x49, 0x45, + 0x44, 0x10, 0x00, 0x12, 0x11, 0x0a, 0x0d, 0x43, 0x41, 0x43, 0x48, 0x45, 0x5f, 0x54, 0x49, 0x45, + 0x52, 0x5f, 0x54, 0x31, 0x10, 0x01, 0x12, 0x11, 0x0a, 0x0d, 0x43, 0x41, 0x43, 0x48, 0x45, 0x5f, + 0x54, 0x49, 0x45, 0x52, 0x5f, 0x54, 0x32, 0x10, 0x02, 0x12, 0x11, 0x0a, 0x0d, 0x43, 0x41, 0x43, + 0x48, 0x45, 0x5f, 0x54, 0x49, 0x45, 0x52, 0x5f, 0x54, 0x33, 0x10, 0x03, 0x32, 0xcc, 0x06, 0x0a, + 0x0e, 0x49, 0x6e, 0x66, 0x65, 0x72, 0x65, 0x6e, 0x63, 0x65, 0x43, 0x61, 0x63, 0x68, 0x65, 0x12, + 0x71, 0x0a, 0x0e, 0x52, 0x65, 0x6e, 0x64, 0x65, 0x72, 0x54, 0x65, 0x6d, 0x70, 0x6c, 0x61, 0x74, + 0x65, 0x12, 0x2e, 0x2e, 0x69, 0x6e, 0x66, 0x65, 0x72, 0x65, 0x6e, 0x63, 0x65, 0x63, 0x61, 0x63, + 0x68, 0x65, 0x2e, 0x76, 0x31, 0x61, 0x6c, 0x70, 0x68, 0x61, 0x31, 0x2e, 0x52, 0x65, 0x6e, 0x64, + 0x65, 0x72, 0x54, 0x65, 0x6d, 0x70, 0x6c, 0x61, 0x74, 0x65, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, + 0x74, 0x1a, 0x2f, 0x2e, 0x69, 0x6e, 0x66, 0x65, 0x72, 0x65, 0x6e, 0x63, 0x65, 0x63, 0x61, 0x63, + 0x68, 0x65, 0x2e, 0x76, 0x31, 0x61, 0x6c, 0x70, 0x68, 0x61, 0x31, 0x2e, 0x52, 0x65, 0x6e, 0x64, + 0x65, 0x72, 0x54, 0x65, 0x6d, 0x70, 0x6c, 0x61, 0x74, 0x65, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, + 0x73, 0x65, 0x12, 0x68, 0x0a, 0x0b, 0x4c, 0x6f, 0x6f, 0x6b, 0x75, 0x70, 0x52, 0x6f, 0x75, 0x74, + 0x65, 0x12, 0x2b, 0x2e, 0x69, 0x6e, 0x66, 0x65, 0x72, 0x65, 0x6e, 0x63, 0x65, 0x63, 0x61, 0x63, + 0x68, 0x65, 0x2e, 0x76, 0x31, 0x61, 0x6c, 0x70, 0x68, 0x61, 0x31, 0x2e, 0x4c, 0x6f, 0x6f, 0x6b, + 0x75, 0x70, 0x52, 0x6f, 0x75, 0x74, 0x65, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x2c, + 0x2e, 0x69, 0x6e, 0x66, 0x65, 0x72, 0x65, 0x6e, 0x63, 0x65, 0x63, 0x61, 0x63, 0x68, 0x65, 0x2e, + 0x76, 0x31, 0x61, 0x6c, 0x70, 0x68, 0x61, 0x31, 0x2e, 0x4c, 0x6f, 0x6f, 0x6b, 0x75, 0x70, 0x52, + 0x6f, 0x75, 0x74, 0x65, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x6e, 0x0a, 0x0d, + 0x4c, 0x6f, 0x6f, 0x6b, 0x75, 0x70, 0x50, 0x44, 0x52, 0x6f, 0x75, 0x74, 0x65, 0x12, 0x2d, 0x2e, + 0x69, 0x6e, 0x66, 0x65, 0x72, 0x65, 0x6e, 0x63, 0x65, 0x63, 0x61, 0x63, 0x68, 0x65, 0x2e, 0x76, + 0x31, 0x61, 0x6c, 0x70, 0x68, 0x61, 0x31, 0x2e, 0x4c, 0x6f, 0x6f, 0x6b, 0x75, 0x70, 0x50, 0x44, + 0x52, 0x6f, 0x75, 0x74, 0x65, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x2e, 0x2e, 0x69, 0x6e, 0x66, 0x65, 0x72, 0x65, 0x6e, 0x63, 0x65, 0x63, 0x61, 0x63, 0x68, 0x65, 0x2e, 0x76, 0x31, - 0x61, 0x6c, 0x70, 0x68, 0x61, 0x31, 0x2e, 0x53, 0x74, 0x72, 0x65, 0x61, 0x6d, 0x45, 0x76, 0x65, - 0x6e, 0x74, 0x73, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x23, 0x2e, 0x69, 0x6e, 0x66, - 0x65, 0x72, 0x65, 0x6e, 0x63, 0x65, 0x63, 0x61, 0x63, 0x68, 0x65, 0x2e, 0x76, 0x31, 0x61, 0x6c, - 0x70, 0x68, 0x61, 0x31, 0x2e, 0x43, 0x61, 0x63, 0x68, 0x65, 0x45, 0x76, 0x65, 0x6e, 0x74, 0x30, - 0x01, 0x12, 0x61, 0x0a, 0x0d, 0x53, 0x74, 0x72, 0x65, 0x61, 0x6d, 0x4d, 0x65, 0x74, 0x72, 0x69, - 0x63, 0x73, 0x12, 0x2d, 0x2e, 0x69, 0x6e, 0x66, 0x65, 0x72, 0x65, 0x6e, 0x63, 0x65, 0x63, 0x61, - 0x63, 0x68, 0x65, 0x2e, 0x76, 0x31, 0x61, 0x6c, 0x70, 0x68, 0x61, 0x31, 0x2e, 0x53, 0x74, 0x72, - 0x65, 0x61, 0x6d, 0x4d, 0x65, 0x74, 0x72, 0x69, 0x63, 0x73, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, - 0x74, 0x1a, 0x1f, 0x2e, 0x69, 0x6e, 0x66, 0x65, 0x72, 0x65, 0x6e, 0x63, 0x65, 0x63, 0x61, 0x63, - 0x68, 0x65, 0x2e, 0x76, 0x31, 0x61, 0x6c, 0x70, 0x68, 0x61, 0x31, 0x2e, 0x4d, 0x65, 0x74, 0x72, - 0x69, 0x63, 0x30, 0x01, 0x42, 0x6f, 0x5a, 0x6d, 0x67, 0x69, 0x74, 0x68, 0x75, 0x62, 0x2e, 0x63, - 0x6f, 0x6d, 0x2f, 0x63, 0x61, 0x63, 0x68, 0x65, 0x62, 0x6f, 0x78, 0x2d, 0x70, 0x72, 0x6f, 0x6a, - 0x65, 0x63, 0x74, 0x2f, 0x69, 0x6e, 0x66, 0x65, 0x72, 0x65, 0x6e, 0x63, 0x65, 0x2d, 0x63, 0x61, - 0x63, 0x68, 0x65, 0x2f, 0x70, 0x6b, 0x67, 0x2f, 0x73, 0x65, 0x72, 0x76, 0x65, 0x72, 0x2f, 0x70, - 0x72, 0x6f, 0x74, 0x6f, 0x2f, 0x69, 0x6e, 0x66, 0x65, 0x72, 0x65, 0x6e, 0x63, 0x65, 0x63, 0x61, - 0x63, 0x68, 0x65, 0x2f, 0x76, 0x31, 0x61, 0x6c, 0x70, 0x68, 0x61, 0x31, 0x3b, 0x69, 0x6e, 0x66, - 0x65, 0x72, 0x65, 0x6e, 0x63, 0x65, 0x63, 0x61, 0x63, 0x68, 0x65, 0x76, 0x31, 0x61, 0x6c, 0x70, - 0x68, 0x61, 0x31, 0x70, 0x62, 0x62, 0x06, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x33, + 0x61, 0x6c, 0x70, 0x68, 0x61, 0x31, 0x2e, 0x4c, 0x6f, 0x6f, 0x6b, 0x75, 0x70, 0x50, 0x44, 0x52, + 0x6f, 0x75, 0x74, 0x65, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x6e, 0x0a, 0x0d, + 0x47, 0x65, 0x74, 0x43, 0x61, 0x63, 0x68, 0x65, 0x53, 0x74, 0x61, 0x74, 0x65, 0x12, 0x2d, 0x2e, + 0x69, 0x6e, 0x66, 0x65, 0x72, 0x65, 0x6e, 0x63, 0x65, 0x63, 0x61, 0x63, 0x68, 0x65, 0x2e, 0x76, + 0x31, 0x61, 0x6c, 0x70, 0x68, 0x61, 0x31, 0x2e, 0x47, 0x65, 0x74, 0x43, 0x61, 0x63, 0x68, 0x65, + 0x53, 0x74, 0x61, 0x74, 0x65, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x2e, 0x2e, 0x69, + 0x6e, 0x66, 0x65, 0x72, 0x65, 0x6e, 0x63, 0x65, 0x63, 0x61, 0x63, 0x68, 0x65, 0x2e, 0x76, 0x31, + 0x61, 0x6c, 0x70, 0x68, 0x61, 0x31, 0x2e, 0x47, 0x65, 0x74, 0x43, 0x61, 0x63, 0x68, 0x65, 0x53, + 0x74, 0x61, 0x74, 0x65, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x5d, 0x0a, 0x10, + 0x52, 0x65, 0x70, 0x6f, 0x72, 0x74, 0x43, 0x61, 0x63, 0x68, 0x65, 0x53, 0x74, 0x61, 0x74, 0x65, + 0x12, 0x29, 0x2e, 0x69, 0x6e, 0x66, 0x65, 0x72, 0x65, 0x6e, 0x63, 0x65, 0x63, 0x61, 0x63, 0x68, + 0x65, 0x2e, 0x76, 0x31, 0x61, 0x6c, 0x70, 0x68, 0x61, 0x31, 0x2e, 0x43, 0x61, 0x63, 0x68, 0x65, + 0x53, 0x74, 0x61, 0x74, 0x65, 0x55, 0x70, 0x64, 0x61, 0x74, 0x65, 0x1a, 0x1c, 0x2e, 0x69, 0x6e, + 0x66, 0x65, 0x72, 0x65, 0x6e, 0x63, 0x65, 0x63, 0x61, 0x63, 0x68, 0x65, 0x2e, 0x76, 0x31, 0x61, + 0x6c, 0x70, 0x68, 0x61, 0x31, 0x2e, 0x41, 0x63, 0x6b, 0x28, 0x01, 0x12, 0x51, 0x0a, 0x0c, 0x50, + 0x75, 0x62, 0x6c, 0x69, 0x73, 0x68, 0x45, 0x76, 0x65, 0x6e, 0x74, 0x12, 0x23, 0x2e, 0x69, 0x6e, + 0x66, 0x65, 0x72, 0x65, 0x6e, 0x63, 0x65, 0x63, 0x61, 0x63, 0x68, 0x65, 0x2e, 0x76, 0x31, 0x61, + 0x6c, 0x70, 0x68, 0x61, 0x31, 0x2e, 0x43, 0x61, 0x63, 0x68, 0x65, 0x45, 0x76, 0x65, 0x6e, 0x74, + 0x1a, 0x1c, 0x2e, 0x69, 0x6e, 0x66, 0x65, 0x72, 0x65, 0x6e, 0x63, 0x65, 0x63, 0x61, 0x63, 0x68, + 0x65, 0x2e, 0x76, 0x31, 0x61, 0x6c, 0x70, 0x68, 0x61, 0x31, 0x2e, 0x41, 0x63, 0x6b, 0x12, 0x68, + 0x0a, 0x11, 0x53, 0x74, 0x72, 0x65, 0x61, 0x6d, 0x43, 0x61, 0x63, 0x68, 0x65, 0x45, 0x76, 0x65, + 0x6e, 0x74, 0x73, 0x12, 0x2c, 0x2e, 0x69, 0x6e, 0x66, 0x65, 0x72, 0x65, 0x6e, 0x63, 0x65, 0x63, + 0x61, 0x63, 0x68, 0x65, 0x2e, 0x76, 0x31, 0x61, 0x6c, 0x70, 0x68, 0x61, 0x31, 0x2e, 0x53, 0x74, + 0x72, 0x65, 0x61, 0x6d, 0x45, 0x76, 0x65, 0x6e, 0x74, 0x73, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, + 0x74, 0x1a, 0x23, 0x2e, 0x69, 0x6e, 0x66, 0x65, 0x72, 0x65, 0x6e, 0x63, 0x65, 0x63, 0x61, 0x63, + 0x68, 0x65, 0x2e, 0x76, 0x31, 0x61, 0x6c, 0x70, 0x68, 0x61, 0x31, 0x2e, 0x43, 0x61, 0x63, 0x68, + 0x65, 0x45, 0x76, 0x65, 0x6e, 0x74, 0x30, 0x01, 0x12, 0x61, 0x0a, 0x0d, 0x53, 0x74, 0x72, 0x65, + 0x61, 0x6d, 0x4d, 0x65, 0x74, 0x72, 0x69, 0x63, 0x73, 0x12, 0x2d, 0x2e, 0x69, 0x6e, 0x66, 0x65, + 0x72, 0x65, 0x6e, 0x63, 0x65, 0x63, 0x61, 0x63, 0x68, 0x65, 0x2e, 0x76, 0x31, 0x61, 0x6c, 0x70, + 0x68, 0x61, 0x31, 0x2e, 0x53, 0x74, 0x72, 0x65, 0x61, 0x6d, 0x4d, 0x65, 0x74, 0x72, 0x69, 0x63, + 0x73, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x1f, 0x2e, 0x69, 0x6e, 0x66, 0x65, 0x72, + 0x65, 0x6e, 0x63, 0x65, 0x63, 0x61, 0x63, 0x68, 0x65, 0x2e, 0x76, 0x31, 0x61, 0x6c, 0x70, 0x68, + 0x61, 0x31, 0x2e, 0x4d, 0x65, 0x74, 0x72, 0x69, 0x63, 0x30, 0x01, 0x42, 0x6f, 0x5a, 0x6d, 0x67, + 0x69, 0x74, 0x68, 0x75, 0x62, 0x2e, 0x63, 0x6f, 0x6d, 0x2f, 0x63, 0x61, 0x63, 0x68, 0x65, 0x62, + 0x6f, 0x78, 0x2d, 0x70, 0x72, 0x6f, 0x6a, 0x65, 0x63, 0x74, 0x2f, 0x69, 0x6e, 0x66, 0x65, 0x72, + 0x65, 0x6e, 0x63, 0x65, 0x2d, 0x63, 0x61, 0x63, 0x68, 0x65, 0x2f, 0x70, 0x6b, 0x67, 0x2f, 0x73, + 0x65, 0x72, 0x76, 0x65, 0x72, 0x2f, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x2f, 0x69, 0x6e, 0x66, 0x65, + 0x72, 0x65, 0x6e, 0x63, 0x65, 0x63, 0x61, 0x63, 0x68, 0x65, 0x2f, 0x76, 0x31, 0x61, 0x6c, 0x70, + 0x68, 0x61, 0x31, 0x3b, 0x69, 0x6e, 0x66, 0x65, 0x72, 0x65, 0x6e, 0x63, 0x65, 0x63, 0x61, 0x63, + 0x68, 0x65, 0x76, 0x31, 0x61, 0x6c, 0x70, 0x68, 0x61, 0x31, 0x70, 0x62, 0x62, 0x06, 0x70, 0x72, + 0x6f, 0x74, 0x6f, 0x33, }) var ( diff --git a/pkg/server/server_test.go b/pkg/server/server_test.go index af3428b0..dc9c95ff 100644 --- a/pkg/server/server_test.go +++ b/pkg/server/server_test.go @@ -1454,6 +1454,42 @@ func TestPublishEventAppliesToIndex(t *testing.T) { } } +// TestPublishEventBaseEvictionScopedSparesLoRA pins the service→index +// translation: a base-model PREFIX_EVICTED carrying adapter_id="" with +// adapter_scoped=true drops ONLY the base ("") partition. The handler must +// translate adapter_scoped into index.Event.AdapterScoped — reading the adapter_id +// VALUE ("") instead would misclassify a base-scoped eviction as the legacy +// wildcard and wrongly sweep the co-resident LoRA hint for the same hash. This +// exercises the handler in-process (a real icpb.CacheEvent → index.Event); the +// actual gRPC wire is covered end-to-end by the install-smoke adapter_scoped probe. +func TestPublishEventBaseEvictionScopedSparesLoRA(t *testing.T) { + svc := newTestService() + for _, a := range []string{"", "sql-lora"} { + svc.index.Ingest(index.Update{ + ReplicaID: "replica-0", Model: "m", Tenant: "t", HashScheme: "vllm", + Adapter: a, + Prefixes: []index.PrefixRef{{PrefixHash: []byte("p"), TokenCount: 10}}, + }) + } + ack, err := svc.PublishEvent(context.Background(), &icpb.CacheEvent{ + Type: icpb.CacheEvent_PREFIX_EVICTED, ReplicaId: "replica-0", + ModelId: "m", TenantId: "t", PrefixHash: []byte("p"), + AdapterId: "", AdapterScoped: true, // base partition only, not the wildcard + }) + if err != nil { + t.Fatalf("PublishEvent: %v", err) + } + if !ack.GetAccepted() { + t.Fatalf("ack.Accepted = false, want true") + } + if got := svc.index.Lookup(index.LookupRequest{Model: "m", Tenant: "t", HashScheme: "vllm", Adapter: "", PrefixHash: []byte("p")}); len(got) != 0 { + t.Errorf("base entry survived its own eviction: %+v", got) + } + if got := svc.index.Lookup(index.LookupRequest{Model: "m", Tenant: "t", HashScheme: "vllm", Adapter: "sql-lora", PrefixHash: []byte("p")}); len(got) != 1 { + t.Errorf("sql-lora entry = %+v, want it untouched by the base-model eviction", got) + } +} + func TestEventTypeFromProto(t *testing.T) { cases := map[icpb.CacheEvent_Type]index.EventType{ icpb.CacheEvent_PREFIX_ADDED: index.EventPrefixAdded, diff --git a/proto/inferencecache/v1alpha1/inferencecache.proto b/proto/inferencecache/v1alpha1/inferencecache.proto index 8e7e5fa5..2718e646 100644 --- a/proto/inferencecache/v1alpha1/inferencecache.proto +++ b/proto/inferencecache/v1alpha1/inferencecache.proto @@ -297,15 +297,25 @@ message CacheEvent { string tenant_id = 4; bytes prefix_hash = 5; int64 timestamp_us = 6; - // Adapter partition a PREFIX_EVICTED removal targets. Set, the server drops - // the prefix only from that partition; empty, it drops the prefix across - // EVERY adapter partition — the conservative legacy behavior (removal is - // soft state: an over-eager drop costs a cache miss, never a wrong answer), - // and identical to today for producers that never populate the field. - // Ignored by ALL_CLEARED (a cache flush clears the replica across every - // adapter) and by REPLICA_UPDATED (liveness is adapter-independent). - // Additive, v1alpha1-compatible. + // Adapter partition a PREFIX_EVICTED removal targets. Honored when + // adapter_scoped is set, OR when non-empty (a pre-adapter_scoped producer that + // named a LoRA adapter). "" is the base-model / non-LoRA partition. Ignored by + // ALL_CLEARED (a cache flush clears the replica across every adapter) and by + // REPLICA_UPDATED (liveness is adapter-independent). Additive, + // v1alpha1-compatible. string adapter_id = 7; + // adapter_scoped marks adapter_id as authoritative for a PREFIX_EVICTED so that + // adapter_id="" targets the base-model partition instead of sweeping every + // partition — the signal that lets a base-model eviction avoid discarding live + // LoRA hints for the same token-derived hash. An adapter-aware producer always + // sets it. The server ALSO narrows on a non-empty adapter_id when this is false, + // so a pre-adapter_scoped producer's LoRA eviction keeps narrowing on a + // mixed-version fleet; only an empty adapter_id with adapter_scoped=false takes + // the conservative cross-partition sweep. Removal is soft state: an over-eager + // drop costs a cache miss, never a wrong answer. A new bool rather than making + // adapter_id `optional`, which the wire-compat gate rejects as a cardinality + // change. Additive; ignored outside PREFIX_EVICTED. + bool adapter_scoped = 8; } message StreamEventsRequest {