Skip to content

Commit 1f18375

Browse files
pierDipivMaroon
authored andcommitted
Update documentation
Signed-off-by: Pierangelo Di Pilato <[email protected]>
1 parent 7b56484 commit 1f18375

File tree

2 files changed

+7
-4
lines changed

2 files changed

+7
-4
lines changed

deploy/config/sim-epp-kvcache-config.yaml

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -15,7 +15,8 @@ plugins:
1515
blockSize: 16 # must match vLLM block size if not default (16)
1616
hashSeed: "42" # must match PYTHONHASHSEED in vLLM pods
1717
tokenizersPoolConfig:
18-
tokenizersCacheDir: "/cache/tokenizers"
18+
hf:
19+
tokenizersCacheDir: "/cache/tokenizers"
1920
kvBlockIndexConfig:
2021
enableMetrics: false # enable kv-block index metrics (prometheus)
2122
metricsLoggingInterval: 6000000000 # log kv-block metrics as well (1m in nanoseconds)

docs/architecture.md

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -330,7 +330,8 @@ plugins:
330330
blockSize: 64
331331
hashSeed: "12345"
332332
tokenizersPoolConfig:
333-
huggingFaceToken: your_hf_token_here # automatically set by `HF_TOKEN` environment variable
333+
hf:
334+
huggingFaceToken: your_hf_token_here # automatically set by `HF_TOKEN` environment variable
334335
kvBlockIndexConfig:
335336
enableMetrics: true
336337
```
@@ -359,8 +360,9 @@ plugins:
359360
enableMetrics: true
360361
tokenizersPoolConfig:
361362
workersCount: 8
362-
huggingFaceToken: your_hf_token_here # automatically set by `HF_TOKEN` environment variable
363-
tokenizersCacheDir: /tmp/tokenizers
363+
hf:
364+
huggingFaceToken: your_hf_token_here # automatically set by `HF_TOKEN` environment variable
365+
tokenizersCacheDir: /tmp/tokenizers
364366
```
365367
366368
---

0 commit comments

Comments
 (0)