Update logprobs to vocab size

XkunW · web-flow · commit f1b3360379c8 · 2024-09-04T15:15:15.000-04:00
diff --git a/examples/logits/logits.py b/examples/logits/logits.py
@@ -7,7 +7,7 @@
     model="Meta-Llama-3.1-8B-Instruct",
     prompt="Where is the capital of Canada?",
     max_tokens=1,
-    logprobs=32000,  # Set to model vocab size to get logits
+    logprobs=128256,  # Set to model vocab size to get logits
 )
 
 print(completion.choices[0].logprobs)

Original file line number	Diff line number	Diff line change
`@@ -7,7 +7,7 @@`
`7`	`7`	`model="Meta-Llama-3.1-8B-Instruct",`
`8`	`8`	`prompt="Where is the capital of Canada?",`
`9`	`9`	`max_tokens=1,`
`10`		`- logprobs=32000, # Set to model vocab size to get logits`
	`10`	`+ logprobs=128256, # Set to model vocab size to get logits`
`11`	`11`	`)`
`12`	`12`
`13`	`13`	`print(completion.choices[0].logprobs)`