Skip to content

Commit f777109

Browse files
authored
[Rust] Bump huggingface tokenizer to 0.21.2 (#81)
This PR bumps the huggignface tokenizer dependency to version 0.21.2 to address the tokenizer build issue in MLC.
1 parent e896487 commit f777109

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

rust/Cargo.toml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@ crate-type = ["staticlib"]
88

99
[dependencies]
1010

11-
tokenizers = { version = "0.21.1", default-features = false, features = ["onig"] }
11+
tokenizers = { version = "0.21.2", default-features = false, features = ["onig"] }
1212
serde = { version = "1.0", features = [ "derive" ] }
1313
serde_json = "1.0"
1414
ahash = "0.8.12"

0 commit comments

Comments
 (0)