trying something out

[ArthurZucker]

What does this PR do?

Update

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[github-actions]

Failure summary

By test

Test	Failures	Full error(s)
tests/models/llama/test_tokenization_llama.py::LlamaTokenizationTest::test_add_prefix_space	1	1× /usr/local/lib/python3.10/unittest/case.py:675: AssertionError: Lists differ: [1, 29950, 1032, 920, 526, 366, 2599] != [1, 18637, 920, 526, 366, 2599]
tests/models/llama/test_tokenization_llama.py::LlamaIntegrationTest::test_some_edge_cases	1	1× /usr/local/lib/python3.10/unittest/case.py:675: AssertionError: Lists differ: ['▁'] != ['▁▁']
tests/models/llama/test_tokenization_llama.py::CommonSpmIntegrationTests::test_add_dummy_prefix	1	1× /usr/local/lib/python3.10/unittest/case.py:675: AssertionError: Lists differ: [4, 156, 86, 20] != [7, 4, 156, 86, 20]
tests/models/llama/test_tokenization_llama.py::CommonSpmIntegrationTests::test_character_after_special_token	1	1× /usr/local/lib/python3.10/unittest/case.py:675: AssertionError: Lists differ: [262, 15, 30, 1, 100] != [156, 30, 1, 100]
tests/models/llama/test_tokenization_llama.py::CommonSpmIntegrationTests::test_special_tokens_strip	1	1× /usr/local/lib/python3.10/unittest/case.py:675: AssertionError: Lists differ: [516, 1, 156] != [284, 1, 156]
tests/models/glm46v/test_processor_glm46v.py::Glm46VProcessorTest::test_apply_chat_template_video_frame_sampling	1	1× /usr/local/lib/python3.10/site-packages/torch/_ops.py:841: RuntimeError: expected scalar type Long but found Double

By model

Model	Failures	Full error(s)
llama	5	1× /usr/local/lib/python3.10/unittest/case.py:675: AssertionError: Lists differ: [1, 29950, 1032, 920, 526, 366, 2599] != [1, 18637, 920, 526, 366, 2599]; 1× /usr/local/lib/python3.10/unittest/case.py:675: AssertionError: Lists differ: ['▁'] != ['▁▁']; 1× /usr/local/lib/python3.10/unittest/case.py:675: AssertionError: Lists differ: [4, 156, 86, 20] != [7, 4, 156, 86, 20]; 1× /usr/local/lib/python3.10/unittest/case.py:675: AssertionError: Lists differ: [262, 15, 30, 1, 100] != [156, 30, 1, 100]; 1× /usr/local/lib/python3.10/unittest/case.py:675: AssertionError: Lists differ: [516, 1, 156] != [284, 1, 156]
glm46v	1	1× /usr/local/lib/python3.10/site-packages/torch/_ops.py:841: RuntimeError: expected scalar type Long but found Double

[github-actions]

View the CircleCI test collection helper for this PR:\n\nhttps://huggingface.co/spaces/transformers-community/circleci-test-collection-helper?repo=huggingface%2Ftransformers&pr=42240&sha=9664edac08729aabb154073c9cd52e944b7cbb3d

[ydshieh]

thanks. will finalize this on my side

[github-actions]

View the CircleCI test collection helper for this PR:\n\nhttps://huggingface.co/spaces/transformers-community/circleci-test-collection-helper?repo=huggingface%2Ftransformers&pr=42240&sha=215ea77a013c435cfc01194c5b089173cf96ada0

[github-actions]

View the CircleCI test collection helper for this PR:\n\nhttps://huggingface.co/spaces/transformers-community/circleci-test-collection-helper?repo=huggingface%2Ftransformers&pr=42240&sha=b2f0d8e0d4d0795f463b975be208866ec04fc587

[github-actions]

View the CircleCI test collection helper for this PR:\n\nhttps://huggingface.co/spaces/transformers-community/circleci-test-collection-helper?repo=huggingface%2Ftransformers&pr=42240&sha=b5fdb80c080328bb4731b25c6ac2211b02c5296b

[github-actions]

[For maintainers] Suggested jobs to run (before merge)

run-slow: llama, mixtral