CI

BOS/EOS/PAD options in `tokens` cli; speed up tokenization by segmenting paragraphs. #427

Sign in to view logs

Triggered via pull request January 20, 2024 23:01

soldni

synchronize #102

soldni/tokenizer

Status Success

Total duration 4m 21s

Artifacts –

This run and associated checks have been archived and are scheduled for deletion. Learn more about checks retention

CI.yml

on: pull_request

Matrix: build-linux

Matrix: build-macos

Matrix: build-windows

Prepare Virtual Env

Matrix: tests