Skip to content

fix: include Korean Hangul and Japanese kana in CJK token heuristic#2

Open
haosenwang1018 wants to merge 1 commit intoaeromomo:mainfrom
haosenwang1018:fix/cjk-regex-korean-japanese
Open

fix: include Korean Hangul and Japanese kana in CJK token heuristic#2
haosenwang1018 wants to merge 1 commit intoaeromomo:mainfrom
haosenwang1018:fix/cjk-regex-korean-japanese

Commits

Commits on Feb 24, 2026