This package (website: https://rezonators.github.io/segsimflex/index.html) aims to calculate a flexible segmentation similarity measure and a corresponding inter-annotator agreement score (between two annotators). It is currently under development. This score is similar to lexical edit distance (Fournier & Inkpen 2013), but can evaluate similarity between two segmentations of the same text with multiple, mutually exclusive boundary types, and also account for varying identifiability and confusability between boundary types.
If you wish to cite this package, please minimally cite:
Lai, Ryan Ka Yau, Yujie Li and Shujie Zhang (2023). Text Segmentation Similarity Revisited: A Flexible Distance-based Approach for Multiple Boundary Types. Proceedings of the Society for Computation in Linguistics 6. doi:10.7275/fk79-fv58
Optionally, you may also cite:
Li, Yujie, Ryan Ka Yau Lai (2023). segsimflex: Similarity Score and Inter-Annotator Agreement Calculation. R package version 0.1.3.