Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make text cleaning optional. #57

Open
ruben-dedoncker opened this issue Dec 26, 2022 · 1 comment
Open

Make text cleaning optional. #57

ruben-dedoncker opened this issue Dec 26, 2022 · 1 comment

Comments

@ruben-dedoncker
Copy link

Is your feature request related to a problem? Please describe.
The cleaning of the text makes it impossible to link annotated spans to the character indices of the original text. This in turn makes it impossible to compare the performance of this model to other ner models.

Describe the solution you'd like
Make the text cleaning step optional. When the cleaning step is omitted, then abv_text == immutable_text.

Describe alternatives you've considered
Provide additional metadata containing the start and end character indices of each annotated span linked to the original text rather in addition to the boundaries linked to the cleaned text

@AnAnalogGuy
Copy link

You could instantiate your own empty skillNer.cleaner.Cleaner to bypass text cleaning. However you also want to protect abv_text from later processing, which would require some changes in the code.

grafik

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants