This is a free/libre open source morphology of Finnish: a database, tools and APIs. Everything you need to build NLP applications processing Finnish language words and texts.
- 🇫🇮 high-quality Finnish text analysis and generation
- 🩸 bleeding edge
- ⚡ blazing fast
I try to keep this README very condensed for github.
For more detailed information, see github pages for
omorfi.
Citation information is available in github's cite this repository function, backed by the CITATION.cff. For further details, see omorfi articles.
Omorfi source packages can be downloaded from github:
or the most current version using git. For more information, see Release policy
- hfst-3.15 or greater,
- python-3.5 or greater,
- hfst-python,
- C++ compiler and libtool
- GNU autoconf-2.64, automake-1.12; compatible pkg-config implementation
Optionally:
- VISL CG 3
- hfst-ospell-0.2.0 or greater needed for spell-checking
- Java 7, or greater, for Java bindings
For detailed instructions and explanations of different options, see Installation instructions on the github pages site. This readme is a quick reference.
Requires all dependencies to be installed.
autoreconf -i
./configure
make
make install
Will install binaries and scripts for all users on typical environments
To skip language model building and use some of the scripts locally.
autoreconf -i
./configure
src/bash/omorfi-download.bash
This will download some of the pre-compiled dictionaries into your current working directory.
It is possible to install within python via pip or anaconda. The
dependencies that are not available in pip or anaconda will not be usable, e.g.
syntactic analysis and disambiguation using VISL CG 3.
pip install omorfi
conda install -c flammie omorfi
NB: since conda does not have new version of hfst buildable with recent pythons or something, only older versions are available on conda.
It is possible to use omorfi with a ready-made docker container, there is a
Dockerfile in src/docker/Dockerfile for that.
docker build -t "omorfi:Dockerfile" .
docker run -it "omorfi:Dockerfile" bash
Omorfi can be used from command line using following commands:
omorfi-disambiguate-text.sh: analyse and disambiguateomorfi-analyse-text.sh: analyseomorfi-spell.sh: spell-check and correctomorfi-segment.sh: morphologically segmentomorfi-conllu.bash: analyse in CONLL-U formatomorfi-freq-evals.bash: analyse coverage and statisticsomorfi-ftb3.bash: analyse in FTB-3 format (CONLL-X)omorfi-factorise.bash: analyse in Moses-SMT factorised formatomorfi-vislcg.bash: analyse in VISL CG 3 formatomorfi-analyse-tokenised.sh: analyse word per line (faster)omorfi-generate.sh: generate word-forms from omor descriptionsomorfi-download.bash: download language models from latest release
For further details please refer to:
Omorfi can be used via very simple programming APIs, the design is detailed in omorfi API design
There are various binaries for language models that can be used with specialised tools like HFST. For further details, see our usage examples.
For full descriptions and archived problems, see: Troubleshooting in github pages
Update HFST.
In order for python scripts to work you need to install them to same prefix as
python, or define PYTHONPATH, e.g. export PYTHONPATH=/usr/local/lib/python3.11/site-packages/
This can easily happen for legit reasons. It can be reduced by filtering overlong tokens out. Or processing texts in smaller pieces.
Get more RAM or swap space.
Omorfi code and data are free and libre open source, and community-driven, to participate, read further information in CONTRIBUTING
- Issues and problems may be filed in our github issue tracker, including support questions
- Matrix channel omorfi is particularly good for live chat for support questions, suggestions and discussions
- omorfi-devel mailing list is good for longer more involved discussions
You can always discuss in English or Finnish on any of the channels.
See our code of conduct.
A lot of omorfi development has been done on spare time and by volunteers, if you want to support Flammie you can use the github's ❤️Sponsor button, or any of the services below:
