Skip to content

Commit 158b8e7

Browse files
niklubnik
niklub
and
nik
authored
fix: LEAP-938: fix NeMo ASR ML backend (#489)
* fix: LEAP-938: fix NeMo ASR ML backend * Remove 'nemo' directory * Downgrade python version in docker * Modify requirements and packages --------- Co-authored-by: nik <[email protected]>
1 parent bf8a785 commit 158b8e7

18 files changed

+286
-259
lines changed

‎README.md

Lines changed: 16 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -27,22 +27,22 @@ The following models are supported in the repository. Some of them working witho
2727
require additional parameters to be set.
2828
Please check **Required parameters** column to see if you need to set any additional parameters.
2929

30-
| MODEL_NAME | Description | Required parameters |
31-
|----------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------|---------------------|
32-
| [segment_anything_model](/label_studio_ml/examples/segment_anything_model) | General-purpose interactive image segmentation [from Meta](https://segment-anything.com/) | None |
33-
| [llm_interactive](/label_studio_ml/examples/llm_interactive) | Prompt engineering, data collection and model evaluation workflows for LLM ([OpenAI](https://platform.openai.com/), Azure) | OPENAI_API_KEY |
34-
| [grounding_dino](/label_studio_ml/examples/grounding_dino) | Object detection with text prompts ([details](https://github.com/IDEA-Research/GroundingDINO)) | None |
35-
| [tesseract](/label_studio_ml/examples/tesseract) | Optical Character Recognition (OCR) by drawing bounding boxes ([details](https://github.com/tesseract-ocr/tesseract)) | None |
36-
| [easyocr](/label_studio_ml/examples/easyocr) | Another OCR tool from [EasyOCR](https://github.com/JaidedAI/EasyOCR) | None |
37-
| [spacy](/label_studio_ml/examples/spacy) | Named entity recognition model from [SpaCy](https://spacy.io/) | None |
38-
| [flair](/label_studio_ml/examples/flair) | NLP models by [flair](https://flairnlp.github.io/) | None |
39-
| [bert_classifier](/label_studio_ml/examples/bert_classifier) | Text classification models from [Huggingface](https://huggingface.co/transformers/v3.0.2/model_doc/auto.html#automodelforsequenceclassification) | None |
40-
| [huggingface_llm](/label_studio_ml/examples/huggingface_llm) | Text generation models by [Hugging Face](https://huggingface.co/tasks/text-generation) | None |
41-
| [huggingface_ner](/label_studio_ml/examples/huggingface_ner) | NER models (TokenClassification) by [Hugging Face](https://huggingface.co/docs/transformers/en/tasks/token_classification) | None |
42-
| [nemo](/label_studio_ml/examples/nemo) | Speech transcription models by [NVIDIA NeMo](https://github.com/NVIDIA/NeMo) | None |
43-
| [mmetection](/label_studio_ml/examples/mmetection) | Object detection models by [OpenMMLab](https://github.com/open-mmlab/mmdetection) | None |
44-
| [sklearn_text_classifier](/label_studio_ml/examples/sklearn_text_classifier) | Simple trainable text classification model powered by [scikit-learn](https://scikit-learn.org/stable/) | None |
45-
| [interactive_substring_matching](/label_studio_ml/examples/interactive_substring_matching) | Interactively select keywords to highlight all occurrences of the keyword in the text | None |
30+
| MODEL_NAME | Description | Required parameters |
31+
|--------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------|---------------------|
32+
| [segment_anything_model](/label_studio_ml/examples/segment_anything_model) | General-purpose interactive image segmentation [from Meta](https://segment-anything.com/) | None |
33+
| [llm_interactive](/label_studio_ml/examples/llm_interactive) | Prompt engineering, data collection and model evaluation workflows for LLM ([OpenAI](https://platform.openai.com/), Azure) | OPENAI_API_KEY |
34+
| [grounding_dino](/label_studio_ml/examples/grounding_dino) | Object detection with text prompts ([details](https://github.com/IDEA-Research/GroundingDINO)) | None |
35+
| [tesseract](/label_studio_ml/examples/tesseract) | Optical Character Recognition (OCR) by drawing bounding boxes ([details](https://github.com/tesseract-ocr/tesseract)) | None |
36+
| [easyocr](/label_studio_ml/examples/easyocr) | Another OCR tool from [EasyOCR](https://github.com/JaidedAI/EasyOCR) | None |
37+
| [spacy](/label_studio_ml/examples/spacy) | Named entity recognition model from [SpaCy](https://spacy.io/) | None |
38+
| [flair](/label_studio_ml/examples/flair) | NLP models by [flair](https://flairnlp.github.io/) | None |
39+
| [bert_classifier](/label_studio_ml/examples/bert_classifier) | Text classification models from [Huggingface](https://huggingface.co/transformers/v3.0.2/model_doc/auto.html#automodelforsequenceclassification) | None |
40+
| [huggingface_llm](/label_studio_ml/examples/huggingface_llm) | Text generation models by [Hugging Face](https://huggingface.co/tasks/text-generation) | None |
41+
| [huggingface_ner](/label_studio_ml/examples/huggingface_ner) | NER models (TokenClassification) by [Hugging Face](https://huggingface.co/docs/transformers/en/tasks/token_classification) | None |
42+
| [nemo_asr](/label_studio_ml/examples/nemo_asr) | Speech transcription models by [NVIDIA NeMo](https://github.com/NVIDIA/NeMo) | None |
43+
| [mmetection](/label_studio_ml/examples/mmetection) | Object detection models by [OpenMMLab](https://github.com/open-mmlab/mmdetection) | None |
44+
| [sklearn_text_classifier](/label_studio_ml/examples/sklearn_text_classifier) | Simple trainable text classification model powered by [scikit-learn](https://scikit-learn.org/stable/) | None |
45+
| [interactive_substring_matching](/label_studio_ml/examples/interactive_substring_matching) | Interactively select keywords to highlight all occurrences of the keyword in the text | None |
4646

4747
# (Advanced usage) Develop your model
4848

‎label_studio_ml/examples/nemo/.dockerignore

Lines changed: 0 additions & 11 deletions
This file was deleted.

‎label_studio_ml/examples/nemo/Dockerfile

Lines changed: 0 additions & 24 deletions
This file was deleted.

‎label_studio_ml/examples/nemo/README.md

Lines changed: 0 additions & 32 deletions
This file was deleted.

‎label_studio_ml/examples/nemo/asr.py

Lines changed: 0 additions & 66 deletions
This file was deleted.

‎label_studio_ml/examples/nemo/docker-compose.yml

Lines changed: 0 additions & 28 deletions
This file was deleted.

‎label_studio_ml/examples/nemo/requirements.txt

Lines changed: 0 additions & 8 deletions
This file was deleted.

‎label_studio_ml/examples/nemo/supervisord.conf

Lines changed: 0 additions & 40 deletions
This file was deleted.

‎label_studio_ml/examples/nemo/uwsgi.ini

Lines changed: 0 additions & 10 deletions
This file was deleted.

‎label_studio_ml/examples/nemo_asr/Dockerfile

Lines changed: 50 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,50 @@
1+
# syntax=docker/dockerfile:1
2+
ARG PYTHON_VERSION=3.11
3+
4+
FROM python:${PYTHON_VERSION}-slim AS python-base
5+
ARG TEST_ENV
6+
7+
WORKDIR /app
8+
9+
ENV PYTHONUNBUFFERED=1 \
10+
PYTHONDONTWRITEBYTECODE=1 \
11+
PORT=${PORT:-9090} \
12+
PIP_CACHE_DIR=/.cache \
13+
WORKERS=1 \
14+
THREADS=8
15+
16+
# Update the base OS
17+
RUN --mount=type=cache,target="/var/cache/apt",sharing=locked \
18+
--mount=type=cache,target="/var/lib/apt/lists",sharing=locked \
19+
set -eux; \
20+
apt-get update; \
21+
apt-get upgrade -y; \
22+
apt install --no-install-recommends -y \
23+
git g++ freeglut3-dev build-essential libx11-dev \
24+
ffmpeg libsm6 libxext6 libffi-dev python3-dev python3-pip gcc; \
25+
apt-get autoremove -y
26+
27+
# install base requirements
28+
COPY requirements-base.txt .
29+
RUN --mount=type=cache,target=${PIP_CACHE_DIR},sharing=locked \
30+
pip install -r requirements-base.txt
31+
32+
# install custom requirements
33+
RUN pip install Cython
34+
COPY requirements.txt .
35+
RUN --mount=type=cache,target=${PIP_CACHE_DIR},sharing=locked \
36+
pip install -r requirements.txt
37+
38+
# install test requirements if needed
39+
COPY requirements-test.txt .
40+
# build only when TEST_ENV="true"
41+
RUN --mount=type=cache,target=${PIP_CACHE_DIR},sharing=locked \
42+
if [ "$TEST_ENV" = "true" ]; then \
43+
pip install -r requirements-test.txt; \
44+
fi
45+
46+
COPY . .
47+
48+
EXPOSE 9090
49+
50+
CMD gunicorn --preload --bind :$PORT --workers $WORKERS --threads $THREADS --timeout 0 _wsgi:app

0 commit comments

Comments
 (0)