Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions docs/_config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -122,6 +122,8 @@ docs_nav:
url: "/docs/models/gemma/"
- label: MedGemma
url: "/docs/models/medgemma/"
- label: TranslateGemma
url: "/docs/models/translategemma/"
- label: gpt-oss
url: "/docs/models/gpt-oss/"
- label: LiquidAI/LFM
Expand Down
25 changes: 0 additions & 25 deletions docs/docs/models/gemma.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,29 +47,4 @@ flm run gemma3:4b
/input "file/to/image.jpg" describe this image.
```

---

## 🧩 Model Card: [gemma-3-270m-it](https://huggingface.co/google/gemma-3-270m-it)

- **Type:** Image-Text-to-Text
- **Think:** No
- **Tool Calling Support:** No
- **Base Model:** [google/gemma-3-270m-it](https://huggingface.co/google/gemma-3-270m-it)
- **Quantization:** Q4_1
- **Max Context Length:** 32k tokens
- **Default Context Length:** 32k tokens ([change default](https://fastflowlm.com/docs/instructions/cli/#-change-default-context-length-max))
- **[Set Context Length at Launch](https://fastflowlm.com/docs/instructions/cli/#-set-context-length-at-launch)**

▶️ Run with FastFlowLM in PowerShell:

```shell
flm run gemma3:270m
```

⚠️ **Warning:**
> – `gemma3:270m` is **Experimental** in FLM
> – Limited accuracy; may produce errors
> – Can loop on long outputs (quirk from Unsloth weights, also seen in LM Studio)
> – Experimenting with different quantization + hyperparameters

---
1 change: 1 addition & 0 deletions docs/docs/models/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ has_children: true
- [Qwen](qwen/)
- [Gemma](gemma/)
- [MedGemma](medgemma/)
- [TranslateGemma](translategemma/)
- [gpt-oss](gpt-oss/)
- [LiquidAI/LFM](lfm/)
- [Phi](phi/)
Expand Down
42 changes: 36 additions & 6 deletions docs/docs/models/medgemma.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,12 +22,6 @@ parent: Models
flm run medgemma:4b
```

🔐 Why It Matters

- **Privacy First** — There is nothing more personal than your health!
- **Powered by NPU** — Leverages AMD Ryzen™ AI NPU for fast, low-power inference.
- **Healthcare Applications** — A concrete example of how local LLMs + NPUs enable privacy-preserving, research-driven healthcare workflows.

📺 Demo Video

- [MedGemma:4B (Multimodal) on AMD Ryzen™ AI NPU — Demo Video](https://www.youtube.com/watch?v=KWzXZEOcgK4&list=PLf87s9UUZrJoDdz639Yc6w1UTyJ4cFHZ1&index=5&ab_channel=FastFlowLM)
Expand All @@ -39,4 +33,40 @@ flm run medgemma:4b
This tool (**MedGemma + FastFlowLM**) is **not** a diagnostic or clinical tool.
Always consult a licensed medical professional for healthcare decisions.

---

## 🧩 Model Card: [medgemma1.5:4b](https://huggingface.co/google/medgemma-1.5-4b-it)

- **Type:** Image-Text-to-Text
- **Think:** No
- **Tool Calling Support:** No
- **Base Model:** [google/medgemma1.5-4b-it](https://huggingface.co/google/medgemma-1.5-4b-it)
- **Quantization:** Q4_1
- **Max Context Length:** 128k tokens
- **Default Context Length:** 64k tokens ([change default](https://fastflowlm.com/docs/instructions/cli/#-change-default-context-length-max))
- **[Set Context Length at Launch](https://fastflowlm.com/docs/instructions/cli/#-set-context-length-at-launch)**

▶️ Run with FastFlowLM in PowerShell:

```shell
flm run medgemma1.5:4b
```

⚠️ **Disclaimer**

This tool (**MedGemma + FastFlowLM**) is **not** a diagnostic or clinical tool.
Always consult a licensed medical professional for healthcare decisions.

📺 Demo Video

- [MedGemma1.5:4B (Multimodal) on AMD Ryzen™ AI NPU — Demo Video]()
- [Reaseach Blog](https://research.google/blog/next-generation-medical-image-interpretation-with-medgemma-15-and-medical-speech-to-text-with-medasr/)

---

🔐 Why They Matter

- **Privacy First** — There is nothing more personal than your health!
- **Powered by NPU** — Leverages AMD Ryzen™ AI NPU for fast, low-power inference.
- **Healthcare Applications** — A concrete example of how local LLMs + NPUs enable privacy-preserving, research-driven healthcare workflows.
---
42 changes: 42 additions & 0 deletions docs/docs/models/qwen.md
Original file line number Diff line number Diff line change
Expand Up @@ -156,6 +156,27 @@ flm run qwen3-it:4b
flm run qwen3vl-it:4b
```

▶️ Image Resize Options

You can control image resizing when running or serving the model using the `--resize` flag:

```shell
flm run qwen3vl-it:3b --resize 1
```

```shell
flm serve qwen3vl-it:3b --resize 1
```

The `--resize` option determines the target length of the image’s **longest** side:

- -1 — Disable resizing (use the original image)
- 0 — Resize so the longest side is **1080 px** (Default)
- 1 — Resize so the longest side is **1920 px**
- 2 — Resize so the longest side is **2560 px**

> Don't worry—if your image is already smaller than 1080px, it keeps its original resolution! ✨

📝 **Note**

- Image understanding adapts to image size. Image TTFT can range from under 1 second to ~200 seconds depending on resolution. Use lower-resolution images (720p or below) unless high resolution is required (e.g. OCR on small text).
Expand Down Expand Up @@ -199,6 +220,27 @@ flm run qwen2.5-it:3b
flm run qwen2.5vl-it:3b
```

▶️ Image Resize Options

You can control image resizing when running or serving the model using the `--resize` flag:

```shell
flm run qwen2.5vl-it:3b --resize 1
```

```shell
flm serve qwen2.5vl-it:3b --resize 1
```

The `--resize` option determines the target length of the image’s **longest** side:

- -1 — Disable resizing (use the original image)
- 0 — Resize so the longest side is **1080 px** (Default)
- 1 — Resize so the longest side is **1920 px**
- 2 — Resize so the longest side is **2560 px**

> Don't worry—if your image is already smaller than 1080px, it keeps its original resolution! ✨

📝 **Note**

- Image understanding adapts to image size. Image TTFT can range from under 1 second to ~200 seconds depending on resolution. Use lower-resolution images (720p or below) unless high resolution is required (e.g. OCR on small text).
Expand Down
45 changes: 45 additions & 0 deletions docs/docs/models/translategemma.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
---
layout: docs
title: TranslateGemma
nav_order: 11
parent: Models
---

## 🧩 Model Card: [translategemma:4b](https://huggingface.co/google/translategemma-4b-it)

- **Type:** Image-Text-to-Text
- **Think:** No
- **Tool Calling Support:** No
- **Base Model:** [google/translategemma-4b-it](https://huggingface.co/google/translategemma-4b-it)
- **Quantization:** Q4_1
- **Max Context Length:** 128k tokens
- **Default Context Length:** 64k tokens ([change default](https://fastflowlm.com/docs/instructions/cli/#-change-default-context-length-max))
- **[Set Context Length at Launch](https://fastflowlm.com/docs/instructions/cli/#-set-context-length-at-launch)**

▶️ Run with FastFlowLM in PowerShell:

```shell
flm run translategemma:4b
```

📖 Prompt Guide

**Prompt Format**

TranslateGemma expects a single **user message** with this structure:

You are a professional {SOURCE_LANG} ({SOURCE_CODE}) to {TARGET_LANG} ({TARGET_CODE}) translator. Your goal is to accurately convey the meaning and nuances of the original {SOURCE_LANG} text while adhering to {TARGET_LANG} grammar, vocabulary, and cultural sensitivities.
Produce only the {TARGET_LANG} translation, without any additional explanations or commentary. Please translate the following {SOURCE_LANG} text into {TARGET_LANG}:


{TEXT}

**Important**: There are two blank lines before the text to translate.

🦙Refer to the Ollama model library for detailed usage [examples](https://ollama.com/library/translategemma#:~:text=text%20to%20translate.-,Examples,-English%20to%20Spanish).

📺 Demo Video

- [Translategemma1.5:4B (Multimodal) on AMD Ryzen™ AI NPU — Demo Video]()
- [Reaseach Blog](https://blog.google/innovation-and-ai/technology/developers-tools/translategemma/)
- [Technical Report](https://arxiv.org/pdf/2601.09012)