diff --git a/docs/_config.yml b/docs/_config.yml index 5a60dd60..1cb23fb5 100644 --- a/docs/_config.yml +++ b/docs/_config.yml @@ -122,6 +122,8 @@ docs_nav: url: "/docs/models/gemma/" - label: MedGemma url: "/docs/models/medgemma/" + - label: TranslateGemma + url: "/docs/models/translategemma/" - label: gpt-oss url: "/docs/models/gpt-oss/" - label: LiquidAI/LFM diff --git a/docs/docs/models/gemma.md b/docs/docs/models/gemma.md index e2344db8..8846f443 100644 --- a/docs/docs/models/gemma.md +++ b/docs/docs/models/gemma.md @@ -47,29 +47,4 @@ flm run gemma3:4b /input "file/to/image.jpg" describe this image. ``` ---- - -## 🧩 Model Card: [gemma-3-270m-it](https://huggingface.co/google/gemma-3-270m-it) - -- **Type:** Image-Text-to-Text -- **Think:** No -- **Tool Calling Support:** No -- **Base Model:** [google/gemma-3-270m-it](https://huggingface.co/google/gemma-3-270m-it) -- **Quantization:** Q4_1 -- **Max Context Length:** 32k tokens -- **Default Context Length:** 32k tokens ([change default](https://fastflowlm.com/docs/instructions/cli/#-change-default-context-length-max)) -- **[Set Context Length at Launch](https://fastflowlm.com/docs/instructions/cli/#-set-context-length-at-launch)** - -▶️ Run with FastFlowLM in PowerShell: - -```shell -flm run gemma3:270m -``` - -⚠️ **Warning:** -> – `gemma3:270m` is **Experimental** in FLM -> – Limited accuracy; may produce errors -> – Can loop on long outputs (quirk from Unsloth weights, also seen in LM Studio) -> – Experimenting with different quantization + hyperparameters - --- \ No newline at end of file diff --git a/docs/docs/models/index.md b/docs/docs/models/index.md index d16c394d..7ccc0e98 100644 --- a/docs/docs/models/index.md +++ b/docs/docs/models/index.md @@ -20,6 +20,7 @@ has_children: true - [Qwen](qwen/) - [Gemma](gemma/) - [MedGemma](medgemma/) +- [TranslateGemma](translategemma/) - [gpt-oss](gpt-oss/) - [LiquidAI/LFM](lfm/) - [Phi](phi/) diff --git a/docs/docs/models/medgemma.md b/docs/docs/models/medgemma.md index 2593a042..2942ab6c 100644 --- a/docs/docs/models/medgemma.md +++ b/docs/docs/models/medgemma.md @@ -22,12 +22,6 @@ parent: Models flm run medgemma:4b ``` -🔐 Why It Matters - -- **Privacy First** — There is nothing more personal than your health! -- **Powered by NPU** — Leverages AMD Ryzen™ AI NPU for fast, low-power inference. -- **Healthcare Applications** — A concrete example of how local LLMs + NPUs enable privacy-preserving, research-driven healthcare workflows. - 📺 Demo Video - [MedGemma:4B (Multimodal) on AMD Ryzen™ AI NPU — Demo Video](https://www.youtube.com/watch?v=KWzXZEOcgK4&list=PLf87s9UUZrJoDdz639Yc6w1UTyJ4cFHZ1&index=5&ab_channel=FastFlowLM) @@ -39,4 +33,40 @@ flm run medgemma:4b This tool (**MedGemma + FastFlowLM**) is **not** a diagnostic or clinical tool. Always consult a licensed medical professional for healthcare decisions. +--- + +## 🧩 Model Card: [medgemma1.5:4b](https://huggingface.co/google/medgemma-1.5-4b-it) + +- **Type:** Image-Text-to-Text +- **Think:** No +- **Tool Calling Support:** No +- **Base Model:** [google/medgemma1.5-4b-it](https://huggingface.co/google/medgemma-1.5-4b-it) +- **Quantization:** Q4_1 +- **Max Context Length:** 128k tokens +- **Default Context Length:** 64k tokens ([change default](https://fastflowlm.com/docs/instructions/cli/#-change-default-context-length-max)) +- **[Set Context Length at Launch](https://fastflowlm.com/docs/instructions/cli/#-set-context-length-at-launch)** + +▶️ Run with FastFlowLM in PowerShell: + +```shell +flm run medgemma1.5:4b +``` + +⚠️ **Disclaimer** + +This tool (**MedGemma + FastFlowLM**) is **not** a diagnostic or clinical tool. +Always consult a licensed medical professional for healthcare decisions. + +📺 Demo Video + +- [MedGemma1.5:4B (Multimodal) on AMD Ryzen™ AI NPU — Demo Video]() +- [Reaseach Blog](https://research.google/blog/next-generation-medical-image-interpretation-with-medgemma-15-and-medical-speech-to-text-with-medasr/) + +--- + +🔐 Why They Matter + +- **Privacy First** — There is nothing more personal than your health! +- **Powered by NPU** — Leverages AMD Ryzen™ AI NPU for fast, low-power inference. +- **Healthcare Applications** — A concrete example of how local LLMs + NPUs enable privacy-preserving, research-driven healthcare workflows. --- \ No newline at end of file diff --git a/docs/docs/models/qwen.md b/docs/docs/models/qwen.md index 9f2afba3..d167bec8 100644 --- a/docs/docs/models/qwen.md +++ b/docs/docs/models/qwen.md @@ -156,6 +156,27 @@ flm run qwen3-it:4b flm run qwen3vl-it:4b ``` +▶️ Image Resize Options + +You can control image resizing when running or serving the model using the `--resize` flag: + +```shell +flm run qwen3vl-it:3b --resize 1 +``` + +```shell +flm serve qwen3vl-it:3b --resize 1 +``` + +The `--resize` option determines the target length of the image’s **longest** side: + +- -1 — Disable resizing (use the original image) +- 0 — Resize so the longest side is **1080 px** (Default) +- 1 — Resize so the longest side is **1920 px** +- 2 — Resize so the longest side is **2560 px** + +> Don't worry—if your image is already smaller than 1080px, it keeps its original resolution! ✨ + 📝 **Note** - Image understanding adapts to image size. Image TTFT can range from under 1 second to ~200 seconds depending on resolution. Use lower-resolution images (720p or below) unless high resolution is required (e.g. OCR on small text). @@ -199,6 +220,27 @@ flm run qwen2.5-it:3b flm run qwen2.5vl-it:3b ``` +▶️ Image Resize Options + +You can control image resizing when running or serving the model using the `--resize` flag: + +```shell +flm run qwen2.5vl-it:3b --resize 1 +``` + +```shell +flm serve qwen2.5vl-it:3b --resize 1 +``` + +The `--resize` option determines the target length of the image’s **longest** side: + +- -1 — Disable resizing (use the original image) +- 0 — Resize so the longest side is **1080 px** (Default) +- 1 — Resize so the longest side is **1920 px** +- 2 — Resize so the longest side is **2560 px** + +> Don't worry—if your image is already smaller than 1080px, it keeps its original resolution! ✨ + 📝 **Note** - Image understanding adapts to image size. Image TTFT can range from under 1 second to ~200 seconds depending on resolution. Use lower-resolution images (720p or below) unless high resolution is required (e.g. OCR on small text). diff --git a/docs/docs/models/translategemma.md b/docs/docs/models/translategemma.md new file mode 100644 index 00000000..f66d05f7 --- /dev/null +++ b/docs/docs/models/translategemma.md @@ -0,0 +1,45 @@ +--- +layout: docs +title: TranslateGemma +nav_order: 11 +parent: Models +--- + +## 🧩 Model Card: [translategemma:4b](https://huggingface.co/google/translategemma-4b-it) + +- **Type:** Image-Text-to-Text +- **Think:** No +- **Tool Calling Support:** No +- **Base Model:** [google/translategemma-4b-it](https://huggingface.co/google/translategemma-4b-it) +- **Quantization:** Q4_1 +- **Max Context Length:** 128k tokens +- **Default Context Length:** 64k tokens ([change default](https://fastflowlm.com/docs/instructions/cli/#-change-default-context-length-max)) +- **[Set Context Length at Launch](https://fastflowlm.com/docs/instructions/cli/#-set-context-length-at-launch)** + +▶️ Run with FastFlowLM in PowerShell: + +```shell +flm run translategemma:4b +``` + +📖 Prompt Guide + +**Prompt Format** + +TranslateGemma expects a single **user message** with this structure: + + You are a professional {SOURCE_LANG} ({SOURCE_CODE}) to {TARGET_LANG} ({TARGET_CODE}) translator. Your goal is to accurately convey the meaning and nuances of the original {SOURCE_LANG} text while adhering to {TARGET_LANG} grammar, vocabulary, and cultural sensitivities. + Produce only the {TARGET_LANG} translation, without any additional explanations or commentary. Please translate the following {SOURCE_LANG} text into {TARGET_LANG}: + + + {TEXT} + +**Important**: There are two blank lines before the text to translate. + +🦙Refer to the Ollama model library for detailed usage [examples](https://ollama.com/library/translategemma#:~:text=text%20to%20translate.-,Examples,-English%20to%20Spanish). + +📺 Demo Video + +- [Translategemma1.5:4B (Multimodal) on AMD Ryzen™ AI NPU — Demo Video]() +- [Reaseach Blog](https://blog.google/innovation-and-ai/technology/developers-tools/translategemma/) +- [Technical Report](https://arxiv.org/pdf/2601.09012) \ No newline at end of file