TranslateGemma stops the inference at a random point

[ssechi2]

Gemma3 seems working fine, but TranslateGemma is not, I can't compare it with Gemma4, the converter gives an error:
File "C:\Python311\model\convert.py", line 6712, in dump_config assert txt_config.use_bidirectional_attention == 'vision'
TranslateGemma 4b/12b@Q8/FP16 (any backend) stops inference at some point, the problem always appears on the prompt: "List the programming languages whose code you can edit and what exactly you can do with the code of these languages"
chatllm version: 0.22+
CLI: main.exe -c 4096 -i -ngl all -m C:\Models\gemma-4-E2B-it-q8_0.bin

You are served by translategemma-4b-it, 
with 3880263168 (3.9B) parameters.

You  > List the programming languages whose code you can edit and what exactly you can do with the code of these languages.
A.I. > Okay, here's a list of programming languages I can work with, along with a description of what I can do with their code:

**Languages I Can Work With:**

*   **Python:**
    *   **What I can do:**
        *   **Code Generation:** I can generate Python code snippets, functions, classes, and even entire programs based on your instructions.
        *   **Code Completion:** I can suggest code completions as you type, helping you write code faster and with fewer errors.
        *   **Code Explanation:** I can explain what a piece of Python code does, line by line or at a higher level.
        *   **Code Translation:** I can translate code from other languages (like JavaScript or C++) to Python, or vice versa (though the accuracy depends on the complexity).
        *   **Code Debugging:** I can help identify potential errors and suggest fixes in Python code.
        *   **Code Optimization:** I can suggest ways to improve the performance and efficiency of Python code.
        *   **Code Refactoring:** I can help you refactor your code to make it more readable, maintainable, and efficient.
        *   **Code Documentation:** I can generate documentation for your Python code.
        *   **Code Testing:** I can help you write unit tests for your Python code.
        *   **Code Style Enforcement:** I can check your code for adherence to Python style guides (like PEP 8).

*   **JavaScript:**
    *   **What I can do:**
        *   **Code Generation:** I can generate JavaScript code snippets, functions, classes, and entire programs.
        *   **Code Completion:** I can suggest code completions.
        *   **Code Explanation:** I can explain what a piece of JavaScript code does.
        *   **Code Translation:** I can translate code from other languages (like Python or C++) to JavaScript, or vice versa.
        *   **Code Debugging:** I can help identify potential errors.
        *   **Code Optimization:** I can suggest ways to improve performance.
        *   **Code Refactoring:** I can help with refactoring.
        *   **Code Documentation:** I can generate documentation.
        *   **Code Testing:** I can help with writing tests.
        *   **Code Style Enforcement:** I can check for adherence to style guides (like Airbnb or Google).

*

i.e. the inference stops at about 30%

[foldl]

I can't reproduce this.

Please note that TranslateGemma is for translation, I don't think it can be used for general Q&A.

As per the assertion assert txt_config.use_bidirectional_attention == 'vision', which model (huggingface URL) are you converting?

main.exe --max-length 16000 -ngl all -i -m quantized\gemma-4-e2b-it.bin
    ________          __  __    __    __  ___ 
   / ____/ /_  ____ _/ /_/ /   / /   /  |/  /_________  ____  
  / /   / __ \/ __ `/ __/ /   / /   / /|_/ // ___/ __ \/ __ \ 
 / /___/ / / / /_/ / /_/ /___/ /___/ /  / // /__/ /_/ / /_/ / 
 \____/_/ /_/\__,_/\__/_____/_____/_/  /_(_)___/ .___/ .___/  
You are served by Gemma-4,                    /_/   /_/       
with 5131890659 (5.1B) parameters.

You  > List the programming languages whose code you can edit and what exactly you can do with the code of these languages.
A.I. > As a large language model, I don't "edit" code in the same way a human programmer does (i.e., I don't run an IDE, compile, or deploy software). However, I have been trained on massive amounts of code from many programming languages. This allows me to perform sophisticated **text-based operations** on code.

Here is a breakdown of the programming languages I can effectively process, and what I can do with their code:

---

## Programming Languages I Can Process Well

I am proficient in understanding, generating, debugging, and refactoring code in the following major categories:

### 1. Python
* **What I can do:**
    * **Generate Code:** Write scripts, functions, classes, and entire applications based on a description of the desired functionality.
    * **Explain Code:** Break down complex Python logic, explaining what specific functions or libraries do.
    * **Debug:** Identify syntax errors, logical flaws, and runtime exceptions in existing Python code.
    * **Refactor/Optimize:** Suggest ways to make Python code more efficient, idiomatic (Pythonic), and readable.
    * **Library Usage:** Help implement tasks using popular libraries like NumPy, Pandas, Django, or Flask.

### 2. JavaScript (and TypeScript)
* **What I can do:**
    * **Generate Code:** Write front-end logic (React, Vue, plain JS), back-end APIs (Node.js/Express), or serverless functions.
    * **Debug:** Find errors in asynchronous operations, scope issues, and DOM manipulation problems.
    * **Translate:** Convert code snippets from JavaScript to TypeScript, or vice-versa.
    * **Explain Frameworks:** Describe how specific patterns work within React components or Node.js middleware.

### 3. Java
* **What I can do:**
    * **Generate Code:** Create object-oriented structures, implement algorithms, and design classes.
    * **Explain OOP Concepts:** Detail inheritance, polymorphism, interfaces, and design patterns (e.g., Singleton, Factory).
    * **Review for Best Practices:** Check Java code against standard conventions and suggest improvements for performance or maintainability.
    * **Handle Data Structures:** Implement solutions using Java collections (Lists, Maps, Trees).

### 4. C++
* **What I can do:**
    * **Algorithm Implementation:** Write efficient solutions for complex computational problems.
    * **Memory Management Analysis:** Discuss potential memory leaks or inefficient use of pointers and smart pointers.
    * **Template Usage:** Help write and understand complex template metaprogramming.
    * **Performance Tuning:** Suggest ways to optimize loops and data structures for speed.

### 5. C#
* **What I can do:**
    * **Generate Code:** Create solutions for Windows applications, web APIs (ASP.NET Core), or game development (Unity).
    * **LINQ Mastery:** Help write complex queries using Language Integrated Query (LINQ).
    * **Asynchronous Programming:** Assist in structuring `async/await` patterns correctly.

### 6. Go (Golang)
* **What I can do:**
    * **Concurrency Modeling:** Help structure goroutines and channels for efficient parallel processing.
    * **API Development:** Generate clean, idiomatic Go code for building microservices.
    * **Error Handling:** Review and improve Go's specific error handling patterns.

### 7. SQL (Structured Query Language)
* **What I can do:**
    * **Query Generation:** Write complex `SELECT`, `JOIN`, `GROUP BY`, and window functions.
    * **Schema Design:** Suggest optimal database schemas (tables, relationships) based on the required data model.
    * **Optimization:** Analyze slow queries and suggest indexing strategies to improve performance.

### 8. HTML/CSS
* **What I can do:**
    * **Markup Generation:** Create semantic HTML structures.
    * **Styling Implementation:** Write CSS for layout (Flexbox, Grid), responsive design, and visual effects.
    * **Accessibility Review:** Check code for basic accessibility compliance.

---

## What I **Cannot** Do (Limitations)

It is crucial to understand that I am a **text-based AI**, not an execution environment. Therefore, I **cannot** perform the following actions directly:

1. **Execute or Run Code:** I cannot compile, run, test, or debug code in a live environment. If you give me code, I can tell you *what* the error *should* be, but I cannot confirm if it *actually* runs correctly on your machine.
2. **Interact with External Systems:** I cannot connect to databases, deploy applications to servers, or interact with APIs unless you explicitly provide me with the necessary connection details and library context.
3. **Visual Design/UI Testing:** While I can write the *code* for a UI, I cannot visually render the result or test the user experience (UX) in a browser.
4. **Handle Massive, Context-Heavy Projects:** For extremely large codebases (millions of lines), I might lose context across the entire project unless you feed me specific, relevant files or modules.

**In summary, think of me as an extremely knowledgeable, tireless pair programmer, technical documentation expert, and code reviewer.** I excel at the cognitive tasks of understanding, generating, and refining the *text* of the code.

[ssechi2]

Please note that TranslateGemma is for translation, I don't think it can be used for general Q&A.
I probably exaggerated with 100%, now I converted 4b model and checked it again: TG 4b Q8 returned the answer in full 1 out of 5 times.

As per the assertion assert txt_config.use_bidirectional_attention == 'vision', which model (huggingface URL) are you converting?
https://huggingface.co/google/gemma-4-E2B-it/tree/main
Other models that I use (Gemma 3, Translate Gemma, Qwen-ASR, Qwen3.5) convert fine.

[foldl]

the assertion is fixed by 75e1485.

[ssechi2]

the assertion is fixed
Thanks, the conversion was successful, and the model seems to be working reliably. However, during the conversion, there were numerous warnings like these:

Warning: tensor model.layers.23.self_attn.k_norm.weight is ignored
Warning: tensor model.layers.23.self_attn.k_proj.weight is ignored
Warning: tensor model.layers.23.self_attn.v_proj.weight is ignored
Warning: tensor model.layers.24.self_attn.k_norm.weight is ignored
Warning: tensor model.layers.24.self_attn.k_proj.weight is ignored
Warning: tensor model.layers.24.self_attn.v_proj.weight is ignored

Is this how it should be?

[foldl]

This is okay. These tensors are not used.