Leading newline after trans block with Jinja trim options results in translation lookup failure

[vaughnkoch]

Hi, thanks for this foundational library.

I'm using Jinja's trans tag and found that when the trans tag is immediately followed by a newline, and either trim_blocks or ext.i18n.trimmed are True, Jinja fails to use the translation string in the .mo file.

This occurs because in Jinja's ext.py, the string used for the msgid lookup has its leading newline stripped off. If you manually add the string back during the gettext call, gettext is able to find the correct msgid in the .mo file.

Instrumentation

Test template:

{% trans %}
Hi Jinja
{% endtrans %}

Here's a code block I added to ext.py for instrumentation, just modified the gettext wrapper:

def _make_new_gettext(func: t.Callable[[str], str]) -> t.Callable[..., str]:
    @pass_context
    def gettext(__context: Context, __string: str, **variables: t.Any) -> str:
        rv = __context.call(func, __string)
        # breakpoint()
        print (f'__context.environment.trim_blocks: {__context.environment.trim_blocks}')
        print (f'__context.environment.policies["ext.i18n.trimmed"]: {__context.environment.policies["ext.i18n.trimmed"]}')
        print ('__string (input) repr: ' + repr(__string))

        print ('Expected result: "Hola Jinja"')
        print ('gettext result repr: ' + repr(__context.call(func, __string)))
        print ('gettext result repr after adding leading newline: ' + repr(__context.call(func, '\n' + __string)))
        ...

Results

No trimming - works as expected

__context.environment.trim_blocks: False
__context.environment.policies["ext.i18n.trimmed"]: False
__string (input) repr: '\nHi Jinja\n'
Expected result: "Hola Jinja"
gettext result repr: '\nHola Jinja\n'
gettext result repr after adding leading newline: '\n\nHi Jinja\n'

Hola Jinja

trim_blocks: True - does not translate

__context.environment.trim_blocks: True
__context.environment.policies["ext.i18n.trimmed"]: False
__string (input) repr: 'Hi Jinja\n'
Expected result: "Hola Jinja"
gettext result repr: 'Hi Jinja\n'
gettext result repr after adding leading newline: '\nHola Jinja\n'
Hi Jinja

ext.i18n.trimmed: True - does not translate

__context.environment.trim_blocks: False
__context.environment.policies["ext.i18n.trimmed"]: True
__string (input) repr: 'Hi Jinja'
Expected result: "Hola Jinja"
gettext result repr: 'Hi Jinja'
gettext result repr after adding leading newline: '\nHi Jinja'
Hi Jinja

My versions

• jinja2==3.1.2
• django==4.2
• django-jinja==2.11.0 (implements python manage.py makemessages for Jinja2)

If I want to use either trim_blocks or ext.i18n.trimmed, the only thing that I've found to work is to remove the newline (and presumably traling whitespace as well), e.g.

{% trans %}Hi Jinja{% endtrans %}

That works fine for very short tags, but when you have multiple variables and the line is long (especially for a .txt template), the code is harder to read. Also, it's easy to forget to remove the leading newline, especially if you don't know this behavior/workaround.

[topspinj]

I'm working on this at the PyCon 2024 Pallets sprint. Let's see how far I get :)