Feature Request: Hebrew Glyphs in Cascadia Code

[schorrm]

Summary of the new feature/enhancement

Basically, include Hebrew glyphs in Cascadia.

In general, in terminal contexts, using Hebrew is a total nightmare. Very, very few Monospace fonts actually work with Hebrew, and using font fallback isn't a good solution (look at what it does).

Having Hebrew glyphs supported in a modern terminal font would just make the experience work well out of the box.

Proposed technical implementation details (optional)

I understand that Arabic and Farsi is very complicated. But Hebrew really isn't. The entire Unicode block is 88 points. Hebrew doesn't have ligatures or any complex font behavior, at least for letters. And even diacritics could be omitted and it'd still be super duper useful and cover 99%+ of use cases (Hebrew is almost never written with vocalization). So just covering the Hebrew alphabet would only require 27 glyphs.
And even if they're super basic and ugly it would still be really useful, just not having the layout break and stuff. Even just copying the glyphs for Hebrew from Courier New or Liberation Mono and resizing them for Cascadia would be a major help.

[zadjii-msft]

This seems like something that'd be good for the Cascadia Code repo, when that's available cough @cinnamon-msft cough

[egmontkob]

I can see two differences between the two screenshots.

One is that on the right hand size, the letters are somewhat bolder. I find this variant more pleasing to my eyes and more similar to the English text, although I cannot read Hebrew, so it's only a personal preference.

The other is that the words are in reverse order in one of them. I'm wondering: Which one is the correct one? (But first: Are the letters of each word in the correct order?) Were the two screenshots produced by the same command (if so, what)? How can a font "fix" or "break" the order of words (but not the order of letters within a word), and why would it be the font's responsibility to fix it?

Having Hebrew glyphs supported in a modern terminal font would just make the experience work well out of the box. [...] I understand that Arabic and Farsi is very complicated. But Hebrew really isn't.

These claims are not true. Making sure that each character cell contains the exact letter it needs to contain (i.e. words are not written backwards) is extremely hard when it comes to BiDi in terminals. Up until about half a year ago there wasn't even a remotely useful specification for the desired behavior. See #538, and my work on a specification linked from there.

It's unclear to me which one of these two issues (different look of glyphs vs. order of words) you meant to report here, could you please clarify? [Disclaimer: I'm not a Microsoft Terminal developer, maybe they fully understand what you meant.]

[schorrm]

Which one is the correct one?

The one on the right (see at top).

Were the two screenshots produced by the same command (if so, what)?

Yes. I echoed the label I wanted on top and cated the sample text file.

How can a font "fix" or "break" the order of words (but not the order of letters within a word), and why would it be the font's responsibility to fix it?

Well, the logical order is not at play, nor should it be changed. Nobody wants pipes to behave differently. This is an example of rendering issues. My best guess is that the ordering gets fudged because of the run analysis. The DxWriters analyze the sequences. As far as I can tell, lets say, spaces or punctuation should not break up a run. However, this seems to take a back seat to the preference of using its primary font. So these will be separated by the spaces, or the punctuation marks. And that'll incur changing a bidi level. We want to minimize the number of times bidi is switched for a layout thing. Completely revamping the layout engine so that font fallback will be seamless seems really really hard. Adding a few glyphs so it'll be averted in the first place seems an awful lot easier. And in general, having out of the box support on the font level for Hebrew and not having some crazy hodgepodge would be lovely.
So let me be clear: this is not reporting a bug. As far as I can tell, the layout engine is behaving the way it ought to by specs. This is about a request for the upcoming Cascadia Code font.

[egmontkob]

Well, the logical order is not at play, nor should it be changed. Nobody wants pipes to behave differently. This is an example of rendering issues.

Agreed.

As far as I can tell, lets say, spaces or punctuation should not break up a run.

Yup.

However, this seems to take a back seat to the preference of using its primary font. So these will be separated by the spaces, or the punctuation marks. And that'll incur changing a bidi level.

I truly don't understand it. How does a missing glyph (and thus having to take a fallback glyph from another font) influence the BiDi algorithm, how does it break up the run and change the BiDi level? It really shouldn't. I have no idea what a DxWriter is, but reshuffling should go according to the Unicode Bidirectional Algorithm which cares about the codepoints, but not at all about font lookup.

As far as I can tell, the layout engine is behaving the way it ought to by specs.

What specs you mean? The layout engine's specs? If the visual order of the words depends on the choice of the font then there's a giant bug. If this is what the layout engine's spec says to do then the layout engine's spec is terribly useless.

This is about a request for the upcoming Cascadia Code font.

Thanks for your clarification! I accept this, and thus don't want to hijack this bugreport for something else.

[DHowett-MSFT]

@miniksa thinks we can improve the font fallback configuration to choose a better font if the selected font needs falling-back, so I'm going to tag this one for rendering and we'll file a followup on the Cascadia Code repository since we agree that it's a good idea.

[zadjii-msft]

Hey, now that there actually is a Cascadia Code repo, I'm going to close this issue in favor of the upstream one at microsoft/cascadia-code#65