Proper display of text on the surface of a node(box)

[VanessaE]

Sapier requested that I put this into a feature request so as not to get lost.

As it stands, in order to display text on the surface of a node or nodebox, it is necessary to convert the text into an image and use an entity to display it, as this is the only method the API seems to allow for creating/updating an image in realtime on the client and no method to actually put text into the world exists yet aside from Waypoints and other HUD elements.

As many people know, the entity-based method started from jin_xi's painting mod, the method of which was then used in PilzAdam's/xyz's signs mod and now signs_lib. It is my and others' opinion that signs_lib implements the most complete possible way of handling these signs under the engine's present capabilities; see here for details: https://forum.minetest.net/viewtopic.php?pid=26061

Particularly, the signs are now capable of this:

Doing it the way we do now is of course hacky and uses a lot of texture memory (about 90 kB per sign). It works well for PCs but not so much so for tablets: a typical spawn might have 200 or so signs around it, leading to round about 18 MB of texture memory used just for these signs, in addition to hundreds, possibly a few thousand bytes per sign for the compositing string that actually constructs the text on the client. This, in turn, costs FPS, drastically so on some systems. Of course the proper way to do this would be to use the display/rendering engine to actually place text onto the node/nodebox's surface instead of using an entity.

Regarding the signs API, a sign needs to do these things:

• Exactly what the shape and textures of the node or nodebox are on which the text will be placed. At present, this is accomplished by just defining a node in the usual manner and passing that node definition to the library when registering a new sign.
• Where within that node's space to put the text. Both this and the sign's shape are entirely free-form in signs_lib.
• Which way the text faces relative to the underlying node's orientation.
• Which faces have what text. signs_lib can put text on multiple places and orientations on a single sign, with each place having different content and different facing direction -- think of street intersection signposts and their in-game street_signs versions, where one line of content on the sign faces e.g. east and west, and the second line faces north and south.
• Line breaks. Right now it works like in a text editor, that is, Enter adds a line break like normal, including inserting empty lines if needed.
• Font size and how much text fits on a sign. Currently, a sign can have any font size, and holds a variable amount of text between that and its proportional font, and there are two resolutions (so that large signs can have less-grainy text). Any number of lines is allowed, within the limits of the sign's configured font size.
• Colored text. Currently, we use #x where x is a single hex digit from the standard CGA/Linux/IRC color palette to change the text color. That palette is shown numerically in the bottom row of the sign in the image above. Thus, #9foo would for example display "foo" in light blue. Any number of color changes may be used on a sign. Ideally this should be expanded to allow for any #RRGGBB value. That's impractical in signs_lib, but would probably be easy if done in the engine.
• Text is currently centered left-to-right per line (though sometimes this doesn't happen and the text ends up left-justified), and displayed starting from the top left pixel in the sign clip region's bounding box (plus or minus any configured offsets).

[webD97]

This is something Minetest definetly needs!

[neoascetic]

Absolutely +1 on this, and yes, this is not low priority!

[C1ffisme]

Yes, yes, yes and yes.

And if this gets added, signs need to be made to use this IMMEDIATELY. I mean, what if your making a highway and people can't reach the signs to read while driving?

[neoascetic]

[C1ffisme]

Man, you guys just throw out bounty's like you own millions (rephrase : billions) of dollars.

By the way, do we want the text to be 3d, or 2d? Just wandering.

And, does it have to be just text? I mean, what about itemframes?

[sofar]

This should be done entirely in core, client side. The client already knows how to render monospaced fonts and has a font engine, so what needs to be done is modify the SIGNLIKE drawtype to render the font on the proper plane (I could care less right now if it handled color, but once a concept is working, that should be doable as well).

We don't want every map update to cause every sign to be re-font-drawn, so most likely there needs to be a cache of rendered sign text, such that content_mapblock.cpp can just use the cache.

This doesn't sound particularly hard to implement, if I may say so myself, and should be nice and performant and obsolete signs_lib entirely, thus removing a major piece of multiplayer lag problems.

[benrob0329]

Any progress on this?

[sofar]

I have an implementation concept idea:

facetext texture modifier

When specifying tiles for nodes, one can apply a transform to the texture. This texture transform is passed to the client, and so the client builds the texture as needed when the mesh updates.

We apply a generic "placeholder" text to a tile because we want to assure that the face we need to have it applies to is the only one affected.

tiles = { "top.png", "bottom.png", "side.png^[facetext", "side.png" }

Then, in the node metadata, we store the text content and attributes:

facetext = "<fontname>:<size>:<startcolor>:<text>",

• fontname: I expect to add several fonts. There are many good 8px monospaced fonts that are ready to be used. There should be a default, but we could add more free fonts later.
• size: a font scalar. 1 means 1 character per node. 2 means 4 characters per node, 3 means 9 characters per node. 16 means 256 characters per node.
• startcolor: rrggbbaa value for the characters
• text: 7bit ascii with color escapes

Note: only one facetext can be added per node.

Note: we should only ever use monospaced fonts.

content_mapblock.cpp would then create the texture for the tile as needed and put it on the appropriate surface.

If no nodemeta facetext is present, the base image is displayed. This removes the need for background color stuff.

This method will work for any nodebox. For a margin, the lua code should insert spaces to pad and move the characters to the node surface location. The z-fighting problems will be eliminated. The entities will be gone.

[numberZero]

@sofar
Sounds good. It would be very nice if implemented that way, although I have some questions:

• Why only one facetext per node? The modifier could reference metadata entry name, thus allowing different text to be shown at different sides. Not sure does that worth it, though, as thin signs on node faces could be used instead.
• Will a subgame or mod be able to add its own fonts?
• Usually characters are non-square. Is it possible to use different aspect ratio? Or even add fonts with different ratios, say, 1:1 and 3:4.
• What if someone sets color for each and every character in the text? (That might be used to create a raster display in-game)
• How many colors you plan to add? It might use old-style #e color escapes, or something longer, like #rgb or even #{rrggbb}
• While I understand that monospaced fonts are much easier to work with, currently signs support variable-width fonts; these look better (I even use non-monospaced font in my IDE, that’s more comfortable). There should be a way to use a variable-width font, or the entities won’t gone IMO.
• Are there any plans to extend character range from ASCII? E.g. even German needs ß, ö, etc., and Russian needs немного больше ;) (I’m only asking, that need not to be added in first versions, the support might be added later not breaking compatibility. It just should never crash on non-ASCII characters =)

[sofar]

* Why only one facetext per node? The modifier could reference metadata entry name, thus allowing different text to be shown at different sides. Not sure does that worth it, though, as thin signs on node faces could be used instead.

I think it will be easy to have the same text on several faces, but allowing different text or face would be very complex quickly.

* Will a subgame or mod be able to add its own fonts?

I don't know. Probably not, because it would require a new asset to be sent to each client, and that doesn't seem worth it if you ask me.

* Usually characters are non-square. Is it possible to use different aspect ratio? Or even add fonts with different ratios, say, 1:1 and 3:4.

You know, seriously, just... Heck no. We don't have any good way of doing node text. If we make it complex, it will never happen.

* What if someone sets color for each and every character in the text? (That might be used to create a raster display in-game)

That's fine, I think we can do that, and the idea already was to allow color escapes since chat supports them.

* How many colors you plan to add? It might use old-style `#e` color escapes, or something longer, like `#rgb` or even `#{rrggbb}`

We should solve that in the color chat escape sequence code. Don't see why it couldn't support arbitrary colors. Maybe it already does.

* While I understand that monospaced fonts are much easier to work with, currently signs support variable-width fonts; these look better (I even use non-monospaced font in my IDE, that’s more comfortable). There should be a way to use a variable-width font, or the entities won’t gone IMO.

You know, I completely disagree. Nodes are square. Do you want rectangular, proportional nodes? This feature should absolutely not exist to add perfectly outlayed text with decorations. It's there to make signs simple and efficient. Very simple. Very efficient. So hell no.

* Are there any plans to extend character range from ASCII? E.g. even German needs ß, ö, etc., and Russian needs немного больше ;) (I’m only asking, that need not to be added in first versions, the support might be added later not breaking compatibility. It just should never crash on non-ASCII characters =)

That's a font issue, but again, I just don't care enough to even want to spend time on that. We don't even have the basics (ASCII) yet and you shouldn't run before you can walk. Crashing would be a no-no of course. If you want this feature to succeed, and pave the way for future extensions that make it do some of the things you asked for, then don't kill it with unreasonable expectations. I'm not mad at you for asking, someone was going to anyway. I'm just going to be clear about what I think will be realistic. I don't even know if I could implement this myself. Probably not, even. :)

[sfan5]

@sofar
Doing this as a texture modifier is a horrible idea:

Right now texture generation is completely disconnected with the node the texture is going to be on, one "texture string" always corresponds to one texture.
Firstly making it depend on the nodes metadata will require extensive code changes to the texture generation process. Additionally it will completely break caching unless:

• it's disabled for those textures (which would cause the texture to be regenerated on each mesh update)
• or
• the node metadata is somehow(?) also cached with it (which would result in quite many useless cached textures)

[sofar]

@sfan5 well using entities for text IMHO is a complete waste of time. They're unreliable, slow to generate, get lost and are a pain to work with from lua.

So the only real solution is to have something expressly connected to a node (apart from entities and nodes, we only have particles, and they don't seem up to the task either).

So we logically must have (1) a way to add text to a node, which logically results in (2) each mapnode should have a way to specify text, and (3) some way of determining on which side the text goes on the node.

That's not a horrible idea, it's a logical conclusion.

Node coloring does exactly this already, BTW, the only difference is that we don't color it but overlay it with a client-side generated texture.

I don't see how we can do node coloring based on param2, but not node face text on metadata. It's essentially the exact same mechanism.

I could care less about breaking caching if we can get this feature.

[sfan5]

@sofar I'm not saying it's a horrible idea in general, just using texture modifers for this completely fucks up the system of texture modifiers.
Doing this similar to the node coloring is the correct way to implement this.

[numberZero]

@sofar AFAIK node coloring doesn’t create new textures, using the shader to compose the result. But for a text, new texture is necessary, one for each text. (actually a shader may draw text, combining 2d and 1d textures... not sure how slowly ;)

I could suggest something like Node Entity (https://wiki.minetest.net/User:Numzero/NodeEntity), but it may be considered too complicated. Simpler solution would be to introduce some kind of overlays: similar to a modifier, but 1. may only be applied last, 2. does nothing to the (“base”) texture itself, 3. signals that something should be laid over the texture right before adding it to the mesh, or even after, at the rendering stage (I mean multitexturing). This way, the base texture is created and cached as usual.

By the way, with multitexturing there may be another way: the text layer might be added as just another texture, and the mesh could declare where to use it (as a second texture). Not sure does that fit the current mechanics in any way, though.

@sfan5 Your opinion?

[sofar]

@sfan5 actually, all we want is to piggy back on the fact that tiles have an orientation and size. So we can just add a tile parameter and leave the texture alone:

    tiles = {
        {name = "top.png"},
        {name = "bottom.png"},
        {name = "front.png", facetext = true},
        {name = "sides.png"},
    },

maybe something like that? Since this is a new tiledef prop, it would mean an extra u8 to the serialization for the tilespec, but that's reasonable I think.

How does that look?

[juhdanad]

This is how I think this feature should be implemented:

1. Clone node metadata list to the mesh update thread, not just the raw node data.
2. Create a new render layer for overlays (nothing special, just a new container to ensure correct drawing order)
3. Put sliced nodeboxes to the base layer (see below) and text to the overlay layer.
4. Render base layer, then overlay layer, then translucent nodes.

When two triangles share exactly the same vertex coordinates, the video driver will compute exactly the same z-buffer values for the pixels. This ensures that no z-fighting will occur, just the overlay will overwrite the base layer.

This is why the base layer needs to be split: to ensure that the characters' vertices and the base vertices will match.

With this implementation there is no need to create textures (only one, which holds the font). Any size texture and font image will work together. When many signs are in the world, rendering the text is the last step, which means that no material change is needed for the video driver (only one call per font).

[Cat5TV]

This is not just a "good idea" or "cool feature" - this is a MAJOR FIX to Minetest Core. This is critical, to me and players of TPS.

See, we've been chasing after a seeming "bug" on one of our server for a few weeks. When a player would join the game at spawn, the whole map would cack out, spewing errors into the log, and then timeout all the connected players. It'd then come back after a minute, and all would be well until someone joined again. We thought it was technic machines (near the spawn) but no. The problem continued.

Then, another one of our maps began having the same problem, and then, a third.

@tenplus1 figured out the source: SIGNS contained ABMs which would update themselves. Fine and good! However, someone had built an underground area on one server, with around 200 signs in it. On another server, it was a machine surrounded in explanatory signs, and on the other, a creative server, someone had simply loaded up signs in their inventory and went to town.

So on each of these 3 maps, we were experiencing the same symptom, and it was caused by signs updating all at once when there were a ton of them in the same block. Say... > 99.

Signs is my example, because it happened to me. But this exposes a very serious weakness in Minetest which should be fixed. By integrating @sofar's suggestion, I suspect SIGNS would no longer be an issue (no more huge ABM update when someone enters a block containing a ton of signs). It doesn't "fix" the big problem (Minetest allows players to bring a server to its knees by simply placing things) but at least it'd fix this one which we're experiencing.

Note: I do understand my comment does not directly point to a "bug" in minetest or anything like that. My point is, these kinds of issues should be helped by minetest at its core, but for now, sofar's suggestion will at least fix signs mod implementation so people can't bring servers to their knees "by accident". The issue itself goes way beyond signs. Eg., a pipeworks machine that is constantly filling a chest but doesn't have any trash or collection will also crash a server since the block will eventually overflow with objects. Well, these kinds of things just shouldn't happen, in my opinion. Yes, the mod developers should add protection for it (!!) but minetest itself should protect itself.

[sofar]

@juhdanad I'm not entirely following - it seems you want to create extra triangles? I am more thinking of multitexturing the mesh instead (correct me if I'm wrong but doesn't that make it a lot more simple?)

Would what you suggest change the Lua interface?

[juhdanad]

@sofar this is the drawing order I imagined:

The white lines are the edges of the drawn triangles.
This method works without shaders.

I don't know what you mean by multitexturing. Creating a texture for each sign?

Your suggestion of the interface is good IMO as someone might want to draw text on all surfaces.

tiles = {
     {name = "top.png"},
     {name = "bottom.png"},
     {name = "front.png", facetext = "meta_text1", text_vertex=3, other attributes...},
     {name = "sides.png"},
 },

where:
facetext: the metadata key to the text. This allows more textboxes on a node, which is useful for direction signs etc.
The other attributes are only serialized if facetext exists:
text_vertex: for nodeboxes and meshes, you might want to specify where the text should be. Minetest selects the specified vertex and the next three veritces, then computes the text vertices by bilinear interpolation. The default is 0.
text_size: how big the letters are
font: the font to use
rotation: since you can not control the vertex order of nodeboxes and regular nodes, a rotation attribute is needed to orient your text.

[juhdanad]

You have to cache textures or draw the face twice anyways. The latter works even with 10 000 signs. And face count is only one thing: in a mapblock all these triangles are batched into a hardware buffer, so they are drawable with one call. With caching or dynamically generating textures you have to change matrial for each text, and render them one-by-one.

[numberZero]

@juhdanad, then just do it. Write the code, make a PR, I think we all would be glad.

[gaelysam]

I think text shouldn't be an overlay for the node texture. I don't know how this can work this way.
signs_lib uses entities to draw the texts. That's not a bad idea, it works. Certainly this is a heavy and hacky method, because it uses entities (that could have many other uses, that can move…) and are rendered server-side. But it does not need to change the node textures at all.
So I think we could add an entity-like class in the core, intended for displaying text only, without all entity stuff (like velocity, collision box… that takes time and memory) and rendered client-side (the server gives only the text). I don't know whether it's possible with Irrlicht to display text in the 3D scene.

[sofar]

@juhdanad I wouldn't "name" the facetext, just give it a number so it can be packed in a u8 or something simple.

I would really just make only ONE tile textable however. If people want text on more surfaces they can have the same text on two or more surfaces by using the same tile for more faces. It just makes things a lot more simple.

@Gael-de-Sailly there is no way to make entities more light weight and reliable. They will never be more lightweight than when they're part of the base mesh.

[Fixer-007]

I would like to see this added to engine roadmap :)

[paramat]

I agree.

[rdnuk]

Any update on this? also.... what about reusing the method that nametags use? only make the position and yaw of the text fixed.... I don't know enough about how it works but its just an idea.

[rubenwardy]

@MoNTE48 has placed a bounty of $500 for this feature:

I would clarify that I am interested in a "dynamic" text. Which will be stored in the node metadata. As proposed in #1367. This will be useful for creating signs, blackboards, store ads ("Diamonds, price 3 coin"), etc.

See #9946

[LoneWolfHT]

We don't want every map update to cause every sign to be re-font-drawn, so most likely there needs to be a cache of rendered sign text, such that content_mapblock.cpp can just use the cache.

Is the fact that it's harder to remove unused text textures from the cache the reason texture modifiers wouldn't be a good alternative?

[LoneWolfHT]

I've been fiddling around with various places but don't really know for sure where I should put the code for drawing text over node textures, there seems to be multiple places I could put it.
I read something here about content_mapblock.cpp, would that be the correct place? From what I've seen I'm not sure I can use a irr::video::ITexture with the text applied there but I haven't looked too hard

[numberZero]

@LoneWolfHT Remember that textures are static in Minetest. As soon as you write a texture string, the texture is fully defined (modulo texture packs but these can’t be changed when the game is on). Adding a texture modifier for dynamic text would break that core assumption entirely, requiring rethinking of the entire system.

[numberZero]

content_mapblock.cpp is relevant but not a place for dynamic text itself, that code is executed on mapblock mesh generation which doesn’t run on meta change (it would need to, would juhdanad’s suggestion be accepted). It can’t even access the meta IIRC. It can only leave placeholders, but the text itself has to be rendered somewhere else (and rerendered on change).

[LoneWolfHT]

Looks like you're doing it, enjoy lol

[michieal]

Adding myself to this, so that I am notified, and will get updates.
I've been the person working on the Signs in MineClone 2.

[erlehmann]

@michieal see #9946 (comment) and https://git.minetest.land/erlehmann/unicode_text or https://content.minetest.net/packages/erlehmann/unicode_text/ for my own approach to render Unicode text on signs that does not need texture modifiers or engine changes.

[michieal]

I've followed your discussion on the Mineclone issue tracker. thank you for the links, @erlehmann

[erlehmann]

It is my and others' opinion that signs_lib implements the most complete possible way of handling these signs under the engine's present capabilities

How does https://content.minetest.net/packages/cora/ucsigns/ compare in terms of capabilities and memory consumption?