Skip to content

Editing a floating-point literal in the inspector truncates it to six decimal places #43

Description

@matt-edmondson

What

AstGraphEditor.DrawFractionField draws a AstFieldKind.Fraction property through ImGuiWidgets.PropertyGrid.Value(string, ref double):

// Coder.Graph/AstGraphEditor.cs:550
double value = double.TryParse(ShownValue(field), NumberStyles.Float, CultureInfo.InvariantCulture, out double fraction)
    ? fraction
    : 0d;

if (grid.Value(field.Name, ref value))
{
    HoldEdit(field, value.ToString(CultureInfo.InvariantCulture));
}

That row is implemented as (ktsu.ImGui.Widgets, PropertyGridRows.cs:172):

bool changed = ImGui.InputDouble(Hidden(label), ref value, 0d, 0d, Options.DoubleFormat);

and PropertyGridOptions.DoubleFormat defaults to "%.6f" — fixed notation, six decimal places. The inspector constructs its grid without overriding it.

InputDouble renders the value through that printf format and parses the user's committed text back out of the box, so the format is not merely cosmetic: it decides what value survives an edit.

Consequence

A LiteralExpression<double> in the document loses precision as soon as the user touches its row:

Document value Shown in the inspector Value written back on commit
3.141592653589793 3.141593 3.141593
1e-9 0.000000 0
2.5e-7 0.000000 0
1e30 1000000000000000000000000000000.000000 — (unreadable, but round-trips)

The small-magnitude cases are the damaging ones: the row displays 0.000000, which reads as a genuine zero, and any edit commits an actual 0. A generated program's constant silently changes value.

Note the AST itself is fine — AstFields.Spell(double) and TryParseDouble are both CultureInfo.InvariantCulture and round-trip correctly. The loss is introduced entirely by the display format of the editor row.

The whole-number path (DrawNumberFieldValue(string, ref int)) is unaffected.

Introduced by

969116b"Draw the inspector's properties as a property grid". Before that change a fraction was a text box, which round-tripped whatever the document held.

Suggested fix

Give the inspector's grid an options instance with a round-trip-capable format, e.g.

new ImGuiWidgets.PropertyGridOptions { DoubleFormat = "%g" }

%g switches to exponent notation for very large and very small magnitudes and drops trailing zeros, which is what a literal in source wants. %.17g is the fully lossless choice if exact round-tripping of every double matters more than readability.

Worth also adding an AstGraphEditor test that writes a small-magnitude literal (1e-9), draws the inspector, and asserts the document value is unchanged when no edit is made and preserved through an edit — the existing tests pass because none of them uses a value the format cannot represent.

Arguably %.6f is also a poor default for a general-purpose widget, but that is a separate call for ktsu.ImGui.Widgets; the data loss here is Coder's to fix by passing options.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions