XAML → Markdown UI Indexer (accessibility-first, source generator)

[mattleibow]

XAML → Markdown UI Indexer

A compile-time source generator that creates AI-friendly semantic representations of every XAML page. The indexer outputs what a screen reader would announce — not layout structure — making the entire UI discoverable by AI agents without running the app.

New Projects

Project	Description
Microsoft.Maui.AI.Indexer	Runtime types (UiPageIndex base class, UiPageEntry) + build targets
Microsoft.Maui.AI.Indexer.Generators	Source generator (bundled in the NuGet package)
Microsoft.Maui.AI.Indexer.Tests	104 exact-match tests

How to Use

1. Add the package (one reference — the generator is bundled):

<PackageReference Include="Microsoft.Maui.AI.Indexer" />

2. Build your project. The generator emits:

• One {Page}_UiIndex.g.cs per XAML page (static class with const string Markdown)
• One {AssemblyName}UiIndex.g.cs aggregate (inherits UiPageIndex, has Default singleton)

3. Access at runtime — no reflection, no registration, no startup code:

// The generated class follows the AIToolContext pattern:
// {SanitizedAssemblyName}UiIndex : UiPageIndex
var index = MyAppUiIndex.Default;

// Browse all pages
foreach (var page in index.Pages)
    Console.WriteLine($"{page.Name}: {page.Markdown}");

// Find a specific page
var detail = index.FindByName("ProductDetailPage");
Console.WriteLine(detail?.Markdown);

// Find by Shell route
var orders = index.FindByRoute("orders");

// Access a single page's markdown directly
Console.WriteLine(ProductDetailPage_UiIndex.Markdown);

4. Wire as AI tools (see Garden sample UiDiscovery.cs):

public sealed class UiDiscovery
{
    // Reference the generated index directly — no reflection
    private static UiPageIndex Index => MyAppUiIndex.Default;

    [ExportAIFunction("search_ui")]
    [Description("Search the app's UI pages for controls, labels, or features.")]
    public static string SearchUi(string[] searchTerms)
    {
        foreach (var page in Index.Pages)
            // match terms against page.Markdown ...
    }

    [ExportAIFunction("get_page_ui")]
    [Description("Get the full semantic UI description of a specific page.")]
    public static string GetPageUi(string pageName)
    {
        return Index.FindByName(pageName)?.Markdown ?? "Page not found";
    }
}

5. Multi-assembly support — each assembly gets its own generated index. The consumer collects them:

var allPages = MyAppUiIndex.Default.Pages
    .Concat(MyLibraryUiIndex.Default.Pages)
    .ToList();

What the Generator Produces

Actual generated output for MainPage (with cross-file resolution inlining ChatView → CartPane → CartView):

# MainPage
File: Pages/MainPage.xaml

- Label: "Sage app icon"
- Heading (level 1): "Sage"
- Button: "Products" [hint: Opens the product catalog]
- Button: "Orders" [hint: Opens your past orders]
- Button: "New chat" → StartNewSessionCommand [hint: Starts a new conversation]
- [ChatView]:
  - CollectionView: "{Messages}"
    - Empty view:
      - Heading (level 1): "Welcome to Sage, your personal garden shopper"
      - FlexLayout with items from "{AvailableTools}":
        - Each item:
          - Label: "{Name}"
          - Label: "{Description}"
  - When [visible when IsInputVisible = true]:
    - Entry: "{InputText}"
    - Button: "Send" → SendCommand [hint: Sends your message]
  - When [visible when IsApprovalPending = true]:
    - Button: "Approve" → ApproveCommand [hint: Approves the pending action]
    - Button: "Reject" → RejectCommand [hint: Rejects the pending action]
- [CartPane]:
  - Heading (level 1): "Cart"
  - [CartView]:
    - CollectionView: "{Items}" [visible when IsNormalMode = true]
      - Each item:
        - Label: "{Name}"
        - Label: "{QuantityLine}"
    - When [visible when HasItems = true]:
      - Button: "{CartTotal}" → CheckoutCommand [hint: Proceeds to checkout]

Design Principles

Principle	Details
Accessibility-first	Skips Grid, StackLayout, Border, ScrollView — only semantic content
No reflection	Generated code references types directly. Follows the AIToolContext / JsonSerializerContext pattern
App owns search	The generator produces markdown. Search strategy (simple string match, RAG, vector DB) is up to the consumer
Per-assembly	Each assembly gets its own {Name}UiIndex : UiPageIndex with Default singleton. Multi-assembly apps compose them
Trim-safe	No reflection in any path. Pure const strings and static arrays

Features

Feature	Details
SemanticProperties extraction	Description overrides text, Hint appended, HeadingLevel → heading depth
Cross-file user control resolution	<views:CartView/> → inlined with caching for reuse
Nested resolution	Page → Outer → Inner chains resolve recursively with cycle detection
CollectionView templates	Header, GroupHeader, Items, GroupFooter, Footer, EmptyView — all inlined
BindableLayout templates	Parent as container with Each item: children
Conditional visibility	[visible when X = true], inverse converters, layout condition groups
DataTrigger conditions	[visible when X = True] / [hidden when X = True]
Always-hidden skip	IsVisible="False" elements omitted entirely
Promoted containers	Layout with Description renders with children preserved
Property-element support	ContentPage.Content, ScrollView.Content traversed correctly
Shell routes	TabBar, Tab, FlyoutItem, ShellContent with routes
Duplicate name safety	Namespace-qualified hint names prevent AddSource collisions

Garden Sample Changes

• Added SemanticProperties (Description, Hint, HeadingLevel) across all XAML pages and views
• Added UiDiscovery service with three [ExportAIFunction] tools:
  • search_ui(searchTerms) — multi-term search across all pages with relevant snippets
  • get_page_ui(pageName) — full semantic markdown for a page
  • list_app_pages() — list all pages with routes

Test Coverage

104 exact-match tests using Assert.Equal — no Contains or partial matching.

Suite	Tests	Covers
ExactOutputTests	31	All controls, templates, conditions, cross-file, Shell, errors
AdditionalExactTests	27	Remaining controls, aggregate, converters, edge cases
EmojiAndUnicodeTests	28	Emoji, CJK, RTL, XML entities, empty elements
ReviewFindingTests	18	Duplicate names, layout conditions, property elements, cycles

Future Roadmap

Feature	Impact
ViewModel discovery via x:DataType	Links UI to data layer
DataTemplateSelector variant detection	All template options per list
RAG embedding generation (sample)	Vector search over UI
MCP tool integration (maui_ui_search)	AI agents query via DevFlow
Missing a11y diagnostics	Warn on elements without SemanticProperties

Base branch: mattleibow/ai-annotations

[Copilot]

Copilot wasn't able to review this pull request because it exceeds the maximum number of lines (20,000). Try reducing the number of changed lines and requesting a review from Copilot again.

[github-actions]

Expert Code Review — PR #269

Methodology: 3 independent reviewers with adversarial consensus. Due to the large diff (253 files, 20,980 additions), files were split into 3 batches (infra+source / samples / tests) with each reviewer analyzing one batch. All findings are annotated "low confidence — single reviewer (batch split)" with severity downgraded by one level per protocol.

11 findings posted as inline comments (2 moderate, 9 minor)

Overflow Findings

The following findings could not be posted inline or were lower priority:

#	Severity	File	Line(s)	Finding
12	🟢 Minor	samples/.../MainViewModel.cs	57–70	CancellationToken not propagated to AI tool methods — navigation side effects can fire after session teardown
13	🟢 Minor	samples/.../CartViewModel.cs	62–73	SyncCollection always creates new VMs and replaces items even when already in correct position, defeating in-place diffing
14	🟢 Minor	samples/.../PreferencesOrderArchive.cs	21–26	Read-increment-write on Preferences counter is not atomic; concurrent AI tool calls can produce duplicate order IDs
15	🟢 Minor	samples/.../OrdersViewModel.cs	12	Transient VM registers with WeakReferenceMessenger without paired unregister; each navigation creates a new listener
16	🟢 Minor	tests/.../XamlFileParserTests.cs	308–311	Parse_SkipsXamlWithoutXClass uses XAML with x:Key but missing xmlns:x declaration — test passes because XML parsing fails, not because missing x:Class is handled
17	🟢 Minor	tests/.../XamlFileParserTests.cs	316–319	Test only asserts one specific file is absent; doesn't verify UiIndex.g.cs aggregate exists or is correct
18	🟢 Minor	tests/.../GeneratorCompilationTests.cs	39–62	GetGeneratedSource silently falls back to SyntaxTrees.Last() when context class not found, masking missing output
19	🟢 Minor	eng/Versions.props	12–19	Side-effect version bumps: Microsoft.Extensions.AI 10.3.0→10.4.1, OpenAI 2.6.0→2.9.1, CommunityToolkit.Mvvm 8.4.0→8.4.2 affect existing EssentialsAI product
20	🟢 Minor	src/.../AccessibilityExtractor.cs	27, 31, 35	Operator precedence in semantic property matching lacks explicit parentheses — correct behavior but fragile for refactoring
21	🟢 Minor	src/.../MarkdownBuilder.cs	120	Complex inline string formatting with .TrimStart/.Replace chains has edge cases with bracket formatting

CI Status

• ✅ license/cla: passed
• ✅ build / build (macos-latest) — CI-AI: passed
• ✅ build / build (windows-latest) — CI-AI: passed
• ✅ build / build (ubuntu-24.04) — Linux GTK4: passed
• ❌ build / build (macos-latest) — one workflow (not CI-AI) failed
• 🔄 build-macos / build (macos-latest) — in progress

Test Coverage Assessment

The PR includes 16 XAML indexer tests and extensive attribute generator tests (snapshot + runtime). Coverage for the source generator core logic is good. Gaps: no test for the """ raw string edge case in PageCodeEmitter, no test for IncludeTools + ExcludeTools both set, void-return schema test is misnamed/mislabeled.

Generated by Expert Code Review · 3 independent reviewers with adversarial consensus

Generated by Expert Code Review (auto) for issue #269 · ● 37.7M · ◷

[github-actions]

Expert Code Review: 11 findings posted inline (2 moderate, 9 minor). See the lean summary comment for full details and methodology.

Generated by Expert Code Review (auto) for issue #269 · ● 37.7M

[github-actions]

🟡 MODERATE · low confidence — single reviewer (batch split)

NuGet package does not include the source generator DLL. Unlike Microsoft.Maui.AI.Attributes.csproj (which has a ProjectReference to its generators project with OutputItemType="Analyzer" and a target to pack it into analyzers/dotnet/cs), this csproj has no such reference.

Scenario: A consumer installs Microsoft.Maui.AI.Indexer from NuGet. The build targets add XAML files as AdditionalTexts, but no source generator processes them — nothing is generated, no compiler error, total silent failure.

Recommendation: Add a ProjectReference + pack target mirroring Microsoft.Maui.AI.Attributes.csproj:

<ProjectReference Include="..\Microsoft.Maui.AI.Indexer.Generators\Microsoft.Maui.AI.Indexer.Generators.csproj"
                  OutputItemType="Analyzer"
                  ReferenceOutputAssembly="false" />

Plus the _ResolveGeneratorPath target to pack the generator into analyzers/dotnet/cs.

[mattleibow]

Good catch — this is intentional for now since the package isn't being published yet (it's wired via ProjectReference in the Garden sample). When we package for NuGet, we'll add the ProjectReference with OutputItemType="Analyzer" and the pack target, mirroring the AI Attributes pattern. Tracked for the packaging milestone.

[github-actions]

🟢 MINOR · low confidence — single reviewer (batch split)

Incomplete string escaping. This manual Replace handles \ and " but not \n, \r, \t, null, or other control characters. A keyed service key containing a newline would produce a syntax error in the generated code.

The codebase already has CodeEmitter.Escape (line 302) using SymbolDisplay.FormatLiteral which handles all edge cases correctly.

Recommendation: Replace with SymbolDisplay.FormatLiteral(s, quote: true) to match the existing pattern.

[mattleibow]

This file is from the base branch (mattleibow/ai-annotations) and isn't modified by this PR. I'll track this for a separate fix in the AI Attributes PR.

[github-actions]

🟡 MODERATE · low confidence — single reviewer (batch split)

OnPropertyChanged fires on this (ChatViewModel), not on toolMsg. ObservableObject.OnPropertyChanged is a protected instance method — calling it without a receiver fires on the current instance. ChatViewModel has no HasDetails property, so this notification is entirely useless. Meanwhile, ChatMessageViewModel.HasDetails is a computed property with no [NotifyPropertyChangedFor] on ToolArgs or ToolResult, so the tool-detail expand button stays hidden after results arrive.

Recommendation: Remove this line and add [NotifyPropertyChangedFor(nameof(HasDetails))] to both ToolArgs and ToolResult in ChatMessageViewModel.

[mattleibow]

This file is from the base branch (mattleibow/ai-annotations) — the Garden sample's ChatViewModel isn't modified by the indexer PR. Good find though, I'll fix it in a follow-up to the Garden sample.

[github-actions]

🟢 MINOR · low confidence — single reviewer (batch split)

IReadOnlyList<Order> backed by the mutable List<Order>. External callers can cast to List<Order> and mutate the internal collection, bypassing class invariants. This is a poor pattern to demonstrate in a sample that developers will copy.

Recommendation: Return a defensive copy: public IReadOnlyList<Order> Orders => [.. _orders]; (same pattern already used in CurrentCart.Items).

[mattleibow]

This file is from the base branch (mattleibow/ai-annotations). Agreed on the defensive copy pattern — I'll address in a separate Garden sample cleanup.

[github-actions]

🟢 MINOR · low confidence — single reviewer (batch split)

Test name/implementation mismatch. Schema_void_return_has_null_return_schema tests negate_number (which returns int) and asserts NotNull — so it's not actually testing the void-return case at all. A regression in void-return schema generation would go undetected.

Recommendation: Add a test that uses an actual void-returning exported function and asserts ReturnJsonSchema is null (or whatever the expected behavior is for void).

[mattleibow]

This test file is from the base branch. I'll fix the test name and add proper void-return coverage in a follow-up to the AI Attributes tests.

[github-actions]

🟢 MINOR · low confidence — single reviewer (batch split)

EscapeForRawStringLiteral is a no-op — returns input unchanged. If any XAML page content contains """, the generated raw string literal will be malformed C# that fails compilation with a cryptic error.

Recommendation: Implement the method: detect if text contains """ and use longer delimiters ("""" ... """"), or switch to a regular escaped string literal which handles all content safely.

[mattleibow]

Fixed ✅ — EscapeForRawStringLiteral now detects if the text contains \"\"\" and dynamically extends the delimiter (\"\"\"\" ... \"\"\"\"). Returns a (text, delimiter) tuple. Added a test with triple-quotes in SemanticProperties.Description to verify.

[github-actions]

🟢 MINOR · low confidence — single reviewer (batch split)

MAUIAI005 diagnostic is defined but never emitted. IncludeAndExcludeBothSet is never called from SymbolAnalysis.cs — when both IncludeTools and ExcludeTools are set, the generator silently applies both filters sequentially instead of reporting an error.

Recommendation: Add a check in SymbolAnalysis.GetContextModel before the filter logic:

if (includeTools is { Length: > 0 } && excludeTools is { Length: > 0 })
{
    diagnostics.Add(DiagnosticInfo.IncludeAndExcludeBothSet(typeName, location));
    continue;
}

[mattleibow]

This file is from the base branch (mattleibow/ai-annotations) — not modified by the indexer PR. I'll add the check in a follow-up to the AI Attributes generator.

[github-actions]

🟢 MINOR · low confidence — single reviewer (batch split)

OperationCanceledException is caught as an error. When the user starts a new session, _cts.Cancel() triggers OperationCanceledException which is caught here and surfaced as "Error: The operation was canceled." in the new session's message list. Cancellation is a first-class signal, not an error.

Recommendation: Add a preceding catch:

catch (OperationCanceledException) { /* user-initiated cancellation — not an error */ }
catch (Exception ex) { AddMessage(ChatMessageKind.Error, $"Error: {ex.Message}"); }

[mattleibow]

This file is from the base branch (mattleibow/ai-annotations). Good catch — cancellation should not surface as an error message. I'll fix in a separate Garden sample PR.

[github-actions]

🟢 MINOR · low confidence — single reviewer (batch split)

"Shadow" appears in both StructuralElements (here) and IgnoredElements (line 44). Since StructuralElements is checked first (line 134), Shadow is treated as a structural container whose children are walked — but Shadow is a visual effect with no meaningful children in MAUI. The entry in IgnoredElements is dead code.

Recommendation: Remove "Shadow" from StructuralElements and keep it only in IgnoredElements.

[mattleibow]

Fixed ✅ — removed Shadow from StructuralElements. It remains only in IgnoredElements where it belongs.

[github-actions]

🟢 MINOR · low confidence — single reviewer (batch split)

Bare catch swallows all exceptions including critical ones like OutOfMemoryException. In a source generator running inside the compiler process, this could mask serious issues.

Recommendation: Narrow to catch (System.Xml.XmlException) or at minimum catch (Exception) (which doesn't catch SEH exceptions).

[mattleibow]

Fixed ✅ — narrowed to catch (System.Xml.XmlException). Other exceptions (OOM, etc.) will now propagate correctly.

[jfversluis]

Build errors

[mattleibow]

/evaluate

[github-actions]

Skill Evaluation Results

20260512-183712

No markdown report

Full results