Lossy Abstraction for Generated Responses

[leoafarias]

The ContentGenerator interface currently exposes AI responses as Stream<String>, which discards the rich typed data that AI SDKs provide. The typed infrastructure (MessagePart, TextPart, ToolCallPart, ThinkingPart, etc.) already exists in GenUI but is only used for input to models, not output from them.

Should GenUI expose the full typed AI response to consumers, similar to what the underlying AI clients provide?

The current Stream<String> abstraction prevents us from accessing thinking blocks, tool calls, image/multimodal parts, finish reasons, token usage, safety ratings, and citations, all of which are available from the underlying SDK but discarded before reaching the application layer.

We noticed while implementing an interface that would output all tool calls and results, just so we can review the workflow, and we noticed that we can see the calls on the logs, but we wanted them on the interface. However, I think Gen UI needs to expose all parts of the response, or we lose a lot of features that are currently in the AI SDK itself.

I am still exploring some possible workarounds on this, using the GenUI for this use case, but I think there are many use cases where they do become blockers.

As a bigger question, it seems that the https://github.com/flutter/ai also had to recreate these parts and so on, and there is overlap between the parts and implementations on these two packages. Would maybe consolidating those primitives in a separate package make sense also?

[gspencergoog]

Should GenUI expose the full typed AI response to consumers, similar to what the underlying AI clients provide?

That's a great point! Could you file an issue around that, and we'll see if we can provide an alternate interface that returns the parts. I think we probably should either just convert the Stream<String> to a Stream<Part> or provide an alternate call that does that. The issue will help us prioritize work around it.

As a bigger question, it seems that the flutter/ai also had to recreate these parts and so on, and there is overlap between the parts and implementations on these two packages. Would maybe consolidating those primitives in a separate package make sense also?

That's not a bad idea either. We'd have to figure out a common API that covers all of the models we want to support, but that should be possible, at least for a common subset. What do you think of that, @csells ?

[csells]

There are at least three sets of LLM primitives: the AI Toolkit, GenUI and dartantic_interface. I've implemented two of these and dartantic_interface is far more complete. Do we want to just use dartantic_interface? Or if we'd like to settle on a subset, eg chat messages and parts, I'm happy to produce an "llm_primitives" package that we can all depend on and stop doing the conversions.

[polina-c]

+1 for consolidating those primitives in a separate package

These primitives are used not just by Dash team packages, but also by user packages.

Recently I faced an application that is dealing with two libraries defining these primitives, and thus gets name conflict and necessity to convert one tool call to another tool call.

[polina-c]

Suggesting one more primitive for this list: ToolDefinition.

[gspencergoog]

Yes, @csells let's see if we can convert genui over to use dartantic_interface, or probably a renamed version so that it doesn't seem so exclusively tied to dartantic. I like llm_primitives as a name. It makes sense to not keep reinventing wheels here.

[leoafarias]

We have created a package that has adapters and full compatibility with zod and is currently in beta and does typed schema values using typed extensions with code generation and is working well. Maybe there is opportunity for collaboration on this also.

https://github.com/btwld/ack

We created an adapter to the json_schema_builder and is working well but has some features which are good on DX!

[leoafarias]

If json_schema_builder becomes the de facto standard, would love to use it as the main internals for ack

[gspencergoog]

@leoafarias What does ack add on top of json_schema_builder? It looks pretty similar, just uses a different way to specify args (e.g. Ack.string().minLength(2), instead of S.string(minLength: 2)). Is it the "safeParse" functionality?

[leoafarias]

@gspencergoog we created it before the schema builder, so I think there is overlap now, but a quick sumary from your friend Gemini.

ack vs json_schema_builder

	ack	json_schema_builder
Philosophy	Code-first, Zod-inspired	Spec-first, JSON Schema compliant
Goal	Generate Schema, Parse & transform Dart data	Validate JSON against schema
Output	Typed*, transformed values	Validation errors

In short: ack parses data into typed Dart objects. json_schema_builder validates JSON against a spec.

---

What Makes ack Different

refine() — Custom Validation

final passwordSchema = Ack.string().minLength(8);

final schema = Ack.object({
  'password': passwordSchema,
  'confirmPassword': passwordSchema,
}).refine(
  (data) => data['password'] == data['confirmPassword'],
  message: 'Passwords must match',
);

transform() — Type Conversion

final dateSchema = Ack.string()
    .matches(r'^\d{4}-\d{2}-\d{2}$')
    .transform<DateTime>((s) => DateTime.parse(s!));

dateSchema.parse('2024-01-15'); // Returns DateTime, not String

Schema Composition

final adminSchema = userSchema.extend({'role': Ack.string()});
final updateSchema = userSchema.partial();  // All fields optional
final publicSchema = userSchema.pick(['name', 'email']);
final safeSchema = userSchema.omit(['password']);

---

Code Generation with Extension Types

@AckType()
final userSchema = Ack.object({
  'name': Ack.string(),
  'age': Ack.integer(),
});

// Generated:
extension type UserType(Map<String, Object?> _data)
    implements Map<String, Object?> {
  static UserType parse(Object? data) => UserType(userSchema.parse(data)!);
  
  String get name => _data['name'] as String;
  
  int get age => _data['age'] as int;
  UserType copyWith({String? name, int? age}) => UserType.parse({...});
}

Why extension types?

• Zero-cost — Compiles away, no runtime overhead
• Direct JSON — Map<String, Object?> is native JSON, no serialization needed
• Validation enforced — Must go through parse() to construct

---

Summary

Feature	ack	json_schema_builder
Custom validation	✅ refine()	❌
Type transformation	✅ transform()	❌
Schema composition	✅ extend/partial/pick/omit	❌
Discriminated unions	✅ native	⚠️ manual
JSON Schema export	✅ Draft-7	✅ Draft 2020-12

Bottom line: ack composes schemas via code. json_schema_builder composes via spec keywords.

[gspencergoog]

Oooh, that's pretty nice! I like the transform operation especially.

It seems pretty powerful. And if you did base the validation on json_schema_builder, you'd get the 2020-12 support too.

[polina-c]

I just discussed it with @jacobsimionato

Yes, let's create package ai_primitives in genui/packages and start iterating on it. When it is good enough, we will publish it. And, yes, it is good to start with forking dartantic_interface.

@csells , do you want to create PR that constructs the initial version?

[polina-c]

Thank you!

Published empty package to reserve the name: https://pub.dev/packages/ai_primitives
(It is unlisted now.)

Code: #606

[polina-c]

As discussed in other threads, changed the name: https://pub.dev/packages/genai_primitives/versions

@csells , when you think you can issue the PR? (no rush here, I just want to understand the schedule)

[csells]

Should be early next week. Sooner depending on how my Friday goes. I'm on a 27-hour train ride right now with no Wi-Fi or you'd have it sooner. : )

[polina-c]

@csells , thank you for PR (#619)

I raised a lot of questions/suggestions. Let me know if you want to actively participate in the discussion or you just want me to merge the PR and follow up on other PRs.

We want to be thoughtful about details, because we will have a lot of dependents right away, and breaking changes will be impactful.

[csells]

Let's iterate on this PR tomorrow when I'm off this train.

[jacobsimionato]

A related idea: Maybe Gen UI should switch to using Dartantic as the shared tool loop logic for people building client-side agents.

Currently, we have genui_firebase_ai and genui_google_generative_ai which each implement a tool loop which should be deduplicated (see #555). But building out a Dart agent framework is kind of an adjacent problem to Gen UI, so we ideally wouldn't own that. If people are using genui_firebase_ai, then they'll eventually want other LLM/ agent features like what @leoafarias pointed out - thinking blocks, tool calls etc. And even MCP, A2A etc.

Someone implementing a local agent with Gen UI needs a stack like:

• Gen UI library (Currently genui)
• Agent framework (Gen UI team has two primitive implementations of this in genui_firebase_ai etc)
• LLM API(s) (firebase_ai etc)

What if we swapped the middle layer to always be dartantic_ai via genui_dartantic_ai, so that we don't have to maintain it, and customers get a more complete agent framework?

So we'd just merge genui_dartantic_content_generator and build an integration between firebase_ai and genui_dartantic_content_generator so we can still provide a full stack Google solution with firebase_ai at the bottom.

For server-based agents, we would still provide genui_a2ui which would not use the Dartantic.

All of the packages above would use ai_primitives which would make them super easy to slot together (in theory!)

@sethladd and @polina-c what do you think of depending on dartantic in this way?

@csells do you have any plans for a Firebase AI integration with Dartantic?

[csells]

@gspencergoog makes a compelling point wrt the json schema validation for the schema builder. Of course, the ergonomics are better, too. I can see building the genai_primities on top of the schema builder package and moving dartantic to use it as well.

@leoafarias I've got a PR for firebase_ai that's been waiting on my dartantic 2.0 release, which I've now landed. If you want to use dartantic as the middle layer of genui, I can land the firebase_ai PR sooner rather than later.

[leoafarias]

@csells we have an agent workflow outside of GenAI that Dartantic would work great for, so yes!

[jacobsimionato]

This is all very exciting! So to confirm: we're working towards this incredible package structure?

[csells]

I don't think any is refactoring firebase_ai (or google_cloud_ai_generativelanguage_v1beta or any of the other packages that dartantic_ai depends on to talk to the various LLMs) but the rest looks right.