|
3 | 3 | A Flutter library to enable developers to easily add interactive |
4 | 4 | generative UI to their applications. |
5 | 5 |
|
| 6 | +See the [Getting started with GenUI](https://www.youtube.com/watch?v=nWr6eZKM6no) video for an overview of the package. |
| 7 | + |
| 8 | +[<img src="docs/assets/genui_intro_video_still.png" alt="GenUI Intro video still" height="500">](https://www.youtube.com/watch?v=nWr6eZKM6no) |
| 9 | + |
6 | 10 | ## Status: Highly Experimental |
7 | 11 |
|
8 | 12 | This is a highly experimental package, which means the API will change (sometimes drastically). |
@@ -37,58 +41,6 @@ chatbots and next-generation agent-based user experiences. |
37 | 41 | - Create dynamically composed UIs: an agent can generate a complete form with sliders, date pickers, |
38 | 42 | and text fields on the fly based on a user's request to "book a flight." |
39 | 43 |
|
40 | | -## Example |
41 | | - |
42 | | -### Using the GenUI SDK with Firebase AI Logic |
43 | | - |
44 | | -```dart |
45 | | -// Adding your widgets into the catalog. |
46 | | -// Start with the built-in catalog and add your own custom widgets. |
47 | | -final catalog = CoreCatalogItems.asCatalog().copyWith([ |
48 | | - myCustomNewWidget, |
49 | | -]); |
50 | | -
|
51 | | -/// Initializing the library. |
52 | | -final genUiManager = GenUiManager(catalog: catalog); |
53 | | -final contentGenerator = FirebaseAiContentGenerator( |
54 | | - catalog: catalog, |
55 | | - systemInstruction: ''' |
56 | | - You are a bicycle maintenance assistant who is an expert in diagnosing issues and |
57 | | - giving step-by-step instructions. |
58 | | - ''', |
59 | | -); |
60 | | -late final _genUiConversation = GenUiConversation( |
61 | | - genUiManager: genUiManager, |
62 | | - contentGenerator: contentGenerator, |
63 | | - onSurfaceAdded: _onSurfaceAdded, |
64 | | - onSurfaceDeleted: (_) {}, |
65 | | - onTextResponse: (_) {}, |
66 | | - // ignore: avoid_print |
67 | | - onWarning: (value) => print('Warning from GenUiConversation: $value'), |
68 | | -); |
69 | | -
|
70 | | -// Put the surface, added by AI, to the list of messages that should be rendered, |
71 | | -// trigger re-render, and scroll to bottom. |
72 | | -void _onSurfaceAdded(SurfaceAdded surface) { |
73 | | - if (!mounted) return; |
74 | | - setState(() { |
75 | | - _messages.add(MessageController(surfaceId: surface.surfaceId)); |
76 | | - }); |
77 | | - _scrollToBottom(); |
78 | | -} |
79 | | -
|
80 | | -// Render that UI. |
81 | | -Widget build(BuildContext context) { |
82 | | - if (type == MessageType.genUi) { |
83 | | - return GenUiSurface( |
84 | | - host: _genUiConversation.genUiManager.host, |
85 | | - surfaceId: _surfaceId, |
86 | | - onEvent: _handleEvent, |
87 | | - ); |
88 | | - } |
89 | | -} |
90 | | -``` |
91 | | - |
92 | 44 | ## Look & Feel |
93 | 45 |
|
94 | 46 | ### Interactive [Travel App Example](examples/travel_app/) |
@@ -120,31 +72,44 @@ based on a widget catalog from the developers' project. |
120 | 72 | - **Basic Layout:** LLM-driven basic layout generation. |
121 | 73 | - **Any Model:** Integrate with any LLM that can generate structured JSON output. |
122 | 74 |
|
123 | | -## Some things we're thinking about |
| 75 | +## Connecting to an AI Agent |
124 | 76 |
|
125 | | -- **Genkit Integration:** Integration with Genkit. |
126 | | -- **Expanded LLM Framework Support:** Official support for additional LLM frameworks. |
127 | | -- **Streaming UI:** Support for progressively rendering UI components as they stream from the LLM. |
128 | | -- **Full-Screen Composition:** Enable LLM-driven composition and navigation of entire app screens. |
129 | | -- **A2A Agent Support:** Support for A2A agent interactions. |
130 | | -- **Dart Bytecode:** Future support for Dart Bytecode for even greater dynamism and flexibility. |
| 77 | +The `genui` framework uses a `ContentGenerator` to communicate with a generative AI model, |
| 78 | +allowing `genui` to be backend agnostic. You can choose the implementation that best fits |
| 79 | +your needs, whether it's `FirebaseAiContentGenerator` for production apps, |
| 80 | +`GoogleGenerativeAiContentGenerator` for rapid prototyping, or `A2uiContentGenerator` for |
| 81 | +custom agent servers. |
| 82 | + |
| 83 | +See the package table below for more details on each. |
131 | 84 |
|
132 | 85 | ## Packages |
133 | 86 |
|
134 | | -| Package | Description | |
135 | | -| ---------------------------------------------------------------- | ----------------------------------------------------------------------------- | |
136 | | -| [genui](packages/genui/) | A framework to employ Generative UI. | |
137 | | -| [genui_firebase_ai](packages/genui_firebase_ai/) | Firebase AI integration for genui. | |
138 | | -| [json_schema_builder](packages/json_schema_builder/) | A fully featured Dart JSON Schema package with validation. | |
| 87 | +| Package | Description | |
| 88 | +| ------------------------------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 89 | +| [genui](packages/genui/) | The core framework to employ Generative UI. | |
| 90 | +| [genui_firebase_ai](packages/genui_firebase_ai/) | Provides **`FirebaseAiContentGenerator`** to connect to Gemini via Firebase AI Logic. This is the recommended approach for production apps based on client-side agents. | |
| 91 | +| [genui_google_generative_ai](packages/genui_google_generative_ai/) | Provides **`GoogleGenerativeAiContentGenerator`** for connecting to the Google Generative AI API with only an API key. Ideal for getting started quickly. | |
| 92 | +| [genui_a2ui](packages/genui_a2ui/) | Provides **`A2uiContentGenerator`** for connecting to any server that implements the [A2UI protocol](https://a2ui.org). Use this for integrating with custom agent backends. | |
| 93 | +| [json_schema_builder](packages/json_schema_builder/) | A fully featured Dart JSON Schema package with validation, used by the core framework to define widget data structures. | |
139 | 94 |
|
140 | | -## Usage |
| 95 | +## Getting Started |
141 | 96 |
|
142 | | -See [packages/genui/USAGE.md](packages/genui/USAGE.md). |
| 97 | +See the [genui getting started guide](packages/genui/README.md#getting-started-with-genui). |
143 | 98 |
|
144 | 99 | ## Constraints |
145 | 100 |
|
146 | 101 | This repo requires Flutter version >=3.35.7. |
147 | 102 |
|
| 103 | +## Some things we're thinking about |
| 104 | + |
| 105 | +- **Genkit Integration:** Integration with Genkit. |
| 106 | +- **ADK Plugin:** turnkey integration with ADK. |
| 107 | +- **Expanded LLM Framework Support:** Official support for additional LLM frameworks. |
| 108 | +- **Streaming UI:** Support for progressively rendering UI components as they stream from the LLM. |
| 109 | +- **Full-Screen Composition:** Enable LLM-driven composition and navigation of entire app screens. |
| 110 | +- **A2A Agent Support:** Support for A2A agent interactions. |
| 111 | +- **Dart Bytecode:** Future support for Dart Bytecode for even greater dynamism and flexibility. |
| 112 | + |
148 | 113 | ## Contribute |
149 | 114 |
|
150 | 115 | See [CONTRIBUTING.md](CONTRIBUTING.md) |
0 commit comments