[2] WIP - Stabileo AI multimodal build flow with Claude and image inputs#46
Closed
Batuis wants to merge 2 commits intopr/2-next-product-iterationfrom
Closed
[2] WIP - Stabileo AI multimodal build flow with Claude and image inputs#46Batuis wants to merge 2 commits intopr/2-next-product-iterationfrom
Batuis wants to merge 2 commits intopr/2-next-product-iterationfrom
Conversation
Backend: - Add ImageAttachment type to provider traits (data + media_type) - Claude provider sends multimodal content blocks (image + text) to Anthropic API - BuildModelRequest accepts optional images array - All capabilities updated with images field on AiRequest Frontend: - AiDrawer: image upload button (📎) with file picker, thumbnail preview, remove - client.ts: buildModel accepts optional images parameter - Images sent as base64 in request body, cleared after send Drawer fix: - Move AiDrawer inside pro-body-row flex container so it sits beside the viewport instead of below it (fixes overlap with status bar) Image support is Claude-only for now (other providers ignore the images field).
Frontend: when running on localhost without VITE_AI_API_KEY, auto-use 'dev' token instead of hard-failing. Non-localhost (production) still requires explicit VITE_AI_API_KEY. Backend: DEDALIANO_API_KEY defaults to 'dev' with a warning when not set, instead of panicking. CORS defaults now include localhost:4000. Both sides agree on the 'dev' token for zero-config local development.
Batuis
changed the title
Collaborator
Author
|
Closing: AI multimodal restructuring deferred. Continuing product work in a new stacked PR. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds multimodal image support to the Stabileo AI build workflow. Users can attach reference images in the PRO AI drawer, which are sent through the backend to Claude for vision-assisted structural model generation.
Stacked on PR #45 (
[1]).Changes
Backend (Rust)
ImageAttachmentstruct (base64 data + media_type) added toAiRequestimage+text) to the Anthropic Messages API when images are attached. Falls back to text-only when no images present.BuildModelRequestaccepts optionalimagesarray, forwarded to the provider asImageAttachmentlistimagesfield for compilationFrontend (Svelte/TypeScript)
buildModelaccepts optionalImageAttachmentInput[]parameter, included in POST body when presentDrawer overlap fix
<AiDrawer />rendering inside thepro-body-rowflex container so it sits beside the viewport in PRO mode instead of stacking below it and overlapping the status barScope and limitations
imagesfield but ignore itNot included / deferred