Skip to content

bug: openai base64 image content rejects default image detail #1780

Description

@fangkangmi
  • I have looked for existing issues (including closed) about this

Bug Report

The OpenAI chat-completions converter rejects base64 image content when
Image.detail is None, even though rig::completion::message::Image exposes
detail as Option<ImageDetail> and ImageDetail::default() is Auto.

URL-backed images already treat None as ImageDetail::Auto, but base64-backed
images still return this conversion error:

OpenAI image URI must have image detail

That makes a type-valid image fail only at request-conversion time.

Reproduction

use rig::completion::message::{DocumentSourceKind, Image, ImageMediaType, UserContent};
use rig::providers::openai;

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let image = Image {
        data: DocumentSourceKind::Base64("iVBORw0KGgo=".to_string()),
        media_type: Some(ImageMediaType::PNG),
        detail: None,
        additional_params: None,
    };

    let content = UserContent::Image(image);

    // This is the OpenAI provider conversion path used when building a chat
    // completions request from Rig message content.
    let result = openai::completion::UserContent::try_from(content);

    assert!(result.is_ok());
    Ok(())
}

Today the conversion fails with:

OpenAI image URI must have image detail

The same detail: None value is accepted for URL-backed image content because
that path uses detail.unwrap_or_default().

Expected behavior

Base64 image content should behave the same as URL-backed image content:

detail.unwrap_or_default()

So detail: None should serialize as OpenAI image detail auto.

Additional context

This came up while building a downstream multimodal grading flow. Our message
builder constructed image attachments with detail: None, which is valid at the
Rig type level, but OpenAI request conversion failed at runtime.

The local workaround is to always set:

detail: Some(ImageDetail::Auto)

But since ImageDetail already defaults to Auto, the OpenAI converter can
avoid this footgun by defaulting None in the base64 image path too.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions