CreateImageEditRequest forces an application/octet-stream content type for images

[spantaleev]

I'm trying to use the Create image edit API with the gpt-image-1 model via client.images().create_edit() with a CreateImageEditRequest.

#363 is a blocker for multi-image use-cases, but even single images fail.

My code is something like this:

let image_input = ImageInput {
	source: InputSource::VecU8 {
		filename: "image.png".to_string(),
		vec: get_image_bytes(),
	},
};

let mut request_builder = CreateImageEditRequestArgs::default();

request_builder
	.image(image_input)
	.prompt("A surf propped up on the beach".to_owned())
	.size(DallE2ImageSize::S1024x1024)
	.model(ImageModel::Other("gpt-image-1".to_string()));


let request = request_builder.build()
	.map_err(|e| anyhow::anyhow!("Failed to build CreateImageEditRequest: {}", e))?;

let response = self.client.images().create_edit(request).await?;

This fails with:

invalid_request_error: Invalid file 'image': unsupported mimetype ('application/octet-stream'). Supported file formats are 'image/jpeg', 'image/png', and 'image/webp'. (param: image) (code: unsupported_file_mimetype)

---

The payload that's sent to /v1/images/edits looks like this:

Content-Disposition: form-data; name="image"; filename="image.png"
Content-Type: application/octet-stream
...

It seems like the relevant code is:

async-openai/async-openai/src/types/impls.rs

Lines 892 to 900 in aeb6d1f

	impl AsyncTryFrom<CreateImageEditRequest> for reqwest::multipart::Form {
	type Error = OpenAIError;
	async fn try_from(request: CreateImageEditRequest) -> Result<Self, Self::Error> {
	let image_part = create_file_part(request.image.source).await?;
	let mut form = reqwest::multipart::Form::new()
	.part("image", image_part)
	.text("prompt", request.prompt);

.. and:

async-openai/async-openai/src/util.rs

Lines 28 to 61 in aeb6d1f

	/// Creates the part for the given file for multipart upload.
	pub(crate) async fn create_file_part(
	source: InputSource,
	) -> Result<reqwest::multipart::Part, OpenAIError> {
	let (stream, file_name) = match source {
	InputSource::Path { path } => {
	let file_name = path
	.file_name()
	.ok_or_else(|| {
	OpenAIError::FileReadError(format!(
	"cannot extract file name from {}",
	path.display()
	))
	})?
	.to_str()
	.unwrap()
	.to_string();
	(
	file_stream_body(InputSource::Path { path }).await?,
	file_name,
	)
	}
	InputSource::Bytes { filename, bytes } => (Body::from(bytes), filename),
	InputSource::VecU8 { filename, vec } => (Body::from(vec), filename),
	};
	let file_part = reqwest::multipart::Part::stream(stream)
	.file_name(file_name)
	.mime_str("application/octet-stream")
	.unwrap();
	Ok(file_part)
	}

It appears that create_file_part() hardcodes a mime type of application/octet-stream.

While this is probably OK for older models and some auto-detection may have been happening, the gpt-image-1 model appears to be picky.

---

I'm not sure what the best way to solve this is, but one possibility may be to add an additional field to each InputSource variant. Maybe something like:

 #[derive(Debug, Clone, PartialEq)]
 pub enum InputSource {
-   Path { path: PathBuf },
+   Path { path: PathBuf, mime_type: Option<String> },
-   Bytes { filename: String, bytes: Bytes },
+   Bytes { filename: String, bytes: Bytes, mime_type: Option<String> },
-   VecU8 { filename: String, vec: Vec<u8> },
+   VecU8 { filename: String, vec: Vec<u8>, mime_type: Option<String> },
 }

This could then be used in create_file_part().