Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions config/config.exs
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,7 @@ config :vmemo,
ecto_repos: [Vmemo.Repo],
generators: [timestamp_type: :utc_datetime],
ash_domains: [Vmemo.Admin, Vmemo.Account, Vmemo.Memo, Vmemo.Ai, Vmemo.Chat, Vmemo.Jobs],
image_upload_max_file_size: 50_000_000,
user_data_import_typesense_chunk_size: 50,
user_data_import_typesense_chunk_pause_ms: 50

Expand Down
13 changes: 13 additions & 0 deletions config/runtime.exs
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,19 @@ config :vmemo,

config :vmemo, Vmemo.Repo, url: database_url

if image_upload_max_file_size = System.get_env("IMAGE_UPLOAD_MAX_FILE_SIZE") do
case Integer.parse(image_upload_max_file_size) do
{value, ""} when value > 0 ->
config :vmemo, image_upload_max_file_size: value

_ ->
raise """
environment variable IMAGE_UPLOAD_MAX_FILE_SIZE is invalid.
It must be a positive integer.
"""
end
end

if config_env() in [:dev, :test] do
openrouter_api_key = System.get_env("OPENROUTER_API_KEY")

Expand Down
18 changes: 16 additions & 2 deletions docs/features/public-rest-api.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ Request form fields:

| Name | Type | Required | Description |
|---|---|---|---|
| `file` | File | Yes | Image file (`PNG`, `JPG`, `JPEG`, `GIF`, `WEBP`) |
| `file` | File | Yes | Image file (`PNG`, `JPG`, `JPEG`, `GIF`, `WEBP`, `TIF`, `TIFF`) |
| `note` | String | No | User note for the image |

Example:
Expand Down Expand Up @@ -125,21 +125,35 @@ All API errors follow this shape:
Notes:
- `statusCode` is the HTTP status code.
- `statusMessage` is the standard HTTP reason phrase.
- `413` upload errors include the uploaded file or request body size, the app/API limit, and a retry suggestion.

## API Logging

API requests log a request summary and a response summary for `/api/*` routes.

Logged fields include:
- Request: method, path, content type, content length
- Response: method, path, HTTP status, duration, response content type, user id when available

Sensitive request data such as bearer tokens and file contents are not logged.

### Common Errors

| HTTP | Message Example | Meaning |
|---|---|---|
| `400` | `No file provided` / `Invalid image format` | Invalid upload payload or file content |
| `401` | `Invalid or missing API token` | Missing, invalid, disabled, or expired token |
| `413` | `Uploaded image is ...` / `Uploaded request body is ...` | Uploaded file or request body exceeds the API upload limit |
| `404` | `Image not found` | Image not found or not owned by token user |
| `500` | `Failed to create image` | Failed to create image record |
| `500` | `Failed to delete image` | Failed to delete image |

## File Constraints

- Allowed formats: `PNG`, `JPG`, `JPEG`, `GIF`, `WEBP`
- Allowed formats: `PNG`, `JPG`, `JPEG`, `GIF`, `WEBP`, `TIF`, `TIFF`
- Max file size: `50 MB`
- Validation: both file extension and file header (magic bytes) are checked
- TIFF uploads are converted to PNG before storage so image pages and thumbnails work in browsers.
- Files with non-image content (e.g. a `.png` extension on a PDF) are rejected

## Usage Examples
Expand Down
6 changes: 3 additions & 3 deletions docs/features/upload-images.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,18 +2,18 @@

## 概要
用户通过 `UploadForm` 组件上传图片。每次提交生成一个 `upload_batch_id`,同批上传的图片共享该 ID。上传后图片进入 search embedding(Typesense)和 vision embedding(OpenRouter caption)异步处理流程。
系统会保留上传原图到 storage;仅在调用外部 vision 服务前对大图执行一次预处理,以降低请求体积。
系统会保留上传原图到 storage;TIFF 上传会先规范化保存为 PNG,以保证浏览器可展示。仅在调用外部 vision 服务前对大图执行一次预处理,以降低请求体积。

## 架构

### 上传流程
1. 用户选择图片 → LiveView `allow_upload(:images, ...)` 管理暂存
2. 提交时生成 `upload_batch_id`(UUID)
3. 逐个 `consume_uploaded_entry` → `ImageStorage.cp_file` → `Image.create_with_sync`
3. 逐个 `consume_uploaded_entry` → `ImageUpload.store` → `Image.create_with_sync`
4. 创建成功的图片自动触发 Oban jobs:`sync_typesense` + `generate_caption`

### Vision 调用前图片预处理
- 存储策略:`storage/v1/...` 始终保留原图,不写回压缩图
- 存储策略:`storage/v1/...` 保留上传图片;TIFF 会在入库前转换为 PNG,其他格式不写回压缩图
- 调用策略:仅在外部 vision 请求前处理图片,处理结果只用于本次请求。
- 处理规则:
- 小图(< 500KB)跳过预处理。
Expand Down
28 changes: 28 additions & 0 deletions docs/postmortem/2026-06-08-api-tiff-upload-rejected.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
# API TIFF Upload Rejected And Not Web Displayable

## What happened

`POST /api/v1/images` returned `400` for a clipboard upload whose multipart file had `content_type: "image/tiff"` and a `.tiff` filename.

The request was parsed successfully and was under the configured upload size limit, but the API rejected it during image type validation.

After accepting TIFF, uploaded TIFF files were still stored with a `.tiff` extension. Image detail pages then referenced TIFF storage URLs or TIFF thumbnails, which are not reliably displayable in browsers.

## Root cause

The API image allowlist and magic-byte detection only covered PNG, JPEG, GIF, and WEBP. TIFF files from clipboard or screenshot tools were therefore treated as unsupported image uploads.

TIFF is also a poor web display format: most browsers do not support it natively, so keeping TIFF as the stored display asset made image pages fragile.

## Fix applied

- Added TIFF to the API image MIME allowlist.
- Added TIFF magic-byte detection for little-endian and big-endian TIFF headers.
- Converted TIFF uploads to PNG in `ImageUpload.store/3` before copying into storage.
- Updated UI upload accept lists and REST API docs to include TIFF and document PNG normalization.
- Added a file response MIME mapping for stored `.tif` and `.tiff` files.
- Added regression tests for API TIFF upload, TIFF-to-PNG storage, and TIFF file response content type.

## What we learned

Clipboard image uploads can use image formats beyond the formats selected in the UI. API validation should accept the same practical image formats users can produce from common screenshot and clipboard workflows, but storage should normalize non-web-friendly formats into browser-friendly assets.
27 changes: 27 additions & 0 deletions docs/postmortem/2026-06-08-api-upload-too-large-response.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
# API Upload Too Large Response

## What happened

`POST /api/v1/images` rejected multipart uploads above Plug's default parser body limit before the request reached API authentication or `ImageController`.

This made API uploads fail for files that the web UI accepts, and the too-large failure did not use the REST API JSON error shape.

## Root cause

The web UI used `allow_upload(:images, max_file_size: 12_000_000)`, while the Endpoint `Plug.Parsers` configuration did not set an explicit `:length`.

`Plug.Parsers.RequestTooLargeError` is raised at the Endpoint parser layer, before the API controller can return its standard `%{statusCode, statusMessage, message}` response.

## Fix applied

- Added a shared `:image_upload_max_file_size` application config, overridable with `IMAGE_UPLOAD_MAX_FILE_SIZE`.
- Set the multipart parser body limit high enough to carry the shared image limit plus form overhead.
- Added multipart parser handling for oversized upload bodies so `/api/*` returns HTTP `413` with JSON.
- Added controller validation for actual multipart file size so API files over the UI limit return HTTP `413` with JSON.
- Improved `413` messages to include the uploaded size, configured limits, and retry guidance.
- Added API request/response summary logs without logging bearer tokens or file contents.
- Updated REST API documentation with the image file limit and `413` error.

## What we learned

Endpoint parser limits and LiveView upload limits are separate controls. API upload behavior should share the same file-size source of truth as UI uploads, and parser-layer failures need explicit API JSON handling because they happen before route pipelines and controllers.
27 changes: 27 additions & 0 deletions lib/small_sdk/image_magick.ex
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,18 @@ defmodule SmallSdk.ImageMagick do
end
end

def convert_to_png!(input_path, output_path)
when is_binary(input_path) and is_binary(output_path) do
{tool, use_magick_entrypoint?} = pick_tool!()
args = build_convert_to_png_args(use_magick_entrypoint?, input_path, output_path)
{_output, status} = System.cmd(tool, args, stderr_to_stdout: true)

case status do
0 -> :ok
_ -> raise "ImageMagick PNG conversion failed with exit status #{status}."
end
end

defp pick_tool! do
cond do
executable?("magick") -> {"magick", true}
Expand Down Expand Up @@ -111,6 +123,21 @@ defmodule SmallSdk.ImageMagick do
[in_path, "-auto-orient", "-resize", "#{max_side}x#{max_side}>", "-strip", out_path]
end

defp build_convert_to_png_args(true, in_path, out_path) do
["convert" | build_convert_to_png_args(false, in_path, out_path)]
end

defp build_convert_to_png_args(false, in_path, out_path) do
[
"#{in_path}[0]",
"-auto-orient",
"-strip",
"-define",
"png:compression-level=9",
out_path
]
end

defp build_convert_args(in_path, out_path, mime_type, max_side, quality) do
common = [in_path, "-auto-orient", "-resize", "#{max_side}x#{max_side}>", "-strip"]

Expand Down
12 changes: 7 additions & 5 deletions lib/vmemo/memo/image.ex
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ defmodule Vmemo.Memo.Image do
alias Vmemo.Ai.Caption
alias Vmemo.Memo.Changes.SyncImageTags
alias Vmemo.Jobs.Job
alias Vmemo.Memo.ImageUpload
alias Vmemo.Memo.ImageStorage
alias Vmemo.SearchEngine.TsImage

Expand Down Expand Up @@ -386,14 +387,14 @@ defmodule Vmemo.Memo.Image do
storage_file_id = Ash.ActionInput.get_argument(input, :storage_file_id)
user_id = actor.id

case ImageStorage.cp_file(temp_path, user_id, storage_file_id) do
{:ok, dest} ->
case ImageUpload.store(temp_path, user_id, storage_file_id) do
{:ok, %{dest: dest, filename: stored_filename}} ->
case Ash.create(
__MODULE__,
%{
note: "",
url: Path.join("/", dest),
file_id: storage_file_id,
file_id: stored_filename,
user_id: user_id,
inner_purpose: "search"
},
Expand Down Expand Up @@ -1053,9 +1054,10 @@ defmodule Vmemo.Memo.Image do
tmp_path <-
Path.join("tmp/mcp_uploads", "#{System.system_time(:microsecond)}-#{final_filename}"),
:ok <- File.write(tmp_path, binary),
{:ok, dest} <- ImageStorage.cp_file(tmp_path, user_id, final_filename) do
{:ok, %{dest: dest, filename: stored_filename}} <-
ImageUpload.store(tmp_path, user_id, final_filename) do
_ = File.rm(tmp_path)
{:ok, %{url: Path.join("/", dest), file_id: final_filename}}
{:ok, %{url: Path.join("/", dest), file_id: stored_filename}}
else
:error ->
{:error, "Invalid file payload"}
Expand Down
76 changes: 76 additions & 0 deletions lib/vmemo/memo/image_upload.ex
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
defmodule Vmemo.Memo.ImageUpload do
@moduledoc false

alias SmallSdk.ImageMagick
alias Vmemo.Memo.ImageStorage

def store(src, user_id, filename) do
prepared = prepare_for_storage!(src, filename)

try do
with {:ok, dest} <- ImageStorage.cp_file(prepared.path, user_id, prepared.filename) do
{:ok, %{dest: dest, filename: prepared.filename}}
end
after
if prepared.cleanup? do
_ = File.rm(prepared.path)
end
end
rescue
error -> {:error, error}
end

defp prepare_for_storage!(src, filename) do
if tiff_upload?(src, filename) do
png_path = temp_png_path()
ImageMagick.convert_to_png!(src, png_path)

%{path: png_path, filename: png_filename(filename), cleanup?: true}
else
%{path: src, filename: filename, cleanup?: false}
end
end

defp temp_png_path do
Path.join(System.tmp_dir!(), "vmemo-upload-#{System.unique_integer([:positive])}.png")
end

defp png_filename(filename) do
filename
|> to_string()
|> Path.rootname()
|> Kernel.<>(".png")
end

defp tiff_upload?(src, filename) do
tiff_extension?(filename) or tiff_header?(src)
end

defp tiff_extension?(filename) do
filename
|> to_string()
|> Path.extname()
|> String.downcase()
|> then(&(&1 in [".tif", ".tiff"]))
end

defp tiff_header?(path) when is_binary(path) do
case File.open(path, [:read, :binary]) do
{:ok, io} ->
try do
case :file.read(io, 4) do
{:ok, <<"II", marker, 0>>} when marker in [42, 43] -> true
{:ok, <<"MM", 0, marker>>} when marker in [42, 43] -> true
_ -> false
end
after
File.close(io)
end

{:error, _reason} ->
false
end
end

defp tiff_header?(_path), do: false
end
Loading