MultipartForm + HTTP2 causes very slow file uploads

[JMLX42]

Your issue may already be reported! Please search on the Actix Web issue tracker before creating one.

Expected Behavior

Sub second latency between the logger and the actual entry in the endpoint handler.

Current Behavior

There is a multi-seconds delay between the tracing span creation and actually entering in the endpoint handler.

Possible Solution

IDK

Update: use HTTP 1.1 (cf #3638 (comment)).

Steps to Reproduce (for bugs)

1. Add the TracingLogger to the actix-web App
2. Upload a file using MultipartForm

Context

let app = App::new()
    .wrap(middleware::NormalizePath::new(TrailingSlash::MergeOnly))
    .wrap(tracing_actix_web::TracingLogger::default());

#[derive(ToSchema, MultipartForm)]
pub(crate) struct GlbFileUpload {
    /// The binary glTF (`.glb`) file to import.
    #[schema(format = Binary, content_encoding = "binary", content_media_type = "model/gltf-binary", value_type = String)]
    file: TempFile,
}

#[post("/gltf/import")]
pub async fn import_gltf_file(
    req: HttpRequest,
    db: web::Data<Database>,
    query: web::Query<GltfResourceQueryParameters>,
    MultipartForm(payload): MultipartForm<GlbFileUpload>,
) -> Result<HttpResponse<actix_web::body::BoxBody>, ErrorList> {
    info!(
        "received {:?} ({} bytes) for import",
        payload.file.file_name, payload.file.size,
    );

curl -X 'POST' \
  'https://127.0.0.1:4242/api/gltf/import' \
  -H 'accept: application/vnd.api+json' \
  -H 'Content-Type: multipart/form-data' \
  -F 'file=@/home/jmlx/Downloads/J35Panel7712.vue.scene.glb;type=model/gltf-binary'

2025-05-07T17:36:01.270852Z  INFO HTTP request{http.method=POST http.route=/api/gltf/{gltf_id} http.flavor=2.0 http.scheme=https http.host=127.0.0.1:4242 http.client_ip=127.0.0.1 http.user_agent=curl/8.5.0 http.target=/api/gltf/import otel.name=POST /api/gltf/{gltf_id} otel.kind="server" request_id=0196abd1-94f6-7170-9b4e-003c97b7644d}: tracing_actix_web::root_span_builder: new
2025-05-07T17:36:09.987783Z  INFO HTTP request{http.method=POST http.route=/api/gltf/{gltf_id} http.flavor=2.0 http.scheme=https http.host=127.0.0.1:4242 http.client_ip=127.0.0.1 http.user_agent=curl/8.5.0 http.target=/api/gltf/import otel.name=POST /api/gltf/{gltf_id} otel.kind="server" request_id=0196abd1-94f6-7170-9b4e-003c97b7644d}: gltf_live_api::api::gltf: received Some("J35Panel7712.vue.scene.glb") (30760164 bytes) for import

In this case, 8 seconds for a ~30MB file upload.

Your Environment

• Rust 1.86.0
• Actix Web 4.9.0
• Actix Multipart 0.7.2
• Storage : Samsung SSD 980 PRO NVME
• OS : Ubuntu 24.04.1

[JMLX42]

As far as I can tell, the problem is the same (but harder to quantify) without the TracingLogger.

[JMLX42]

Using curl -vv, the "We are completely uploaded and fine" line appears after an unexpectedly long amount of time. And then the endpoint is entered.

So maybe it's something on my end...

[JMLX42]

Using curl -vv, the "We are completely uploaded and fine" line appears after an unexpectedly long amount of time. And then the endpoint is entered.

Using HTTP 1.1 the problem goes away and the "We are completely uploaded and fine" curl log line appears immediately. I'll update the description accordingly.

2025-05-07T17:57:31.928518Z  INFO HTTP request{http.method=POST http.route=/api/gltf/{gltf_id} http.flavor=1.1 http.scheme=https http.host=127.0.0.1:4242 http.client_ip=127.0.0.1 http.user_agent=curl/8.5.0 http.target=/api/gltf/import otel.name=POST /api/gltf/{gltf_id} otel.kind="server" request_id=0196abe5-4698-7b72-b139-66a0391d6f07}: tracing_actix_web::root_span_builder: new
2025-05-07T17:57:31.952163Z  INFO HTTP request{http.method=POST http.route=/api/gltf/{gltf_id} http.flavor=1.1 http.scheme=https http.host=127.0.0.1:4242 http.client_ip=127.0.0.1 http.user_agent=curl/8.5.0 http.target=/api/gltf/import otel.name=POST /api/gltf/{gltf_id} otel.kind="server" request_id=0196abe5-4698-7b72-b139-66a0391d6f07}: gltf_live_api::api::gltf: received Some("J35Panel7712.vue.scene.glb") (30760164 bytes) for import

2025-05-07T17:58:41.395419Z  INFO HTTP request{http.method=POST http.route=/api/gltf/{gltf_id} http.flavor=2.0 http.scheme=https http.host=127.0.0.1:4242 http.client_ip=127.0.0.1 http.user_agent=curl/8.5.0 http.target=/api/gltf/import otel.name=POST /api/gltf/{gltf_id} otel.kind="server" request_id=0196abe6-55f3-7563-8163-085ccc858105}: tracing_actix_web::root_span_builder: new
2025-05-07T17:58:49.089988Z  INFO HTTP request{http.method=POST http.route=/api/gltf/{gltf_id} http.flavor=2.0 http.scheme=https http.host=127.0.0.1:4242 http.client_ip=127.0.0.1 http.user_agent=curl/8.5.0 http.target=/api/gltf/import otel.name=POST /api/gltf/{gltf_id} otel.kind="server" request_id=0196abe6-55f3-7563-8163-085ccc858105}: gltf_live_api::api::gltf: received Some("J35Panel7712.vue.scene.glb") (30760164 bytes) for impor

• HTTP 1.1: 0.03 second delay
• HTTP 2: 8 seconds delay

[JMLX42]

Based on this CloudFlare blog post this issue might be indeed specific to HTTP/2. And it might boil down to buffer size.

Alas, AFAIK, there is no way to configure the multipart buffer size in actix-multipart. I'll see if I can open a PR.

I will most likely end up using my actix-web service behind a reverse proxy. So it will receive HTTP 1.1. I will list that as a workaround in the description.

[JMLX42]

Alas, AFAIK, there is no way to configure the multipart buffer size in actix-multipart. I'll see if I can open a PR.

I think the payload buffer size is set to 1KB:

actix-web/actix-multipart/src/payload.rs

Line 61 in e27e1e3

buf: BytesMut::with_capacity(1_024), // pre-allocate 1KiB

If that is the case, then it might be the problem. The CloudFlare blog post shows that the performance is comparable or even better than HTTP 1.1 with a 512kB buffer.

[thalesfragoso]

If your problem is with HTTP/2 then you would want to tune at a lower level, namely:
https://docs.rs/h2/0.4.12/h2/server/struct.Builder.html#method.initial_connection_window_size and
https://docs.rs/h2/0.4.12/h2/server/struct.Builder.html#method.initial_window_size

This isn't currently exposed though, so you would need to change actix-http for that.