Multipart post causes error in protocol-http #32

Activity

1. jakeonfire commented on Feb 7, 2024

   

   jakeonfire

   on Feb 7, 2024

   Last edited by jakeonfire

   ContributorAuthor

   More actions

   verified this issue exists with faraday (1.10.3) as well, in which case you can use:

   conn = Faraday.new do |c|
     c.request :multipart
     c.adapter :async_http
   end
   task = Async do
     conn.post("https://httpbin.org/post") do |req|
       req.body = { "file" => Faraday::Multipart::FilePart.new(StringIO.new("file content"), "text/plain", "file.txt") }
     end
   end
   task.wait
2. jakeonfire commented on Feb 7, 2024

   

   jakeonfire

   on Feb 7, 2024

   Last edited by jakeonfire

   ContributorAuthor

   More actions

   separately, my app memory is now running amuck. i'm using falcon and async-http-faraday (the app is basically a proxy to 3rd party REST APIs), and it looks like this combination is causing a memory leak. is using both gems redundant vs just using falcon? or shouldasync-http-faraday provide extra efficiencies even when running on a falcon server?

   

   you can see me using fewer and fewer falcon processes, where it was fine with 6 before the introduction of async-http-faraday.
3. ioquatix commented on Feb 7, 2024

   

   ioquatix

   on Feb 7, 2024

   Member

   More actions

   That does look like a memory leak. Are you able to share more details of the app? I have a memory profiling tool, would you be willing to capture a dump?
4. jakeonfire commented on Feb 7, 2024

   

   jakeonfire

   on Feb 7, 2024

   Last edited by jakeonfire

   ContributorAuthor

   More actions

   more details, hmm... the app (rails 7.1) accepts requests from a client (another app of ours), proxies them (in the loosest sense) to various 3rd party APIs (on a couple dozen different hosts), then formats the response back to the client. we are using async-http-faraday to make those calls to 3rd party APIs. there is no manual parallelization within a request, we are just trying to support more connections from the client with less hardware, seeing as there is a lot of IO wait time when connecting to the 3rd party APIs.

   i would be willing to capture a dump and private-gist it to you.

   also sorry to hijack my own thread, but just want to make sure you saw my first two messages were about a separate (non-memory) issue.
5. jakeonfire commented on Feb 7, 2024

   

   jakeonfire

   on Feb 7, 2024

   Last edited by jakeonfire

   ContributorAuthor

   More actions

   imagine a vanilla rails app, where controller actions might look like:

   class SomeController < ApplicationController
     def update
       response = request_handler.do_post("https://microsoft.com/update_thing", params[:post_body])
       render json: response.body
     end
   
     def request_handler
       @_request_handler ||= RequestHandler.new
     end
   end

   and RequestHandler something like:

   class RequestHandler
     def do_post(url, body)
       task = Async do
         conn.post(url) do |req|
           req.body = body
         end
       end
       response = task.wait
     end
   
     def conn
       Faraday.new do |c|
         c.adapter :async_http
       end
     end
   end

   the app is using falcon host or falcon virtual with the rack configuration, a preload of the rails environment, and endpoint Async::HTTP::Endpoint.parse("http://0.0.0.0:3000"). pretty simple.
6. jakeonfire commented on Feb 7, 2024

   

   jakeonfire

   on Feb 7, 2024

   ContributorAuthor

   More actions

   and fwiw i've tried with both falcon + (non-async) faraday, as well as unicorn (don't ask) + async-http-faraday and both configurations have no memory issues.
7. ioquatix commented on Feb 7, 2024

   

   ioquatix

   on Feb 7, 2024

   Member

   More actions

   I can reproduce the original issue, so let me fix that to start with.
8. ioquatix

   closed this as completedin multipart-post#107on Feb 7, 2024
9. ioquatix commented on Feb 7, 2024

   

   ioquatix

   on Feb 7, 2024

   Member

   More actions

   This is fixed in v0.13.0 of async-http-faraday, along with v2.4.0 of multipart-post.

   Can you try it out and let me know?
10. jakeonfire commented on Feb 7, 2024

    

    jakeonfire

    on Feb 7, 2024

    ContributorAuthor

    More actions

    Can you try it out and let me know?

    yep, it works 🥳 i didn't realize you also work on multipart-post, how fortunate!

    👍React with 👍1Reacted by Samuel Williams

    😄React with 😄1Reacted by Samuel Williams