URL-encoded bodies are not readable - request errors during processing

it seems setting the body to a hash for url_encoded requests results in a String body in the async faraday adapter where it is expecting a Readable?

49.64s     warn: Async::Task: Writing #<Async::HTTP::Faraday::BodyWrapper:0x0000ffffa59bc440> to #<Async::HTTP::Protocol::HTTP2::Response::Stream:0x0000ffffa758a1e0>. [oid=0x23a0] [ec=0x23b4] [pid=149] [2024-02-07 20:54:59 +0000]
               | Task may have ended with unhandled exception.
               |   NoMethodError: undefined method `read' for "foo=42":String
               |   → /usr/local/bundle/gems/async-http-faraday-0.13.0/lib/async/http/faraday/adapter.rb:34 in `read'
               |     /usr/local/bundle/gems/async-http-0.63.0/lib/async/http/protocol/http2/output.rb:83 in `passthrough'
               |     /usr/local/bundle/gems/async-2.8.1/lib/async/task.rb:161 in `block in run'
               |     /usr/local/bundle/gems/async-2.8.1/lib/async/task.rb:331 in `block in schedule'

code to repro:

task = Async do
  conn = Faraday.new do |c|
    c.request :url_encoded
    c.adapter :async_http
  end
  conn.post("https://httpbin.org/post") do |req|
    req.body = {foo: 42}
  end
ensure
  conn&.close
end
task.wait

@iMacTia

@iMacTia is there documentation about what valid body can be and what to do with it i.e. how to get a consistent readable interface?

work-around for now:

task = Async do
  conn = Faraday.new do |c|
    c.adapter :async_http
  end
  conn.post("https://httpbin.org/post") do |req|
    req.body = StringIO.new({foo: 42}.to_query)
    req.headers[Faraday::Request::UrlEncoded::CONTENT_TYPE] ||= Faraday::Request::UrlEncoded.mime_type
  end
ensure
  conn&.close
end
task.wait

I think I found the solution, I'll try to make a release within a few hours including the fix.

also seeing this with e.g.

task = Async do
  conn = Faraday.new do |c|
    c.request :json
    c.adapter :async_http
  end
  conn.post("https://httpbin.org/post") do |req|
    req.body = {foo: 42}
  end
ensure
  conn&.close
end
task.wait

fwiw. almost certainly has the same fix, though.

Do you mind trying out v0.13.1 which was just released. It should fix the issue.

v0.13.1 fixes the issue 🥳 thanks!

@ioquatix

Thank you for the quick fix @ioquatix 🙌

I'm surprised this never happened before because to my knowledge it's fairly common for Faraday to pass a string as the request body for post/put/patch methods 🤔

Is there documentation about what valid body can be and what to do with it
Not really, because what the adapter expects can vary wildly. As I said though, to my knowledge, String is almost always what is given to the adapter (after middleware takes whatever the input is, like a Hash and turns that into a string).

It seems like it's at least either:

A String instance, or
An object that responds to #read.

mmmh it seems like Net::HTTP supports #read as well, but you need to pass it explicitly as body_stream.

To be fair, Faraday have just recently added first-class support for RESPONSE streaming, this looks more like we want to support request body streaming. Might be a good addition to the API.

Adapters like async-http will support it transparently, while others like net-http would require some extra code

For sure it seems needed for multipart post - because that is a rich object which responds to #read so I'm not sure how other clients are handling it (it was simply assigned to #body).

Actually yes, Net::HTTP does support streams, but you also have to set Content-Length in that case or you'll get the following error back:

`send_request_with_body_stream': Content-Length not given and Transfer-Encoding is not `chunked' (ArgumentError)

And this is exactly what faraday-multipart does: https://github.com/lostisland/faraday-multipart/blob/main/lib/faraday/multipart/middleware.rb#L63

So yeah, it seems like adapters should support both. Documentation should definitely point this out more explicitly

Opened lostisland/faraday#1554, thank you so much for the discussion 🙌 !

Navigation Menu

Repository navigation

URL-encoded bodies are not readable - request errors during processing #36

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

URL-encoded bodies are not readable - request errors during processing #36

Description

Activity

ioquatix commented on Feb 7, 2024

jakeonfire commented on Feb 7, 2024

ioquatix commented on Feb 7, 2024

jakeonfire commented on Feb 8, 2024

ioquatix commented on Feb 8, 2024

jakeonfire commented on Feb 8, 2024

iMacTia commented on Feb 8, 2024

ioquatix commented on Feb 8, 2024

iMacTia commented on Feb 8, 2024

ioquatix commented on Feb 8, 2024

iMacTia commented on Feb 8, 2024

iMacTia commented on Feb 8, 2024

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions