Advanced Usage

Client Instances

!!! hint If you are coming from Requests, httpx.Client() is what you can use instead of requests.Session().

Why use a Client?

!!! note "TL;DR" If you do anything more than experimentation, one-off scripts, or prototypes, then you should use a Client instance.

More efficient usage of network resources

When you make requests using the top-level API as documented in the Quickstart guide, HTTPX has to establish a new connection for every single request (connections are not reused). As the number of requests to a host increases, this quickly becomes inefficient.

On the other hand, a Client instance uses HTTP connection pooling. This means that when you make several requests to the same host, the Client will reuse the underlying TCP connection, instead of recreating one for every single request.

This can bring significant performance improvements compared to using the top-level API, including:

Reduced latency across requests (no handshaking).
Reduced CPU usage and round-trips.
Reduced network congestion.

Extra features

Client instances also support features that aren't available at the top-level API, such as:

Cookie persistance across requests.
Applying configuration across all outgoing requests.
Sending requests through HTTP proxies.
Using HTTP/2.

The other sections on this page go into further detail about what you can do with a Client instance.

Usage

The recommended way to use a Client is as a context manager. This will ensure that connections are properly cleaned up when leaving the with block:

with httpx.Client() as client:
    ...

Alternatively, you can explicitly close the connection pool without block-usage using .close():

client = httpx.Client()
try:
    ...
finally:
    client.close()

Making requests

Once you have a Client, you can send requests using .get(), .post(), etc. For example:

>>> with httpx.Client() as client:
...     r = client.get('https://example.com')
...
>>> r
<Response [200 OK]>

These methods accept the same arguments as httpx.get(), httpx.post(), etc. This means that all features documented in the Quickstart guide are also available at the client level.

For example, to send a request with custom headers:

>>> with httpx.Client() as client:
...     headers = {'X-Custom': 'value'}
...     r = client.get('https://example.com', headers=headers)
...
>>> r.request.headers['X-Custom']
'value'

Sharing configuration across requests

Clients allow you to apply configuration to all outgoing requests by passing parameters to the Client constructor.

For example, to apply a set of custom headers on every request:

>>> url = 'http://httpbin.org/headers'
>>> headers = {'user-agent': 'my-app/0.0.1'}
>>> with httpx.Client(headers=headers) as client:
...     r = client.get(url)
...
>>> r.json()['headers']['User-Agent']
'my-app/0.0.1'

Merging of configuration

When a configuration option is provided at both the client-level and request-level, one of two things can happen:

For headers, query parameters and cookies, the values are combined together. For example:

>>> headers = {'X-Auth': 'from-client'}
>>> params = {'client_id': 'client1'}
>>> with httpx.Client(headers=headers, params=params) as client:
...     headers = {'X-Custom': 'from-request'}
...     params = {'request_id': 'request1'}
...     r = client.get('https://example.com', headers=headers, params=params)
...
>>> r.request.url
URL('https://example.com?client_id=client1&request_id=request1')
>>> r.request.headers['X-Auth']
'from-client'
>>> r.request.headers['X-Custom']
'from-request'

For all other parameters, the request-level value takes priority. For example:

>>> with httpx.Client(auth=('tom', 'mot123')) as client:
...     r = client.get('https://example.com', auth=('alice', 'ecila123'))
...
>>> _, _, auth = r.request.headers['Authorization'].partition(' ')
>>> import base64
>>> base64.b64decode(auth)
b'alice:ecila123'

If you need finer-grained control on the merging of client-level and request-level parameters, see Request instances.

Other Client-only configuration options

Additionally, Client accepts some configuration options that aren't available at the request level.

For example, base_url allows you to prepend an URL to all outgoing requests:

>>> with httpx.Client(base_url='http://httpbin.org') as client:
...     r = client.get('/headers')
...
>>> r.request.url
URL('http://httpbin.org/headers')

For a list of all available client parameters, see the Client API reference.

Calling into Python Web Apps

You can configure an httpx client to call directly into a Python web application using the WSGI protocol.

This is particularly useful for two main use-cases:

Using httpx as a client inside test cases.
Mocking out external services during tests or in dev/staging environments.

Here's an example of integrating against a Flask application:

from flask import Flask
import httpx


app = Flask(__name__)

@app.route("/")
def hello():
    return "Hello World!"

with httpx.Client(app=app, base_url="http://testserver") as client:
    r = client.get("/")
    assert r.status_code == 200
    assert r.text == "Hello World!"

For some more complex cases you might need to customize the WSGI transport. This allows you to:

Inspect 500 error responses rather than raise exceptions by setting raise_app_exceptions=False.
Mount the WSGI application at a subpath by setting script_name (WSGI).
Use a given client address for requests by setting remote_addr (WSGI).

For example:

# Instantiate a client that makes WSGI requests with a client IP of "1.2.3.4".
transport = httpx.WSGITransport(app=app, remote_addr="1.2.3.4")
with httpx.Client(transport=transport, base_url="http://testserver") as client:
    ...

Request instances

For maximum control on what gets sent over the wire, HTTPX supports building explicit Request instances:

request = httpx.Request("GET", "https://example.com")

To dispatch a Request instance across to the network, create a Client instance and use .send():

with httpx.Client() as client:
    response = client.send(request)
    ...

If you need to mix client-level and request-level options in a way that is not supported by the default Merging of parameters, you can use .build_request() and then make arbitrary modifications to the Request instance. For example:

headers = {"X-Api-Key": "...", "X-Client-ID": "ABC123"}

with httpx.Client(headers=headers) as client:
    request = client.build_request("GET", "https://api.example.com")

    print(request.headers["X-Client-ID"])  # "ABC123"

    # Don't send the API key for this particular request.
    del request.headers["X-Api-Key"]

    response = client.send(request)
    ...

.netrc Support

HTTPX supports .netrc file. In trust_env=True cases, if auth parameter is not defined, HTTPX tries to add auth into request's header from .netrc file.

!!! note The NETRC file is cached across requests made by a client. If you need to refresh the cache (e.g. because the NETRC file has changed), you should create a new client or restart the interpreter.

As default trust_env is true. To set false:

>>> httpx.get('https://example.org/', trust_env=False)

If NETRC environment is empty, HTTPX tries to use default files. (~/.netrc, ~/_netrc)

To change NETRC environment:

>>> import os
>>> os.environ["NETRC"] = "my_default_folder/.my_netrc"

.netrc file content example:

machine netrcexample.org
login example-username
password example-password

...

When using Client instances, trust_env should be set on the client itself, rather that on the request methods:

client = httpx.Client(trust_env=False)

HTTP Proxying

HTTPX supports setting up HTTP proxies the same way that Requests does via the proxies parameter. For example to forward all HTTP traffic to http://127.0.0.1:3080 and all HTTPS traffic to http://127.0.0.1:3081 your proxies config would look like this:

>>> proxies = {
...     "http": "http://127.0.0.1:3080",
...     "https": "http://127.0.0.1:3081"
... }
>>> with httpx.Client(proxies=proxies) as client:
...     ...

Credentials may be passed in as part of the URL in the standard way, i.e. http://username:password@127.0.0.1:3080.

Proxies can be configured for a specific scheme and host, all schemes of a host, all hosts for a scheme, or for all requests. When determining which proxy configuration to use for a given request this same order is used.

>>> proxies = {
...     "http://example.com":  "...",  # Host+Scheme
...     "all://example.com":  "...",  # Host
...     "http": "...",  # Scheme
...     "all": "...",  # All
... }
>>> with httpx.Client(proxies=proxies) as client:
...     ...
...
>>> proxy = "..."  # Shortcut for {'all': '...'}
>>> with httpx.Client(proxies=proxy) as client:
...     ...

!!! warning To make sure that proxies cannot read your traffic, and even if the proxy_url uses HTTPS, it is recommended to use HTTPS and tunnel requests if possible.

By default httpx.Proxy will operate as a forwarding proxy for http://... requests and will establish a CONNECT TCP tunnel for https:// requests. This doesn't change regardless of the proxy url being http or https.

Proxies can be configured to have different behavior such as forwarding or tunneling all requests:

proxy = httpx.Proxy(
    url="https://127.0.0.1",
    mode="TUNNEL_ONLY"  # May be "TUNNEL_ONLY" or "FORWARD_ONLY". Defaults to "DEFAULT".
)
with httpx.Client(proxies=proxy) as client:
    # This request will be tunneled instead of forwarded.
    r = client.get("http://example.com")

!!! note

To use proxies you must pass the proxy information at `Client` initialization,
rather than on the `.get(...)` call or other request methods.

SOCKS proxies are not supported yet.

Timeout Configuration

HTTPX is careful to enforce timeouts everywhere by default.

The default behavior is to raise a TimeoutException after 5 seconds of network inactivity.

Setting and disabling timeouts

You can set timeouts for an individual request:

# Using the top-level API:
httpx.get('http://example.com/api/v1/example', timeout=10.0)

# Using a client instance:
with httpx.Client() as client:
    client.get("http://example.com/api/v1/example", timeout=10.0)

Or disable timeouts for an individual request:

# Using the top-level API:
httpx.get('http://example.com/api/v1/example', timeout=None)

# Using a client instance:
with httpx.Client() as client:
    client.get("http://example.com/api/v1/example", timeout=None)

Setting a default timeout on a client

You can set a timeout on a client instance, which results in the given timeout being used as the default for requests made with this client:

client = httpx.Client()              # Use a default 5s timeout everywhere.
client = httpx.Client(timeout=10.0)  # Use a default 10s timeout everywhere.
client = httpx.Client(timeout=None)  # Disable all timeouts by default.

Fine tuning the configuration

HTTPX also allows you to specify the timeout behavior in more fine grained detail.

There are four different types of timeouts that may occur. These are connect, read, write, and pool timeouts.

The connect timeout specifies the maximum amount of time to wait until a connection to the requested host is established. If HTTPX is unable to connect within this time frame, a ConnectTimeout exception is raised.
The read timeout specifies the maximum duration to wait for a chunk of data to be received (for example, a chunk of the response body). If HTTPX is unable to receive data within this time frame, a ReadTimeout exception is raised.
The write timeout specifies the maximum duration to wait for a chunk of data to be sent (for example, a chunk of the request body). If HTTPX is unable to send data within this time frame, a WriteTimeout exception is raised.
The pool timeout specifies the maximum duration to wait for acquiring a connection from the connection pool. If HTTPX is unable to acquire a connection within this time frame, a PoolTimeout exception is raised. A related configuration here is the maximum number of allowable connections in the connection pool, which is configured by the pool_limits.

You can configure the timeout behavior for any of these values...

# A client with a 60s timeout for connecting, and a 10s timeout elsewhere.
timeout = httpx.Timeout(10.0, connect=60.0)
client = httpx.Client(timeout=timeout)

response = client.get('http://example.com/')

Pool limit configuration

You can control the connection pool size using the limits keyword argument on the client. It takes instances of httpx.Limits which define:

max_keepalive, number of allowable keep-alive connections, or None to always allow. (Defaults 10)
max_connections, maximum number of allowable connections, or None for no limits. (Default 100)

limits = httpx.Limits(max_keepalive=5, max_connections=10)
client = httpx.Client(limits=limits)

Multipart file encoding

As mentioned in the quickstart multipart file encoding is available by passing a dictionary with the name of the payloads as keys and either tuple of elements or a file-like object or a string as values.

>>> files = {'upload-file': ('report.xls', open('report.xls', 'rb'), 'application/vnd.ms-excel')}
>>> r = httpx.post("https://httpbin.org/post", files=files)
>>> print(r.text)
{
  ...
  "files": {
    "upload-file": "<... binary content ...>"
  },
  ...
}

More specifically, if a tuple is used as a value, it must have between 2 and 3 elements:

The first element is an optional file name which can be set to None.
The second element may be a file-like object or a string which will be automatically encoded in UTF-8.
An optional third element can be used to specify the MIME type of the file being uploaded. If not specified HTTPX will attempt to guess the MIME type based on the file name, with unknown file extensions defaulting to "application/octet-stream". If the file name is explicitly set to None then HTTPX will not include a content-type MIME header field.

>>> files = {'upload-file': (None, 'text content', 'text/plain')}
>>> r = httpx.post("https://httpbin.org/post", files=files)
>>> print(r.text)
{
  ...
  "files": {},
  "form": {
    "upload-file": "text-content"
  },
  ...
}

!!! tip It is safe to upload large files this way. File uploads are streaming by default, meaning that only one chunk will be loaded into memory at a time.

Non-file data fields can be included in the multipart form using by passing them to data=....

You can also send multiple files in one go with a multiple file field form. To do that, pass a list of (field, <file>) items instead of a dictionary, allowing you to pass multiple items with the same field. For instance this request sends 2 files, foo.png and bar.png in one request on the images form field:

>>> files = [('images', ('foo.png', open('foo.png', 'rb'), 'image/png')),
                      ('images', ('bar.png', open('bar.png', 'rb'), 'image/png'))]
>>> r = httpx.post("https://httpbin.org/post", files=files)

Customizing authentication

When issuing requests or instantiating a client, the auth argument can be used to pass an authentication scheme to use. The auth argument may be one of the following...

A two-tuple of username/password, to be used with basic authentication.
An instance of httpx.BasicAuth() or httpx.DigestAuth().
A callable, accepting a request and returning an authenticated request instance.
A subclass of httpx.Auth.

The most involved of these is the last, which allows you to create authentication flows involving one or more requests. A subclass of httpx.Auth should implement def auth_flow(request), and yield any requests that need to be made...

class MyCustomAuth(httpx.Auth):
    def __init__(self, token):
        self.token = token

    def auth_flow(self, request):
        # Send the request, with a custom `X-Authentication` header.
        request.headers['X-Authentication'] = self.token
        yield request

If the auth flow requires more that one request, you can issue multiple yields, and obtain the response in each case...

class MyCustomAuth(httpx.Auth):
    def __init__(self, token):
        self.token = token

    def auth_flow(self, request):
      response = yield request
      if response.status_code == 401:
          # If the server issues a 401 response then resend the request,
          # with a custom `X-Authentication` header.
          request.headers['X-Authentication'] = self.token
          yield request

Custom authentication classes are designed to not perform any I/O, so that they may be used with both sync and async client instances. If you are implementing an authentication scheme that requires the request body, then you need to indicate this on the class using a requires_request_body property.

You will then be able to access request.content inside the .auth_flow() method.

class MyCustomAuth(httpx.Auth):
    requires_request_body = True

    def __init__(self, token):
        self.token = token

    def auth_flow(self, request):
      response = yield request
      if response.status_code == 401:
          # If the server issues a 401 response then resend the request,
          # with a custom `X-Authentication` header.
          request.headers['X-Authentication'] = self.sign_request(...)
          yield request

    def sign_request(self, request):
        # Create a request signature, based on `request.method`, `request.url`,
        # `request.headers`, and `request.content`.
        ...

Similarly, if you are implementing a scheme that requires access to the response body, then use the requires_response_body property. You will then be able to access response body properties and methods such as response.content, response.text, response.json(), etc.

class MyCustomAuth(httpx.Auth):
    requires_response_body = True

    def __init__(self, access_token, refresh_token, refresh_url):
        self.access_token = access_token
        self.refresh_token = refresh_token
        self.refresh_url = refresh_url

    def auth_flow(self, request):
        request.headers["X-Authentication"] = self.access_token
        response = yield request

        if response.status_code == 401:
            # If the server issues a 401 response, then issue a request to
            # refresh tokens, and resend the request.
            refresh_response = yield self.build_refresh_request()
            self.update_tokens(refresh_response)

            request.headers["X-Authentication"] = self.access_token
            yield request

    def build_refresh_request(self):
        # Return an `httpx.Request` for refreshing tokens.
        ...

    def update_tokens(self, response):
        # Update the `.access_token` and `.refresh_token` tokens
        # based on a refresh response.
        data = response.json()
        ...

SSL certificates

When making a request over HTTPS, HTTPX needs to verify the identity of the requested host. To do this, it uses a bundle of SSL certificates (a.k.a. CA bundle) delivered by a trusted certificate authority (CA).

Changing the verification defaults

By default, HTTPX uses the CA bundle provided by Certifi. This is what you want in most cases, even though some advanced situations may require you to use a different set of certificates.

If you'd like to use a custom CA bundle, you can use the verify parameter.

import httpx

r = httpx.get("https://example.org", verify="path/to/client.pem")

Alternatively, you can pass a standard library ssl.SSLContext.

>>> import ssl
>>> import httpx
>>> context = ssl.create_default_context()
>>> context.load_verify_locations(cafile="/tmp/client.pem")
>>> httpx.get('https://example.org', verify=context)
<Response [200 OK]>

We also include a helper function for creating properly configured SSLContext instances.

>>> context = httpx.create_ssl_context()

The create_ssl_context function accepts the same set of SSL configuration arguments (trust_env, verify, cert and http2 arguments) as httpx.Client or httpx.AsyncClient

>>> import httpx
>>> context = httpx.create_ssl_context(verify="/tmp/client.pem")
>>> httpx.get('https://example.org', verify=context)
<Response [200 OK]>

Or you can also disable the SSL verification entirely, which is not recommended.

import httpx

r = httpx.get("https://example.org", verify=False)

SSL configuration on client instances

If you're using a Client() instance, then you should pass any SSL settings when instantiating the client.

client = httpx.Client(verify=False)

The client.get(...) method and other request methods do not support changing the SSL settings on a per-request basis. If you need different SSL settings in different cases you should use more that one client instance, with different settings on each. Each client will then be using an isolated connection pool with a specific fixed SSL configuration on all connections within that pool.

Making HTTPS requests to a local server

When making requests to local servers, such as a development server running on localhost, you will typically be using unencrypted HTTP connections.

If you do need to make HTTPS connections to a local server, for example to test an HTTPS-only service, you will need to create and use your own certificates. Here's one way to do it:

Use trustme-cli to generate a pair of server key/cert files, and a client cert file.
Pass the server key/cert files when starting your local server. (This depends on the particular web server you're using. For example, Uvicorn provides the --ssl-keyfile and --ssl-certfile options.)
Tell HTTPX to use the certificates stored in client.pem:

>>> import httpx
>>> r = httpx.get("https://localhost:8000", verify="/tmp/client.pem")
>>> r
Response <200 OK>

Custom Transports

HTTPX's Client also accepts a transport argument. This argument allows you to provide a custom Transport object that will be used to perform the actual sending of the requests.

A transport instance must implement the Transport API defined by httpcore. You should either subclass httpcore.AsyncHTTPTransport to implement a transport to use with AsyncClient, or subclass httpcore.SyncHTTPTransport to implement a transport to use with Client.

For example, HTTPX ships with a transport that uses the excellent urllib3 library, which can be used with the sync Client...

>>> import httpx
>>> client = httpx.Client(transport=httpx.URLLib3Transport())
>>> client.get("https://example.org")
<Response [200 OK]>

Note that you'll need to install the urllib3 package to use URLLib3Transport.

A complete example of a custom transport implementation would be:

import json

import httpcore
import httpx


class JSONEchoTransport(httpcore.SyncHTTPTransport):
    """
    A mock transport that returns a JSON response containing the request body.
    """

    def request(self, method, url, headers=None, stream=None, timeout=None):
        body = b"".join(stream).decode("utf-8")
        content = json.dumps({"body": body}).encode("utf-8")
        stream = httpcore.SyncByteStream([content])
        headers = [(b"content-type", b"application/json")]
        return b"HTTP/1.1", 200, b"OK", headers, stream

Which we can use in the same way:

>>> client = httpx.Client(transport=JSONEchoTransport())
>>> response = client.post("https://httpbin.org/post", data="Hello, world!")
>>> response.json()
{'body': 'Hello, world!'}

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Advanced Usage

Client Instances

Why use a Client?

More efficient usage of network resources

Extra features

Usage

Making requests

Sharing configuration across requests

Merging of configuration

Other Client-only configuration options

Calling into Python Web Apps

Request instances

.netrc Support

HTTP Proxying

Timeout Configuration

Setting and disabling timeouts

Setting a default timeout on a client

Fine tuning the configuration

Pool limit configuration

Multipart file encoding

Customizing authentication

SSL certificates

Changing the verification defaults

SSL configuration on client instances

Making HTTPS requests to a local server

Custom Transports

Uh oh!

FilesExpand file tree

advanced.md

Latest commit

History

advanced.md

File metadata and controls

Advanced Usage

Client Instances

Why use a Client?

More efficient usage of network resources

Extra features

Usage

Making requests

Sharing configuration across requests

Merging of configuration

Other Client-only configuration options

Calling into Python Web Apps

Request instances

.netrc Support

HTTP Proxying

Timeout Configuration

Setting and disabling timeouts

Setting a default timeout on a client

Fine tuning the configuration

Pool limit configuration

Multipart file encoding

Customizing authentication

SSL certificates

Changing the verification defaults

SSL configuration on client instances

Making HTTPS requests to a local server

Custom Transports