Possible memory leak in the CookieJar

[div1001]

Describe the bug

Memory usage increases indefinitely when making requests with aiohttp>=3.10.0 ClientSession to a server that often sets cookies.

To Reproduce

import asyncio 

async def do_test():
    from aiohttp import ClientSession
    from pprint import pprint

    async with ClientSession() as cs:
        for i in range(10):
            async with cs.get(f'http://httpbin.org/cookies/set/test/val{i}', allow_redirects=False) as r:
                print(f'{len(cs.cookie_jar._cookies)=}', end=' ')
                print(f'{r.headers["set-cookie"]=}')

        print('='*40)
        pprint(dict(cs.cookie_jar._cookies))

if __name__ == '__main__':
    asyncio.run(do_test())

Expected behavior

Eventual decrease in memory usage.

Logs/tracebacks

len(cs.cookie_jar._cookies)=1 r.headers["set-cookie"]='test=val0; Path=/'
len(cs.cookie_jar._cookies)=11 r.headers["set-cookie"]='test=val1; Path=/'
len(cs.cookie_jar._cookies)=13 r.headers["set-cookie"]='test=val2; Path=/'
len(cs.cookie_jar._cookies)=15 r.headers["set-cookie"]='test=val3; Path=/'
len(cs.cookie_jar._cookies)=17 r.headers["set-cookie"]='test=val4; Path=/'
len(cs.cookie_jar._cookies)=19 r.headers["set-cookie"]='test=val5; Path=/'
len(cs.cookie_jar._cookies)=21 r.headers["set-cookie"]='test=val6; Path=/'
len(cs.cookie_jar._cookies)=23 r.headers["set-cookie"]='test=val7; Path=/'
len(cs.cookie_jar._cookies)=25 r.headers["set-cookie"]='test=val8; Path=/'
len(cs.cookie_jar._cookies)=27 r.headers["set-cookie"]='test=val9; Path=/'
========================================
{('', ''): <SimpleCookie: >,
 ('httpbin.org', ''): <SimpleCookie: test='val9'>,
 ('httpbin.org', '/cookies'): <SimpleCookie: >,
 ('httpbin.org', '/cookies/set'): <SimpleCookie: >,
 ('httpbin.org', '/cookies/set/test'): <SimpleCookie: >,
 ('httpbin.org', '/cookies/set/test/val1'): <SimpleCookie: >,
 ('httpbin.org', '/cookies/set/test/val2'): <SimpleCookie: >,
 ('httpbin.org', '/cookies/set/test/val3'): <SimpleCookie: >,
 ('httpbin.org', '/cookies/set/test/val4'): <SimpleCookie: >,
 ('httpbin.org', '/cookies/set/test/val5'): <SimpleCookie: >,
 ('httpbin.org', '/cookies/set/test/val6'): <SimpleCookie: >,
 ('httpbin.org', '/cookies/set/test/val7'): <SimpleCookie: >,
 ('httpbin.org', '/cookies/set/test/val8'): <SimpleCookie: >,
 ('httpbin.org', '/cookies/set/test/val9'): <SimpleCookie: >,
 ('org', ''): <SimpleCookie: >,
 ('org', '/cookies'): <SimpleCookie: >,
 ('org', '/cookies/set'): <SimpleCookie: >,
 ('org', '/cookies/set/test'): <SimpleCookie: >,
 ('org', '/cookies/set/test/val1'): <SimpleCookie: >,
 ('org', '/cookies/set/test/val2'): <SimpleCookie: >,
 ('org', '/cookies/set/test/val3'): <SimpleCookie: >,
 ('org', '/cookies/set/test/val4'): <SimpleCookie: >,
 ('org', '/cookies/set/test/val5'): <SimpleCookie: >,
 ('org', '/cookies/set/test/val6'): <SimpleCookie: >,
 ('org', '/cookies/set/test/val7'): <SimpleCookie: >,
 ('org', '/cookies/set/test/val8'): <SimpleCookie: >,
 ('org', '/cookies/set/test/val9'): <SimpleCookie: >}

Python Version

3.8.16

aiohttp Version

3.10.11

multidict Version

6.1.0

propcache Version

0.2.0

yarl Version

1.15.2

OS

Linux

Related component

Client

Additional context

I use a single session for the app's lifetime, with requests being sent to the single host.
In the span of weeks memory usage increases from ~100Mb to ~5Gb, never decreasing; aiohttp<=3.9.5 does not show such behavior.
In the test script output notice increasing len(cs.cookie_jar._cookies), for some reason despite the server setting a cookie with Path=/ CookieJar creates records for every possible path prefix of every request made, and it seems those never get cleaned.

I'm not sure if this is a bug or necesary behaviuor though.

Code of Conduct

• I agree to follow the aio-libs Code of Conduct

[Dreamsorcerer]

Hmm, we started tracking the paths for cookies properly in 3.9: #6638

Though, as you say, the Path appears to be /, so it looks like it's saving under the wrong path...

It would be great if you could make a PR with a regression test. (If you need a server to reproduce, try copying a test from test_client_functional.py to use as a basis: https://github.com/aio-libs/aiohttp/blob/master/tests/test_client_functional.py#L81).

[Cycloctane]

filter_cookies() is creating empty entries when enumerating domain-path combinations in self._cookie defaultdict.

aiohttp/aiohttp/cookiejar.py

Lines 352 to 358 in c50f332

	pairs = itertools.product(domains, paths)
	path_len = len(request_url.path)
	# Point 2: https://www.rfc-editor.org/rfc/rfc6265.html#section-5.4
	for p in pairs:
	for name, cookie in self._cookies[p].items():
	domain = cookie["domain"]

[bdraco]

Good catch!

[bdraco]

reproducer

"""Test to reproduce the cookie jar memory leak issue #11052"""

import aiohttp
from yarl import URL


def test_cookie_jar_leak():
    """Demonstrate the memory leak in CookieJar.filter_cookies()"""
    jar = aiohttp.CookieJar()
    
    # Set a cookie with Path=/
    jar.update_cookies(
        {"test_cookie": "value; Path=/"},
        URL("http://example.com/")
    )
    
    print(f"Initial cookie count: {len(jar)}")
    print(f"Initial internal storage size: {len(jar._cookies)}")
    
    # Make multiple requests with different paths
    paths = [
        "/",
        "/api",
        "/api/v1", 
        "/api/v1/users",
        "/api/v1/users/123",
        "/api/v1/users/123/profile",
        "/static/css/style.css",
        "/static/js/app.js",
        "/images/logo.png"
    ]
    
    for i, path in enumerate(paths):
        url = URL(f"http://example.com{path}")
        filtered = jar.filter_cookies(url)
        print(f"\nRequest {i+1} to {path}:")
        print(f"  Filtered cookies: {len(filtered)}")
        print(f"  Internal storage size: {len(jar._cookies)}")
    
    print("\n--- Storage details ---")
    for key, cookie in jar._cookies.items():
        print(f"  {key}: {len(cookie)} cookies")


if __name__ == "__main__":
    test_cookie_jar_leak()