Skip to content

Store cached cookie as JSON instead of pickle - #3030

Open
JonathaCosta10 wants to merge 1 commit into
ranaroussi:devfrom
JonathaCosta10:cookie-cache-json
Open

JonathaCosta10 wants to merge 1 commit into
ranaroussi:devfrom
JonathaCosta10:cookie-cache-json

Conversation

@JonathaCosta10

Copy link
Copy Markdown

The cookie cache (_CookieCache in yfinance/cache.py) stores the Yahoo cookie in the local SQLite DB with pickle.dumps and reads it back with pickle.loads. Unpickling executes code, so anything that can write to the cache directory could get code run in every process that later uses yfinance. This is hardening rather than a remote issue (it needs prior write access to the cache dir), but there is no need for pickle here: the cached value is just a dict of http.cookiejar.Cookie objects.

Change

  • store() serialises {domain: {path: {name: Cookie}}} to JSON (all public Cookie attributes plus _rest).
  • lookup() rebuilds the Cookie objects from JSON.
  • Entries that are not valid JSON, such as ones written by older versions with pickle, are treated as a cache miss (None), so a fresh cookie is fetched and the entry is overwritten. Old entries are never unpickled.
  • No schema change: the same cookie_bytes blob column holds the JSON.

data.py is unchanged; _load_cookie_curlCffi still gets the same nested dict of Cookie objects, including .expires.

Testing

Locally: a round trip of a .yahoo.com / A3 cookie preserves every attribute and _rest, and a pickled entry is rejected by the loader instead of being deserialised. I haven't run the full test suite; happy to add a unit test to tests/test_cache.py if you'd like one.

@ValueRaider

ValueRaider commented Oct 31, 2026 •

Copy link
Copy Markdown
Collaborator

Can there be a migration function, to convert a pickled cookie to json? It is useful to preserve it.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants