Repository navigation
Store cached cookie as JSON instead of pickle - #3030
Open
JonathaCosta10 wants to merge 1 commit into
Open
JonathaCosta10 wants to merge 1 commit into
JonathaCosta10 wants to merge 1 commit into
Conversation
Collaborator
|
Can there be a migration function, to convert a pickled cookie to json? It is useful to preserve it. |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The cookie cache (
_CookieCacheinyfinance/cache.py) stores the Yahoo cookie in the local SQLite DB withpickle.dumpsand reads it back withpickle.loads. Unpickling executes code, so anything that can write to the cache directory could get code run in every process that later uses yfinance. This is hardening rather than a remote issue (it needs prior write access to the cache dir), but there is no need for pickle here: the cached value is just a dict ofhttp.cookiejar.Cookieobjects.Change
store()serialises{domain: {path: {name: Cookie}}}to JSON (all publicCookieattributes plus_rest).lookup()rebuilds theCookieobjects from JSON.None), so a fresh cookie is fetched and the entry is overwritten. Old entries are never unpickled.cookie_bytesblob column holds the JSON.data.pyis unchanged;_load_cookie_curlCffistill gets the same nested dict ofCookieobjects, including.expires.Testing
Locally: a round trip of a
.yahoo.com/A3cookie preserves every attribute and_rest, and a pickled entry is rejected by the loader instead of being deserialised. I haven't run the full test suite; happy to add a unit test totests/test_cache.pyif you'd like one.