Repository navigation
test: add a weekly canary against the live feeds - #43
Merged
Merged
Conversation
The suite parses the eight feeds as they were recorded in tests/data, so a publisher changing its feed (an image that stops arriving, a renamed title, a moved URL that answers with an HTML page) left it green. scripts/canary.py now reads every feed live with the same sensor configuration and fails when no entry is found or when a field the recording had in every entry (as feedparser reads it and as the sensor exposes it) is in fewer than half the live entries. Exit 1 means the format changed, 2 that a feed could not be fetched. .github/workflows/canary.yml runs it on Mondays and writes the result into the job summary. scripts/refresh_fixtures.py re-records the feeds byte for byte once the parser is adjusted (--check only reports). tests/test_canary.py pins the canary's reading on the recordings so it cannot rot unnoticed.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The suite parses the eight feeds as recorded in
tests/data, so a publisher changing its feed (an image that stops arriving, a renamed title, a moved URL that now answers with an HTML page) leaves it green. This adds the check at the real seam, modelled on skyline-webcams and bergfex's live check.scripts/canary.pyreads every feed live with the same sensor configuration and fails when no entry is found, or when a field the recording had in every entry (as feedparser reads it and as the sensor exposes it, plus channel keys) is in fewer than half of the live entries. Exit 1 = format changed, 2 = a feed could not be fetched (says nothing about the format)..github/workflows/canary.ymlruns it weekly (Mondays 06:23 UTC) and on manual dispatch; the result goes into the job summary. The job fails on 1 and 2.scripts/refresh_fixtures.py [names] [--check]re-records feeds byte for byte (and setsdownload_date) after the parser has been adjusted.--checkonly reports.tests/test_canary.pypins the canary's reading on the recordings (a recording is never different from itself; renamed titles/links and an HTML page are detected), so the canary cannot rot.pyproject.toml:scriptsadded to pytestpythonpath.The existing recordings were kept as they are (they already are real upstream payloads, with a download script); no fixture was rewritten. No integration code changed.
Tests
pytest tests: 346 passed (298 before, 48 new).python scripts/canary.pyagainst the live feeds: all 8 feeds read, exit 0.refresh_fixtures.py --checkrun as well.Board card c078.