Skip to content

Write what Every's Vibe Checks say the team used into dated history - #12

Draft
kieranklaassen wants to merge 4 commits into
mainfrom
claude/vibe-checks-tool-history-ah2d01
Draft

kieranklaassen wants to merge 4 commits into
mainfrom
claude/vibe-checks-tool-history-ah2d01

Conversation

@kieranklaassen

@kieranklaassen kieranklaassen commented Sep 30, 2026 •

Copy link
Copy Markdown
Collaborator

Requested by Kieran · project thread

Before: a member's history started on the day they first ranked in Toolbox, so "What we used before" had nothing from before this month.

After: what Every's Vibe Checks quoted Kieran, Dan, Katie, Naveen and Mike using is in their dated history, back to March 2025. The coding kind's number one now reads Codex (Oct 2025), Claude Code with Opus 4.5 (Nov), Codex with GPT-5.3 Codex, then GPT-5.4 and GPT-5.5, Claude Code with Fable 5.1 (Sep 1), and Claude Code with Opus 5.5 (Sep 22).

The data is config/vibe_checks.yml: one snapshot per person, kind and article, with the quote it rests on and a note wherever it reads more into the quote than it says (usually the tool when only a model is named). It was read from all 52 articles at every.to/vibe-check. The paywalled parts of older articles were read with a subscriber's access, which added 22 snapshots (including first entries for Nityesh, Alex and Danny) and the retired items Opus 4, Opus 4.1, GPT-5, GPT-4.5, o3, o4-mini and Factory's Droid.

How

  • VibeChecks::Import runs on every deploy after Catalog::Sync (and as bin/rails toolbox:import_vibe_checks). It deletes and rewrites only its own vibe_check rows, never entries or suggestions. It finds verified @every.to members by email, else by name, and skips people who haven't signed up yet.
  • A snapshot holds until the person's next snapshot in that kind, their first change of their own there, or 90 days. Then a closing batch empties the slots, so history still replays to entries (KTD7). Closing rows are flagged details.closing and left out of stories.
  • The older models the articles name (Opus 4.5/4.6/4.8, Sonnet 4.5, 3.7 Sonnet, GPT-5.1 Codex, GPT-5.3 Codex, GPT-5.4, GPT-5.5) are added to the catalog as retired. They are created hidden, stay out of pickers, and NumberOneHistory now counts hidden items (anything not pending).
  • History only counts people who have a pick somewhere today (Audience#counted). A member with no picks yet has rows but doesn't appear in the eras.

Tests: test/services/vibe_checks/import_test.rb covers batches, the stale gap, stopping at the member's own first change, idempotence, name matching, stories, a three-person era with a retired model, and that the shipped file resolves against the catalog. The full Ruby suite passed locally on a copy of the repo; after the paywalled additions, the import and catalog tests pass (43 runs).

🤖 Generated with Claude Code

https://claude.ai/code/session_01FDZZBdXgTcm2Mwsbj9tk9j


Generated by Claude Code

config/vibe_checks.yml records, per team member, the ranked picks a Vibe
Check quoted them using, dated to the article. VibeChecks::Import writes
them as `vibe_check` entry_changes on every deploy, replacing only its own
rows, so "What we used before" reaches back to 2025. Each snapshot holds
until the next one, the member's own first change in the kind, or 90 days,
and a closing batch then empties the slots so history still replays to
the member's picks. Closing rows stay out of stories.

Older models the articles name are added to the catalog as `retired`:
created hidden, out of the pickers, and counted by the number-one history.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FDZZBdXgTcm2Mwsbj9tk9j
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FDZZBdXgTcm2Mwsbj9tk9j
@kieranklaassen kieranklaassen self-assigned this Sep 30, 2026
claude and others added 2 commits September 30, 2026 20:28
…ts failure is not fatal

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FDZZBdXgTcm2Mwsbj9tk9j
Read with a subscriber's access: 22 snapshots for Dan, Kieran, Katie,
Naveen, and first entries for Nityesh, Alex and Danny. Adds Opus 4, Opus
4.1, GPT-5, GPT-4.5, o3, o4-mini and Factory's Droid as retired items.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

Copy link
Copy Markdown
Collaborator Author

Two checks are red on d9a046a, and neither comes from this PR's diff (the last commit only touches config/vibe_checks.yml and config/catalog.yml). Both passed on this branch on Sep 30 and broke on releases published since.

  • scan_ruby: bin/brakeman exits 5 with "Brakeman 8.0.6 is not the latest version 8.1.0". The lockfile pins 8.0.6. Fix: bundle update brakeman.
  • check_js: npm audit --omit=dev --audit-level=moderate now reports source-map-js 1.0.0–1.2.1 (high, GHSA-68fv-2mgg-jv7q) and postcss-selector-parser <7.1.6 (moderate, GHSA-rj75-hqrm-r3gf), which comes in through @tailwindcss/typography. Fix: npm audit fix takes source-map-js to 1.2.2. postcss-selector-parser needs an overrides entry for >=7.1.6 in package.json, since the only fix npm offers is a breaking downgrade of typography to 0.5.4.

main last ran CI on Sep 30, so I expect it to fail the same way on its next run; that is inferred, not observed. No open PR carries either fix. I have not added them here because they are dependency changes outside this PR's scope; I've asked Kieran whether he wants them in this PR or on their own.


Generated by Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants