Skip to content

feat(discovery): add OMP session discovery - #21

Closed
HarshalRathore wants to merge 13 commits into
kunchenguid:mainfrom
HarshalRathore:add-omp-adapter
Closed

HarshalRathore wants to merge 13 commits into
kunchenguid:mainfrom
HarshalRathore:add-omp-adapter

Conversation

@HarshalRathore

@HarshalRathore HarshalRathore commented Aug 23, 2026 •

Copy link
Copy Markdown

What Changed

  • Added an OMP (oh-my-pi) discovery adapter for ~/.omp/agent/sessions, including title-first/session-header classification, model changes, message filtering, and correlated tool results.
  • Registered OMP across discovery, default configuration, CLI help, and documentation, with guidance for explicitly pinned legacy harness lists.
  • Versioned and validated scan-cache descriptors so stale or malformed cached results are reclassified, with fixtures and coverage for OMP parsing, wiring, and cache behavior.

Risk Assessment

⚠️ Medium: The normal OMP path is fixture-backed, but missing-store opacity, a one-time full-cache scan, and real duplicate call records can weaken or distort evidence without raising an error.

Testing

Targeted tests, explicit/default CLI scans, and adapter-to-distiller verification completed successfully; reviewer-visible evidence was saved outside the worktree, and transient test state was cleaned up.

Evidence: Explicit OMP CLI scan

scan exit=0; omp matched=1; nativeId=omp-cli-e2e

{
  "repo": "01M0SEP2GBN47N73SZ6YQJNKG3",
  "perHarness": {
    "omp": {
      "scanned": 1,
      "matched": 1,
      "cached": 0,
      "skipped": 0,
      "self": 0,
      "error": null
    }
  },
  "transcripts": [
    {
      "harness": "omp",
      "id": "omp-omp-cli-e2e",
      "nativeId": "omp-cli-e2e",
      "path": "/tmp/backpass-omp-cli-home-7D7Q3G/.omp/agent/sessions/escaped-cwd/2026-08-24T09-00-00-000Z_omp-cli-e2e.jsonl",
      "cwd": "/home/harshal/.no-mistakes/worktrees/407965ada9bb/01M0SEP2GBN47N73SZ6YQJNKG3",
      "gitBranch": null,
      "title": "CLI OMP smoke test",
      "model": null,
      "startedAt": 1787562000000,
      "mtimeMs": 1787561719753.05,
      "bytes": 728,
      "experimental": false,
      "association": {
        "tier": 1,
        "confidence": "exact",
        "reason": "cwd is worktree /home/harshal/.no-mistakes/worktrees/407965ada9bb/01M0SEP2GBN47N73SZ6YQJNKG3"
      },
      "extra": {}
    }
  ]
}
Evidence: Default harness scan

default scan exit=0; omp matched=1; harnesses=claude,codex,omp,pi,opencode,grok,cursor,hermes

{
  "repo": "01M0SEP2GBN47N73SZ6YQJNKG3",
  "perHarness": {
    "claude": {
      "scanned": 0,
      "matched": 0,
      "cached": 0,
      "skipped": 0,
      "self": 0,
      "error": null
    },
    "codex": {
      "scanned": 0,
      "matched": 0,
      "cached": 0,
      "skipped": 0,
      "self": 0,
      "error": null
    },
    "omp": {
      "scanned": 1,
      "matched": 1,
      "cached": 0,
      "skipped": 0,
      "self": 0,
      "error": null
    },
    "pi": {
      "scanned": 0,
      "matched": 0,
      "cached": 0,
      "skipped": 0,
      "self": 0,
      "error": null
    },
    "opencode": {
      "scanned": 0,
      "matched": 0,
      "cached": 0,
      "skipped": 0,
      "self": 0,
      "error": null
    },
    "grok": {
      "scanned": 0,
      "matched": 0,
      "cached": 0,
      "skipped": 0,
      "self": 0,
      "error": null
    },
    "cursor": {
      "scanned": 0,
      "matched": 0,
      "cached": 0,
      "skipped": 0,
      "self": 0,
      "error": null
    },
    "hermes": {
      "scanned": 0,
      "matched": 0,
      "cached": 0,
      "skipped": 0,
      "self": 0,
      "error": null
    }
  },
  "transcripts": [
    {
      "harness": "omp",
      "id": "omp-omp-default-e2e",
      "nativeId": "omp-default-e2e",
      "path": "/tmp/backpass-omp-default-home-oQQZAj/.omp/agent/sessions/escaped-cwd/2026-08-24T09-00-00-000Z_omp-default-e2e.jsonl",
      "cwd": "/home/harshal/.no-mistakes/worktrees/407965ada9bb/01M0SEP2GBN47N73SZ6YQJNKG3",
      "gitBranch": null,
      "title": "Default OMP scan",
      "model": null,
      "startedAt": 1787562000000,
      "mtimeMs": 1787561800656.2932,
      "bytes": 483,
      "experimental": false,
      "association": {
        "tier": 1,
        "confidence": "exact",
        "reason": "cwd is worktree /home/harshal/.no-mistakes/worktrees/407965ada9bb/01M0SEP2GBN47N73SZ6YQJNKG3"
      },
      "extra": {}
    }
  ]
}
Evidence: OMP normalized trace

harness: omp model: openai-codex/gpt-5.5 tool: bash "npm run deploy" [error] -> deployment failed

# session omp-1234
harness: omp
model: openai-codex/gpt-5.5
date: 2026-08-20T10:00:00.000Z
cwd: /repo/demo
association: tier 1 (exact)

\### turn 1 · user
What changed in the parser?

tool: bash "npm test" -> 2 passing
\### turn 2 · assistant
Running the tests first.

\### turn 3 · assistant
Tests pass; the parser fix is safe.

tool: bash "npm run deploy" [error] -> deployment failed

---
raw transcript: test/fixtures/omp-session.jsonl
Tool calls above are one-line summaries and tool output is truncated. Open the raw
transcript only if a specific claim needs the full text.

Pipeline

Updates from git push no-mistakes

⏭️ **intent** - skipped

✅ No issues found.

✅ **Rebase** - passed

✅ No issues found.

⚠️ **Review** - 3 warnings
  • ⚠️ src/discovery/adapters/omp.js:41 - omp.enumerate() calls the fail-soft directory helper at this line, which returns an empty list for a missing or unreadable ~/.omp/agent/sessions; malformed files then return null from classify() and are silently counted as skipped. backpass scan therefore reports no OMP history with no named warning, contrary to the fail-soft contract requiring drifted or missing stores to be distinguishable.
  • ⚠️ src/discovery/index.js:19 - The global cache epoch invalidates every pre-existing file-adapter entry after this OMP-only change. A repository with thousands of cached Codex, Claude, or Pi sessions will synchronously reclassify and rewrite the entire store on the first scan, despite those classifiers being unchanged. Use an adapter-specific cache epoch or preserve compatible entries.
  • ⚠️ src/discovery/adapters/omp.js:123 - OMP records can contain duplicate assistant toolCall entries with the same id. For input [tool(id=x), tool(id=x), tool-result(id=x,error)], attachToolResults maps the id to the last call, leaving the first call result-less while emitting both calls. This produces an incorrect normalized trace and can inflate tool-call counts or lose the failure signal; deduplicate identical same-id calls or correlate duplicates deterministically before attaching results.
✅ **Test** - passed

✅ No issues found.

  • pnpm exec node --test test/adapters.test.js test/config.test.js test/self-session.test.js
  • node bin/backpass.js scan --harness omp --since all --json --quiet with a temporary OMP store
  • node bin/backpass.js scan --since all --json --quiet with default harness configuration
  • readTranscript() plus distill() on test/fixtures/omp-session.jsonl
  • Evidence JSON and trace readability validation
✅ **Document** - passed

✅ No issues found.

✅ **Lint** - passed

✅ No issues found.

✅ **Push** - passed

✅ No issues found.

kunchenguid added a commit that referenced this pull request Aug 23, 2026
* docs: add VISION.md draft and review answers scaffold

Evidence-grounded from-scratch draft mined from the 20 merged PRs in this
repo, open issues #23/#24/#25, community PR #21, and module headers.
Under review with the author on a /vision board; verdicts fold in next.

* docs(vision): fold in CLAUDE.md and the neural-net post's rationale

Board feedback r1:
- opener names CLAUDE.md alongside AGENTS.md, and separates the project
  file it trains from the user-level ~/.claude/CLAUDE.md it never writes
- budget section names the four decay states and why appending wins
- batch size, step size and skill-vs-memory placement carry their reasons
- adds: rules are enforced in code, not asked for in a prompt

* docs(vision): fold in all 12 board verdicts

6 in vision, 5 off mission, 1 conditional. Three reversed the draft:
- H-1 team/multi-machine pooling is in vision, so the locality section is
  now about owning nothing rather than about one machine
- H-3 an opt-in unattended apply is still the human in control
- H-11 counting uncorroborated gaps is allowed, proposing them is not
H-2 makes today's ungated bootstrap apply a deviation to fix.
Verdicts and reasoning recorded verbatim in VISION-ANSWERS.md.

* docs: point AGENTS.md at the vision and its recorded answers
@kunchenguid

Copy link
Copy Markdown
Owner

Speaking as Kun's firstmate.

VISION verdict (per rule):

  • Evidence is the only currency: aligns if omp jsonl is real turns (same shape as pi: session header + message roles). The PR claims that and reads user/assistant/toolResult.
  • The human owns the weights: aligns (discovery only).
  • Nothing the model says is taken on faith: cannot tell / incomplete. VISION: supporting a harness means a pinned fixture and a fail-soft adapter, not a row in the README. This PR adds omp.js and registers it; I do not see a pinned fixture or adapter test.
  • The budget is the constraint: n/a.
  • It reads what you already have: aligns in kind (local store ~/.omp/agent/sessions, no upload, no key). More harnesses are welcome if they cannot destabilise existing ones. Fail-soft for a missing dir is implied via shared listDirs, not proven by a fixture.
  • Failure is loud and named: cannot tell without a missing-store test.
  • Scope: aligns (adapter, not a writer/provider).

Reviewed the diff (cli.js, config.js ALL_HARNESSES, omp.js, discovery/index.js). No security issue in the adapter itself. This is a first-time fork PR. Merge state is CONFLICTING/DIRTY against current main (main already has hermes and more adapters). no-mistakes has not raised.

This is waiting on you, not on the captain: rebase onto current main, add a pinned omp fixture plus a test that fails if the adapter is removed, then git push no-mistakes. I will not resolve the conflict until the PR is otherwise auto-merge-ready (fixture, green CI, safe review).

@HarshalRathore

Copy link
Copy Markdown
Author

Rebased onto current main (e468608) — conflict resolved, single commit. Addressing the review checklist:

Pinned fixture + tests added (the "evidence" ask):

  • test/fixtures/omp-session.jsonl — pinned to a real OMP store (~10k files), not invented.
  • test/adapters.test.js — 3 new tests: classify through the title line, read/model/tool-folding with thinking dropped, plus an explicit registration test that fails if the adapter is unhooked from ADAPTERS or ALL_HARNESSES.

Real-format findings from the live store (fixed in this commit):

  1. ~85% of sessions start with {type:"title"}, not {type:"session"} — the previous header-only classify would have silently skipped most real sessions. classify() now scans the first few lines for the session header and surfaces the human-readable title.
  2. model_change carries model, never modelId (verified: 58/58 sampled files) — field order flipped and doc comment corrected.
  3. Fixture pins the exact toolCall shape (type/name/arguments), toolResult folding, and thinking-block dropping.

Verification: lint ✓ · prettier ✓ · tsc ✓ · 221/221 tests ✓ · live smoke: scan --harness omp --since 30d --limit 3 against the real store scanned 202 recent files without error.

Ready for re-review.

@HarshalRathore

Copy link
Copy Markdown
Author

cc @kunchenguid — pushed 0a47432 (rebased on main, fixture + tests + real-format fixes per your review). Ready for re-review.

@kunchenguid

Copy link
Copy Markdown
Owner

Speaking as Kun's firstmate.

Re-reviewed the new commit (0a47432), not only the comment.

VISION:

  • Evidence: aligns. I inspected test/fixtures/omp-session.jsonl: title line then session header, user/assistant/toolResult turns, model_change.model. classify() scans HEADER_SCAN_LINES for that header instead of skipping title-first files.
  • Human owns the weights: aligns (discovery only).
  • Nothing on faith: aligns now. Pinned fixture plus tests, including one that fails if omp is dropped from ADAPTERS or ALL_HARNESSES.
  • Budget: n/a.
  • Reads what you have: aligns. Local ~/.omp store, fail-soft empty enumerate. More harnesses welcome; this should not destabilise the others (own adapter, hermes import kept).
  • Failure loud: aligns for classify-null skip; missing dir is silent empty like pi (same shared listDirs).
  • Scope: aligns.

Conflicts are gone (MERGEABLE). I approved first-time fork CI on this SHA. Adding omp to the default harness list matches how hermes landed, so I am not holding that as a captain default-behavior call.

Still waiting on you, not on the captain: git push no-mistakes. no-mistakes did not run on 0a47432 (that workflow is opened/edited/reopened, not synchronize), and the PR body still has no attestation. That check is blocking. I will not merge until it is green.

@HarshalRathore HarshalRathore changed the title Add omp (oh-my-pi) transcript adapter feat(discovery): add OMP session discovery Aug 24, 2026
@HarshalRathore

Copy link
Copy Markdown
Author

The no-mistakes pipeline reviewed the branch and applied your three findings in 594accd: error-status preservation on tool results, titles read from session headers (+ pinned fixture), and the README harness-count/table/config drift. Full suite passes locally (221/221) and the already-approved runs on 0a47432 were green.

The re-pushed head 594accd needs first-time-fork CI approval again (runs are sitting in action_required) — could you approve those when you get a chance?

…ig note

- classify(): invalid session timestamp falls back to file mtime instead of
  caching NaN (which crashed distillation later)
- read(): track tool-call ids per read and drop results whose call was
  truncated away by the 64MB tail window, instead of letting them attach to
  an unrelated later call
- README: document that explicit discovery.harnesses replaces defaults
@kunchenguid

Copy link
Copy Markdown
Owner

Speaking as Kun's firstmate.

Re-reviewed the no-mistakes follow-up, not only the comment. Inspected commit 594accd:

  • toolResult now sets status: message.isError ? "error" : "completed" (src/discovery/adapters/omp.js). Fixture pins a failed bash deploy with isError true; test asserts completed vs error.
  • classify() also takes title from the session header when there is no leading title line; pinned omp-session-header-title.jsonl.
  • README: eight harnesses, omp row, default harness list includes omp.

VISION:

  • Evidence: aligns. Failed tools stay visible instead of looking successful.
  • Human owns the weights: aligns (discovery only).
  • Nothing on faith: aligns (fixture + tests for the new paths).
  • Budget: n/a.
  • Reads what you have: aligns. Local store, default harness list matches how hermes landed.
  • Failure is loud and named: aligns. Dropping isError would have papered over a real loss signal; this keeps it.
  • Scope: aligns.

Approved first-time fork CI on current head fc6af9a.

Still waiting on you, not on the captain: the no-mistakes attestation in the PR body is bound to 0a47432, but HEAD is fc6af9a. That SHA mismatch is blocking. Run git push no-mistakes again so the attestation matches this head. I will not merge until that check is green on the current SHA.

@HarshalRathore HarshalRathore changed the title feat(discovery): add OMP session discovery feat(discovery): add OMP session ingestion Aug 24, 2026
@HarshalRathore

Copy link
Copy Markdown
Author

Final head is e0c1070 (no-mistakes attestation rewritten for it ✓). The review pipeline ran fully green locally; the remaining red X's are just the first-time-fork workflow approvals (action_required) for CI / Guard / no-mistakes on this SHA — could you approve those? After that this is auto-merge-ready.

@HarshalRathore

Copy link
Copy Markdown
Author

Correction to my last comment: the gate pushed one more hardening commit, so the final head is 7ccab95 (adds a string session-id guard + scan-cache descriptor validation, both with regression tests — suite is 227/227 locally). The no-mistakes attestation in the PR body is being rewritten to bind 7ccab95 as its ci-watch step closes.

Could you approve the first-time-fork workflow runs sitting in action_required on this head (7ccab95) — CI / Guard / no-mistakes? Those are the ones that matter; anything older can be ignored.

@HarshalRathore HarshalRathore changed the title feat(discovery): add OMP session ingestion feat(discovery): add OMP session discovery Aug 24, 2026
@HarshalRathore

Copy link
Copy Markdown
Author

Final head is now 05b9b441 — it folds in the three review findings (duplicate toolCall-id correlation, adapter-scoped cache epochs, plus a doc note). Local suite: 230/230 green.

One heads-up on process: the gate's ci-watcher was misreading GitHub's action_required approval wall as a fixable CI failure and pushing after the attestation step, which kept re-staling the signed SHA binding. I've disabled that behavior (auto_fix.ci: 0), so the branch is now frozen — whatever you approve is final.

Could you approve the first-time-fork runs on this exact head? They're sitting in action_required:

  • CI → run 32712608177
  • Guard generated files → run 32712608199

Once those are green I'll do one last gate pass purely to re-sign the attestation against 05b9b441, and this will be merge-ready.

@kunchenguid

Copy link
Copy Markdown
Owner

Speaking as Kun's firstmate.

Re-reviewed head 05b9b44, not only the comment.

Inspected:

  • omp.enumerate now throws storeStatus missing/unreadable instead of returning []; discovery names that and skips. That is the loud missing-store path VISION wants.
  • classifierVersion is adapter-scoped (omp v2); other adapters keep compatible cache entries. usableDescriptor rejects malformed cached ids/cwds.
  • duplicate same-id tool calls keep the first; unmatched results after a tail-read are dropped instead of folding onto the next call.
  • Workflow: .github/workflows/guard-generated-files.yml only changes how CHANGELOG.md and .release-please-manifest.json are selected (git diff -- path instead of grep). Same two files, no permissions change. Not treating that as a security risk.

VISION: evidence/accuracy aligns (failed tools, no mis-attributed results). Human owns weights: discovery only. Nothing on faith: fixtures + tests. Budget n/a. Reads what you have: local store, named skip if missing. Failure loud: missing store is named. Scope: adapter + cache epoch, not a writer.

Approved first-time fork CI and Guard on 05b9b44 (runs 32712608177, 32712608199).

Still waiting on you, not on the captain: the no-mistakes attestation in the body is bound to 23267f4, not 05b9b44. Re-sign so the bound SHA is this head. I will not merge until that check is green on 05b9b44.

@kunchenguid

Copy link
Copy Markdown
Owner

Speaking as Kun's firstmate: closing this as stale. It has been waiting on a contributor update for 14+ days with no author push or comment (needed: re-sign no-mistakes attestation on head 05b9b441; also now CONFLICTING vs main). Reopen if you want to pick it back up. Issue #53 (OMP) stays ready-for-pr.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants