Skip to content

fix(composer): preserve extra on accept rows so x402 v1 validation accepts them#814

Open
scifantastic wants to merge 3 commits into
Merit-Systems:mainfrom
scifantastic:fix/preserve-extra-on-accept-write
Open

fix(composer): preserve extra on accept rows so x402 v1 validation accepts them#814
scifantastic wants to merge 3 commits into
Merit-Systems:mainfrom
scifantastic:fix/preserve-extra-on-accept-write

Conversation

@scifantastic
Copy link
Copy Markdown

Summary

The agent composer's tool picker (api.public.tools.search) silently drops every resource whose accept rows were stored with extra: null. This affects every Solana provider (and any other provider that sends an extra payload) — they show up in the resource list but never in the composer's tool search.

Symptom

For deepnets.ai, all ~34 endpoints send a populated extra: { feePayer: <pubkey> } on their 402 (Solana relayed-tx fee payer). They all appear in public.resources.search, none appear in public.tools.search — including with showExcluded: true, showDeprecated: true. Composer search for "deepnets" returns zero results.

Root cause

Two compounding bugs, both in this repo:

1. Writer drops extraapps/scan/src/lib/resources.ts:138 hard-codes extra: undefined on every mapped accept row regardless of what the source 402 sent. Prisma persists undefined as null in the JSONB column.

2. Reader can't parse nullpaymentRequirementsSchemaV1 extends PaymentRequirementsSchema from x402/types, which declares extra: z.record(z.any()).optional(). Zod's optional() means T | undefined, not T | null — so a stored null fails validation. When searchX402Tools() (apps/scan/src/services/agent/search-tools.ts:38-54) calls paymentRequirementsSchemaV1.safeParse(...) and it fails, the resource is silently continued.

So the pipeline writes data its own reader can't read.

Fix

Two minimal, surgical changes:

  • apps/scan/src/lib/resources.ts — copy opt.extra through to the mapped accept when the source provides it. Falls back to undefined (preserving prior behaviour) when omitted. Stops new rows from being persisted with null.

  • apps/scan/src/lib/x402/index.ts — defensive read-side coercion in coerceAcceptForV1Schema: strip null from extra before validation. Unblocks the existing rows already in the DB without requiring a re-crawl.

Verification

  • pnpm types:check from apps/scan: zero new errors (unchanged at 324 — all pre-existing in packages/internal/databases/transfers due to ungenerated Prisma clients).
  • npx eslint apps/scan/src/lib/resources.ts apps/scan/src/lib/x402/index.ts: zero new errors (unchanged).
  • npx prettier --check ...: clean.

Manual repro of the symptom (before this fix)

# Sample deepnets resource — extra is populated:
curl -s "https://api.deepnets.ai/api/token-safety/DezXAZ8z7PnrnRJjz3wXBoRgixCa6xjnB7YaB1pPB263" \
  -D - -o /dev/null \
  | grep -i '^payment-required:' | awk '{print $2}' | tr -d '\r' \
  | base64 -d | python3 -m json.tool | grep -A 2 '"extra"'

# Yet on x402scan, public.resources.search returns 34 deepnets rows,
# while public.tools.search returns zero.

Happy to split this into two PRs (writer-side and reader-side) if preferred. There's a separate but related bug in register-origin.ts where OpenAPI-discovered routes are stored with un-substituted path templates (/api/token-safety/{mint} literal); I can file that as a follow-up if you're interested.

@vercel
Copy link
Copy Markdown
Contributor

vercel Bot commented May 2, 2026

@scifantastic is attempting to deploy a commit to the Merit Systems Team on Vercel.

A member of the Team first needs to authorize it.

@scifantastic
Copy link
Copy Markdown
Author

scifantastic commented May 3, 2026

Quick follow-up — both fixes verified end-to-end against a real provider since filing:

  • npx @agentcash/discovery discover api.deepnets.ai returns ok: true, resources: 18, errors: 0, warnings: 0
  • npx agentcash register https://api.deepnets.ai registered 18/18 endpoints cleanly on x402scan
  • AgentCash search at https://agentcash.dev now lists deepnets — was hidden until this fix

The extra: null rejection was the proximate cause of deepnets being invisible in the composer despite being correctly registered (every Solana service that emits extra: { feePayer } hits this). cc @shafu0x since you're most recently active on the repo.

Happy to provide more repro detail or rebase if anything's stale.

@scifantastic
Copy link
Copy Markdown
Author

@shafu0x ping — this PR is the fix for the specific deepnets origin issue you're looking into on Telegram (server id 761de4fb7e0ebdf2b980f4be35e084e66fea06ba0b51fbfce22450460df1e7f4).

The "no x402OriginId assigned" symptom is downstream of paymentRequirementsSchemaV1 rejecting accept rows where extra is stored as null — every Solana service that includes extra: { feePayer } (which the SVM scheme requires) hits this. The v1 schema declares extra as optional() (i.e. T | undefined), but Prisma persists missing extra as null, so the reader can't parse what the writer wrote.

This PR coerces null → undefined in coerceAcceptForV1Schema before validation — should restore the assignment pipeline for our origin and any other Solana provider hitting the same wall.

@scifantastic
Copy link
Copy Markdown
Author

Friendly nudge — this bug is still actively biting on prod as of today (2026-05-04).

Re-confirmed against api.deepnets.ai after canonicalising our URLs to query-string form (so each route is a stable resource URL, not a path-template). Registration via public.resources.registerFromOrigin succeeds:

{ "registered": 16, "failed": 0, "deprecated": 15, "source": "openapi" }

All 16 canonical entries land in public.resources.search cleanly, on Solana, with outputSchema populated and accepts[].extra originally containing { feePayer: "<pubkey>" } in our 402 — but tools.search returns 0/16 for any query that should match ("deepnets", "token-safety", even chains: ["solana"] with no search term — none of our routes appear).

Repro for any maintainer who wants to verify:

# Confirm 16 entries exist in resources DB:
curl -s 'https://www.x402scan.com/api/trpc/public.resources.search?input=%7B%22json%22%3A%7B%22search%22%3A%22deepnets%22%2C%22limit%22%3A50%7D%7D' \
  | jq '.result.data.json | map(.resource) | length'
# 50 (incl deprecated legacy)

# Same query through tools.search → empty:
curl -s 'https://www.x402scan.com/api/trpc/public.tools.search?input=%7B%22json%22%3A%7B%22search%22%3A%22deepnets%22%2C%22limit%22%3A50%7D%7D' \
  | jq '.result.data.json | length'
# 0

Re-checking the writer (lib/resources.ts:138), extra: undefined is still hard-coded on every mapped accept — confirming the diagnosis in the PR description hasn't changed. Every Solana provider that emits extra (which is everyone using a fee-payer relay, including via the official Coinbase CDP facilitator) is invisible to the composer.

Happy to rebase if there are conflicts with the recent merges. Anything else needed from my end to move this forward?

@zdql
Copy link
Copy Markdown
Contributor

zdql commented May 7, 2026

Wouldn't a simpler fix be to just swap the optional zod schema to be nullish?

Comment thread apps/scan/src/lib/x402/index.ts Outdated
// Postgres/Prisma round-trips a JSONB `undefined` as `null`, but the
// v1 PaymentRequirementsSchema declares `extra` as `optional()` — i.e.
// `T | undefined`, not `T | null` — so a `null` here trips the parse
// and the resource is silently dropped from the composer's tool list.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

would not be necessary with a zod.nullish?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in 872aa0b9 — overrode extra: z3.record(z3.any()).nullish() on paymentRequirementsSchemaV1 and dropped this null-stripping branch. Cleaner.

Comment thread apps/scan/src/lib/resources.ts Outdated
// `optional()` not `nullish()`) and the resource was silently
// dropped from the agent composer. Falling back to `undefined`
// when the source omits `extra` keeps prior behaviour.
extra:
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Quite an extensive ternary, very difficult to read

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fair, that was ugly. Simplified to a single cast in a77e9c9f@agentcash/discovery's X402V{1,2}PaymentOption types don't declare extra yet, hence the cast; once they do it can go away.

@scifantastic
Copy link
Copy Markdown
Author

Good call — pushed 872aa0b9 overriding extra: z3.record(z3.any()).nullish() on paymentRequirementsSchemaV1 and dropping the coerceAcceptForV1Schema null-stripping branch. Cleaner.

Kept the writer-side change in resources.ts:138, though — that one isn't about validation tolerance, it's preserving the source extra payload itself. Today every mapped accept hard-codes extra: undefined, so even with .nullish() reading cleanly the persisted payload is empty — the original { feePayer: <pubkey> } from each Solana 402 never makes it into the DB. Downstream SVM-scheme tool-execution needs that field to construct the relayed-tx, so we want to preserve it on the way in rather than rely on it being null-tolerant on the way out.

…accepts them

`registerResource()` was hard-coding `extra: undefined` on every mapped
accept row regardless of what the source 402 sent. Prisma persists that
as `extra: null` in the JSONB column, but the v1 PaymentRequirementsSchema
declares `extra` as `optional()` — i.e. `T | undefined`, not `T | null` —
so the read-side `paymentRequirementsSchemaV1.safeParse(...)` in
`searchX402Tools()` rejects every such row. The resource is silently
`continue`d and never reaches the composer's tool list.

Concrete repro: every deepnets.ai endpoint sends `extra: { feePayer: ... }`
on its 402 (Solana fee payer for the relayed-tx flow). All ~34 rows for
the deepnets origin are present in `public.resources.search` results, but
zero appear in `public.tools.search` because every accept row's stored
`extra` is `null` and parse fails.

Two-part fix:

- `apps/scan/src/lib/resources.ts` — copy `opt.extra` through to the
  mapped accept when the source provides it. Falls back to `undefined`
  (preserving prior behaviour) when omitted.

- `apps/scan/src/lib/x402/index.ts` — defensive read-side coercion in
  `coerceAcceptForV1Schema`: strip `null` from known-optional fields
  (currently just `extra`) before validation. This unblocks the existing
  rows already in the DB without requiring a re-crawl.
…dback)

zdql suggested a cleaner read-side fix: rather than coercing `null`
to `undefined` in `coerceAcceptForV1Schema`, just override the
inherited `extra: optional()` on `paymentRequirementsSchemaV1` to
`extra: nullish()`. Same effect at the validation boundary, fewer
moving parts.

- Drop the `if (base.extra === null) delete base.extra` branch in
  `coerceAcceptForV1Schema` — no longer needed.
- Override `extra: z3.record(z3.any()).nullish()` in
  `paymentRequirementsSchemaV1.extend({ ... })` so `null` parses
  cleanly.

Writer-side change in `resources.ts` is intentionally retained — it
preserves the source `extra` payload itself (e.g. Solana fee payer)
rather than dropping it on the floor. That isn't about validation
tolerance; without it, every Solana accept row stored after this
PR still loses its `extra.feePayer`, which downstream tool-execution
needs to construct the relayed-tx.
…w feedback)

zdql noted the previous ternary was hard to read. Simplify to a
single locally-scoped cast — `@agentcash/discovery`'s payment-option
types don't declare `extra` yet, hence the cast; once they do it can
go away entirely.
@scifantastic scifantastic force-pushed the fix/preserve-extra-on-accept-write branch from a77e9c9 to ebf3e4f Compare May 7, 2026 17:17
@scifantastic
Copy link
Copy Markdown
Author

Heads up — that last force-push was a rebase onto current main to resolve the conflict markers GitHub was showing. No substantive change to the diff:

  • Conflicts were entirely in apps/scan/src/lib/resources.ts and came from main's recent restructure of the accept mapping into a two-step allMappedAcceptsmappedAccepts filter + the new "no supported networks advertised" error block.
  • Resolution: kept upstream's two-step structure and "no supported networks" error path verbatim. The only line of mine that lands in the new structure is extra: (opt as { extra?: Record<string, unknown> }).extra, inside the allMappedAccepts.map block, which is the simplified cast from a77e9c9f.
  • Schema-side change (paymentRequirementsSchemaV1.extra: nullish()) and the deletion of the null-stripping branch in coerceAcceptForV1Schema rebased clean.

Final tip is ebf3e4f6 — three commits, no merge commit. PR is back to MERGEABLE.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants