feat(storage): support caller-defined dataset IDs#19370
Conversation
Allow dataset callers to supply durable IDs and atomically resolve compatible retries through each built-in storage adapter. Reject incompatible immutable identity reuse and cover concurrency, retry, normalization, and deletion semantics in the shared storage contract tests. Co-Authored-By: Mastra Code (openai/gpt-5.6-sol) <noreply@mastra.ai>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
1 Skipped Deployment
|
🦋 Changeset detectedLatest commit: 49a57d4 The changes in this PR will be included in the next version bump. This PR includes changesets to release 28 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
|
Caution Review failedThe pull request is closed. ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (5)
WalkthroughDataset creation now accepts caller-defined IDs across core and built-in storage adapters. Compatible retries return existing records, immutable-field conflicts raise ChangesDataset ID resolution
Estimated code review effort: 4 (Complex) | ~45 minutes Possibly related PRs
Suggested reviewers: 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Warning There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure. 🔧 ESLint
ESLint install timed out. The project may have too many dependencies for the sandbox. Comment |
PR triageLinked issue check skipped for core contributor @NikAiyer. PR complexity score
Applied label: Changed test gateChanged tests failed against the base branch as expected. Label: |
Avoid runtime coupling between storage adapters and newly exported core helper functions while preserving shared dataset identity semantics through protected base methods. Consolidate the package release notes into one changeset.\n\nCo-Authored-By: Mastra Code (openai/gpt-5.6-sol) <noreply@mastra.ai>
Raise built-in storage adapter peer floors so protected dataset identity helpers are guaranteed at runtime.\n\nCo-Authored-By: Mastra Code (openai/gpt-5.6-sol) <noreply@mastra.ai>
|
Dependency limit exceeded — report not shown. This pull request scan exceeded the 10,000-dependency limit applied to this scan, so the results are incomplete and may be inaccurate. To avoid reporting false positives, Socket has not posted a report. Upgrade your plan to raise the dependency limit and get complete reports, or view the partial scan in the dashboard. Socket is always free for open source. If this is a non-commercial open source project, contact us to request a free Team account. |
Avoid coupling dataset conflict handling to the formatting of upsert SQL by sharing a dedicated plain INSERT statement builder.\n\nCo-Authored-By: Mastra Code (openai/gpt-5.6-sol) <noreply@mastra.ai>
This PR was opened by the [Changesets release](https://github.com/changesets/action) GitHub action. When you're ready to do a release, you can merge this and publish to npm yourself or [setup this action to publish automatically](https://github.com/changesets/action#with-publishing). If you're not ready to do a release yet, that's fine, whenever you add more changesets to main, this PR will be updated.⚠️ ⚠️ ⚠️ ⚠️ ⚠️ ⚠️ `main` is currently in **pre mode** so this branch has prereleases rather than normal releases. If you want to exit prereleases, run `changeset pre exit` on `main`.⚠️ ⚠️ ⚠️ ⚠️ ⚠️ ⚠️ # Releases ## @mastra/client-js@1.32.0-alpha.13 ### Minor Changes - Added caller-defined dataset item identities for safe retries across all dataset storage adapters. ([#19384](#19384)) Dataset items can now include an `externalId` when calling `addItem` or `addItems`: ```ts await dataset.addItem({ externalId: 'source-item-123', input: { prompt: 'Hello' }, }); ``` Retrying with the same identity and payload returns the existing item. Reusing an identity with different content returns a typed conflict, including during concurrent writes. Updates and deletes preserve the identity, Spanner retries transactions without changing the outcome, and MySQL batch writes now preserve every supported dataset item field. ### Patch Changes - Type structured Agent Controller notification message content so Web clients can render notification provenance from live and persisted transcript messages. ([#19446](#19446)) ```ts if (part.type === 'notification') { renderNotification(part.message, part.source); } ``` - Updated dependencies [[`a99eae8`](a99eae8), [`fd13f8e`](fd13f8e), [`f703f87`](f703f87), [`0ad646f`](0ad646f)]: - @mastra/core@1.51.0-alpha.13 ## @mastra/code-sdk@0.1.0-alpha.13 ### Minor Changes - Added a post-tool observer for custom Mastra Code integrations to react to completed tool calls without replacing built-in tools. ([#19446](#19446)) ```ts await mountAgentControllerOnMastra({ postToolObserver: ({ toolName, output }) => logToolResult(toolName, output), }); ``` ### Patch Changes - Added the authoritative session scope to agent controller request context for scoped session integrations. ([#19446](#19446)) ```ts const controllerContext = requestContext.get('controller'); console.log(controllerContext?.scope); ``` - Updated dependencies [[`a99eae8`](a99eae8), [`fd13f8e`](fd13f8e), [`f703f87`](f703f87), [`0ad646f`](0ad646f), [`0ad646f`](0ad646f)]: - @mastra/core@1.51.0-alpha.13 - @mastra/pg@1.16.0-alpha.0 - @mastra/libsql@1.16.0-alpha.1 ## @mastra/core@1.51.0-alpha.13 ### Minor Changes - Added atomic caller-defined dataset IDs for idempotent dataset creation across built-in storage adapters. Supplying `id` to `mastra.datasets.create()` now creates the dataset once and resolves compatible retries to the persisted record; incompatible immutable identity fields throw `DATASET_ID_CONFLICT`. ([#19370](#19370)) - Added the authoritative session scope to agent controller request context for scoped session integrations. ([#19446](#19446)) ```ts const controllerContext = requestContext.get('controller'); console.log(controllerContext?.scope); ``` - Added caller-defined dataset item identities for safe retries across all dataset storage adapters. ([#19384](#19384)) Dataset items can now include an `externalId` when calling `addItem` or `addItems`: ```ts await dataset.addItem({ externalId: 'source-item-123', input: { prompt: 'Hello' }, }); ``` Retrying with the same identity and payload returns the existing item. Reusing an identity with different content returns a typed conflict, including during concurrent writes. Updates and deletes preserve the identity, Spanner retries transactions without changing the outcome, and MySQL batch writes now preserve every supported dataset item field. ### Patch Changes - Fixed parallel sub-agent approvals so they can be handled in any order. listSuspendedRuns() now returns each pending sub-agent call, and approving one resumes that specific call instead of using another call’s suspended state. ([#19450](#19450)) ## @mastra/server@1.51.0-alpha.13 ### Minor Changes - Added caller-defined dataset item identities for safe retries across all dataset storage adapters. ([#19384](#19384)) Dataset items can now include an `externalId` when calling `addItem` or `addItems`: ```ts await dataset.addItem({ externalId: 'source-item-123', input: { prompt: 'Hello' }, }); ``` Retrying with the same identity and payload returns the existing item. Reusing an identity with different content returns a typed conflict, including during concurrent writes. Updates and deletes preserve the identity, Spanner retries transactions without changing the outcome, and MySQL batch writes now preserve every supported dataset item field. ### Patch Changes - Updated dependencies [[`a99eae8`](a99eae8), [`fd13f8e`](fd13f8e), [`f703f87`](f703f87), [`0ad646f`](0ad646f)]: - @mastra/core@1.51.0-alpha.13 ## @mastra/libsql@1.16.0-alpha.1 ### Minor Changes - Added atomic caller-defined dataset IDs for idempotent dataset creation across built-in storage adapters. Supplying `id` to `mastra.datasets.create()` now creates the dataset once and resolves compatible retries to the persisted record; incompatible immutable identity fields throw `DATASET_ID_CONFLICT`. ([#19370](#19370)) - Added caller-defined dataset item identities for safe retries across all dataset storage adapters. ([#19384](#19384)) Dataset items can now include an `externalId` when calling `addItem` or `addItems`: ```ts await dataset.addItem({ externalId: 'source-item-123', input: { prompt: 'Hello' }, }); ``` Retrying with the same identity and payload returns the existing item. Reusing an identity with different content returns a typed conflict, including during concurrent writes. Updates and deletes preserve the identity, Spanner retries transactions without changing the outcome, and MySQL batch writes now preserve every supported dataset item field. ### Patch Changes - Raised the `@mastra/core` peer dependency floor to `>=1.51.0-0` so the dataset item identity helpers used by the storage adapters are available at runtime. ([#19384](#19384)) - Updated dependencies [[`a99eae8`](a99eae8), [`fd13f8e`](fd13f8e), [`f703f87`](f703f87), [`0ad646f`](0ad646f)]: - @mastra/core@1.51.0-alpha.13 ## @mastra/mongodb@1.13.0-alpha.0 ### Minor Changes - Added atomic caller-defined dataset IDs for idempotent dataset creation across built-in storage adapters. Supplying `id` to `mastra.datasets.create()` now creates the dataset once and resolves compatible retries to the persisted record; incompatible immutable identity fields throw `DATASET_ID_CONFLICT`. ([#19370](#19370)) - Added caller-defined dataset item identities for safe retries across all dataset storage adapters. ([#19384](#19384)) Dataset items can now include an `externalId` when calling `addItem` or `addItems`: ```ts await dataset.addItem({ externalId: 'source-item-123', input: { prompt: 'Hello' }, }); ``` Retrying with the same identity and payload returns the existing item. Reusing an identity with different content returns a typed conflict, including during concurrent writes. Updates and deletes preserve the identity, Spanner retries transactions without changing the outcome, and MySQL batch writes now preserve every supported dataset item field. ### Patch Changes - Raised the `@mastra/core` peer dependency floor to `>=1.51.0-0` so the dataset item identity helpers used by the storage adapters are available at runtime. ([#19384](#19384)) - Updated dependencies [[`a99eae8`](a99eae8), [`fd13f8e`](fd13f8e), [`f703f87`](f703f87), [`0ad646f`](0ad646f)]: - @mastra/core@1.51.0-alpha.13 ## @mastra/mysql@0.4.0-alpha.0 ### Minor Changes - Added atomic caller-defined dataset IDs for idempotent dataset creation across built-in storage adapters. Supplying `id` to `mastra.datasets.create()` now creates the dataset once and resolves compatible retries to the persisted record; incompatible immutable identity fields throw `DATASET_ID_CONFLICT`. ([#19370](#19370)) - Added caller-defined dataset item identities for safe retries across all dataset storage adapters. ([#19384](#19384)) Dataset items can now include an `externalId` when calling `addItem` or `addItems`: ```ts await dataset.addItem({ externalId: 'source-item-123', input: { prompt: 'Hello' }, }); ``` Retrying with the same identity and payload returns the existing item. Reusing an identity with different content returns a typed conflict, including during concurrent writes. Updates and deletes preserve the identity, Spanner retries transactions without changing the outcome, and MySQL batch writes now preserve every supported dataset item field. ### Patch Changes - Raised the `@mastra/core` peer dependency floor to `>=1.51.0-0` so the dataset item identity helpers used by the storage adapters are available at runtime. ([#19384](#19384)) - Updated dependencies [[`a99eae8`](a99eae8), [`fd13f8e`](fd13f8e), [`f703f87`](f703f87), [`0ad646f`](0ad646f)]: - @mastra/core@1.51.0-alpha.13 ## @mastra/pg@1.16.0-alpha.0 ### Minor Changes - Added atomic caller-defined dataset IDs for idempotent dataset creation across built-in storage adapters. Supplying `id` to `mastra.datasets.create()` now creates the dataset once and resolves compatible retries to the persisted record; incompatible immutable identity fields throw `DATASET_ID_CONFLICT`. ([#19370](#19370)) - Added caller-defined dataset item identities for safe retries across all dataset storage adapters. ([#19384](#19384)) Dataset items can now include an `externalId` when calling `addItem` or `addItems`: ```ts await dataset.addItem({ externalId: 'source-item-123', input: { prompt: 'Hello' }, }); ``` Retrying with the same identity and payload returns the existing item. Reusing an identity with different content returns a typed conflict, including during concurrent writes. Updates and deletes preserve the identity, Spanner retries transactions without changing the outcome, and MySQL batch writes now preserve every supported dataset item field. ### Patch Changes - Raised the `@mastra/core` peer dependency floor to `>=1.51.0-0` so the dataset item identity helpers used by the storage adapters are available at runtime. ([#19384](#19384)) - Updated dependencies [[`a99eae8`](a99eae8), [`fd13f8e`](fd13f8e), [`f703f87`](f703f87), [`0ad646f`](0ad646f)]: - @mastra/core@1.51.0-alpha.13 ## @mastra/spanner@1.3.0-alpha.2 ### Minor Changes - Added atomic caller-defined dataset IDs for idempotent dataset creation across built-in storage adapters. Supplying `id` to `mastra.datasets.create()` now creates the dataset once and resolves compatible retries to the persisted record; incompatible immutable identity fields throw `DATASET_ID_CONFLICT`. ([#19370](#19370)) - Added caller-defined dataset item identities for safe retries across all dataset storage adapters. ([#19384](#19384)) Dataset items can now include an `externalId` when calling `addItem` or `addItems`: ```ts await dataset.addItem({ externalId: 'source-item-123', input: { prompt: 'Hello' }, }); ``` Retrying with the same identity and payload returns the existing item. Reusing an identity with different content returns a typed conflict, including during concurrent writes. Updates and deletes preserve the identity, Spanner retries transactions without changing the outcome, and MySQL batch writes now preserve every supported dataset item field. ### Patch Changes - Raised the `@mastra/core` peer dependency floor to `>=1.51.0-0` so the dataset item identity helpers used by the storage adapters are available at runtime. ([#19384](#19384)) - Updated dependencies [[`a99eae8`](a99eae8), [`fd13f8e`](fd13f8e), [`f703f87`](f703f87), [`0ad646f`](0ad646f)]: - @mastra/core@1.51.0-alpha.13 ## @mastra/react@1.2.5-alpha.13 ### Patch Changes - Updated dependencies [[`fd13f8e`](fd13f8e), [`a99eae8`](a99eae8), [`fd13f8e`](fd13f8e), [`f703f87`](f703f87), [`0ad646f`](0ad646f)]: - @mastra/client-js@1.32.0-alpha.13 - @mastra/core@1.51.0-alpha.13 ## @mastra/deployer-cloud@1.51.0-alpha.13 ### Patch Changes - Updated dependencies [[`a99eae8`](a99eae8), [`fd13f8e`](fd13f8e), [`f703f87`](f703f87), [`0ad646f`](0ad646f)]: - @mastra/core@1.51.0-alpha.13 - @mastra/deployer@1.51.0-alpha.13 ## @mastra/longmemeval@1.1.7-alpha.14 ### Patch Changes - Updated dependencies [[`a99eae8`](a99eae8), [`fd13f8e`](fd13f8e), [`f703f87`](f703f87), [`0ad646f`](0ad646f), [`0ad646f`](0ad646f)]: - @mastra/core@1.51.0-alpha.13 - @mastra/libsql@1.16.0-alpha.1 ## @mastra/opencode@0.1.7-alpha.14 ### Patch Changes - Updated dependencies [[`a99eae8`](a99eae8), [`fd13f8e`](fd13f8e), [`f703f87`](f703f87), [`0ad646f`](0ad646f), [`0ad646f`](0ad646f)]: - @mastra/core@1.51.0-alpha.13 - @mastra/libsql@1.16.0-alpha.1 ## mastracode@0.31.0-alpha.14 ### Patch Changes - Updated dependencies [[`a99eae8`](a99eae8), [`fd13f8e`](fd13f8e), [`fd13f8e`](fd13f8e), [`f703f87`](f703f87), [`0ad646f`](0ad646f), [`0ad646f`](0ad646f)]: - @mastra/core@1.51.0-alpha.13 - @mastra/pg@1.16.0-alpha.0 - @mastra/libsql@1.16.0-alpha.1 - @mastra/code-sdk@0.1.0-alpha.13 ## mastra@1.19.0-alpha.14 ### Patch Changes - Updated dependencies [[`a99eae8`](a99eae8), [`fd13f8e`](fd13f8e), [`f703f87`](f703f87), [`0ad646f`](0ad646f)]: - @mastra/core@1.51.0-alpha.13 - @mastra/deployer@1.51.0-alpha.13 ## @mastra/deployer@1.51.0-alpha.13 ### Patch Changes - Updated dependencies [[`a99eae8`](a99eae8), [`fd13f8e`](fd13f8e), [`f703f87`](f703f87), [`0ad646f`](0ad646f)]: - @mastra/core@1.51.0-alpha.13 - @mastra/server@1.51.0-alpha.13 ## @mastra/mcp-docs-server@1.2.7-alpha.23 ### Patch Changes - Updated dependencies [[`a99eae8`](a99eae8), [`fd13f8e`](fd13f8e), [`f703f87`](f703f87), [`0ad646f`](0ad646f)]: - @mastra/core@1.51.0-alpha.13 ## @mastra/playground-ui@41.0.0-alpha.13 ### Patch Changes - Updated dependencies [[`fd13f8e`](fd13f8e), [`a99eae8`](a99eae8), [`fd13f8e`](fd13f8e), [`f703f87`](f703f87), [`0ad646f`](0ad646f)]: - @mastra/client-js@1.32.0-alpha.13 - @mastra/core@1.51.0-alpha.13 - @mastra/react@1.2.5-alpha.13 ## @mastra/express@1.4.7-alpha.13 ### Patch Changes - Updated dependencies [[`a99eae8`](a99eae8), [`fd13f8e`](fd13f8e), [`f703f87`](f703f87), [`0ad646f`](0ad646f)]: - @mastra/core@1.51.0-alpha.13 - @mastra/server@1.51.0-alpha.13 ## @mastra/fastify@1.4.7-alpha.13 ### Patch Changes - Updated dependencies [[`a99eae8`](a99eae8), [`fd13f8e`](fd13f8e), [`f703f87`](f703f87), [`0ad646f`](0ad646f)]: - @mastra/core@1.51.0-alpha.13 - @mastra/server@1.51.0-alpha.13 ## @mastra/hono@1.5.7-alpha.13 ### Patch Changes - Updated dependencies [[`a99eae8`](a99eae8), [`fd13f8e`](fd13f8e), [`f703f87`](f703f87), [`0ad646f`](0ad646f)]: - @mastra/core@1.51.0-alpha.13 - @mastra/server@1.51.0-alpha.13 ## @mastra/koa@1.6.7-alpha.13 ### Patch Changes - Updated dependencies [[`a99eae8`](a99eae8), [`fd13f8e`](fd13f8e), [`f703f87`](f703f87), [`0ad646f`](0ad646f)]: - @mastra/core@1.51.0-alpha.13 - @mastra/server@1.51.0-alpha.13 ## @mastra/nestjs@0.2.7-alpha.13 ### Patch Changes - Updated dependencies [[`a99eae8`](a99eae8), [`fd13f8e`](fd13f8e), [`f703f87`](f703f87), [`0ad646f`](0ad646f)]: - @mastra/core@1.51.0-alpha.13 - @mastra/server@1.51.0-alpha.13 ## @mastra/next@0.2.6-alpha.13 ### Patch Changes - Updated dependencies [[`a99eae8`](a99eae8), [`fd13f8e`](fd13f8e), [`f703f87`](f703f87), [`0ad646f`](0ad646f)]: - @mastra/core@1.51.0-alpha.13 - @mastra/server@1.51.0-alpha.13 - @mastra/hono@1.5.7-alpha.13 ## @mastra/tanstack-start@0.2.6-alpha.13 ### Patch Changes - Updated dependencies [[`a99eae8`](a99eae8), [`fd13f8e`](fd13f8e), [`f703f87`](f703f87), [`0ad646f`](0ad646f)]: - @mastra/core@1.51.0-alpha.13 - @mastra/server@1.51.0-alpha.13 - @mastra/hono@1.5.7-alpha.13 ## @mastra/temporal@0.2.7-alpha.13 ### Patch Changes - Updated dependencies [[`a99eae8`](a99eae8), [`fd13f8e`](fd13f8e), [`f703f87`](f703f87), [`0ad646f`](0ad646f)]: - @mastra/core@1.51.0-alpha.13 - @mastra/deployer@1.51.0-alpha.13 ## create-mastra@1.19.0-alpha.14 ## @internal/playground@1.19.0-alpha.14 ### Patch Changes - Fixed the scorer detail page layout so the Input column no longer stretches far wider than the screen. The column now has a sensible maximum width and truncates long inputs, keeping the table within the viewport. Also fixed the score details pane collapsing to an unusable sliver when opened with the Input column visible — the table now cedes space so the pane keeps its half of the layout. ([#19451](#19451)) - Updated dependencies [[`fd13f8e`](fd13f8e), [`a99eae8`](a99eae8), [`fd13f8e`](fd13f8e), [`f703f87`](f703f87), [`0ad646f`](0ad646f)]: - @mastra/client-js@1.32.0-alpha.13 - @mastra/core@1.51.0-alpha.13 - @mastra/react@1.2.5-alpha.13 - @mastra/playground-ui@41.0.0-alpha.13 Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Allows callers to provide a durable dataset ID through the existing dataset creation API. Storage adapters atomically insert the dataset or return the compatible record that already owns the ID, so concurrent retries converge without a separate resolution API.
Reusing an ID with different immutable tenancy or candidate identity fields now throws
DATASET_ID_CONFLICT. Mutable fields are neither compared nor overwritten, ordinary generated-ID creation is unchanged, and deleting a dataset releases its ID for a fresh generation.The shared storage contract covers concurrent creation, compatible retries, null/undefined normalization, immutable conflicts, and delete/recreate behavior across the built-in adapters.
Verified with the core dataset suite, core typecheck, adapter builds and lint, and the full LibSQL integration suite. PostgreSQL, MySQL, MongoDB, and Spanner integration tests require a running Docker daemon and were not available locally.
ELI5
You can now choose the “dataset ID” when creating a dataset. If you accidentally create the same dataset again with the same ID, Mastra will just give you back the one it already saved—unless that same ID is being used for a different (immutable) owner/identity, in which case it stops and reports a conflict.
Summary
mastra.datasets.create()/DatasetsManager.createto accept an optional caller-provided durableidfor idempotent dataset creation.CreateDatasetInputso storage adapters atomically insert-or-return the existing dataset whenidis provided (no separate resolution API needed).idvalues (DATASET_INVALID_ID) and immutable conflict handling when reusing an ID with different immutable tenancy/identity fields, throwingDATASET_ID_CONFLICT(mutable fields are not compared/overwritten).idand resolving it via the shared immutable-field logic.insertOnly/prepareInsertOnlyStatement) to decouple ID-conflict handling from upsert SQL formatting.@mastra/core >= 1.51.0-0.