Skip to content

feat(observability): route eval scores through ScoreEvent bus#16185

Merged
intojhanurag merged 17 commits into
mastra-ai:mainfrom
intojhanurag:feat/observability-score-events
May 8, 2026
Merged

feat(observability): route eval scores through ScoreEvent bus#16185
intojhanurag merged 17 commits into
mastra-ai:mainfrom
intojhanurag:feat/observability-score-events

Conversation

@intojhanurag

@intojhanurag intojhanurag commented May 4, 2026

Copy link
Copy Markdown
Contributor

Description

Wires Mastra eval scores through the unified observability bus so every exporter implementing onScoreEvent automatically receives them. Replaces the deprecated per-exporter addScoreToTrace side-channel that bypassed the pipeline. Also unblocks Mastra spans appearing under Braintrust experiment eval spans (#11097) as a side effect of the unified path.

  • Producer alignment: packages/core/src/mastra/hooks.ts no longer iterates exporters with addScoreToTrace. MastraScorer.run() (already on main from PR Add scorer tracing and export scores through the observability bus #14920) remains the single producer of ScoreEvents through mastra.observability.addScore().
  • Consumer side: implemented onScoreEvent on Langfuse (LangfuseClient.score.create), Laminar (/v1/evaluators/score), Braintrust (logger.logFeedback), Datadog (tracer.llmobs.submitEvaluation), and LangSmith (Client.createFeedback).
  • Backwards compatibility: Langfuse addScoreToTrace and Laminar _addScoreToTrace are preserved as @deprecated wrappers that forward to a shared private submitScore helper.
  • Bug fix in observability/mastra/src/recorded.ts: buildScoreEvent now forwards scorerName and targetEntityType from ScoreInput to ExportedScore (without this, exporters silently fell back to scorerId).
  • Tests: rewrote core hook test to assert the hook no longer publishes a ScoreEvent; added per-exporter onScoreEvent tests; added a regression test for buildScoreEvent field forwarding.
  • Seven single-package changesets created (@mastra/core, @mastra/observability, @mastra/langfuse, @mastra/laminar, @mastra/braintrust, @mastra/datadog, @mastra/langsmith) per the project's changeset style guide.

Related Issue(s)

Fixes #10896 (also unblocks #11097)

Type of Change

  • Bug fix (non-breaking change that fixes an issue)
  • New feature (non-breaking change that adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Documentation update
  • Code refactoring
  • Performance improvement
  • Test update

Checklist

  • I have made corresponding changes to the documentation (if applicable)
  • I have added tests that prove my fix is effective or that my feature works
  • I have addressed all Coderabbit comments on this PR

ELI5

When Mastra assigns an evaluation score, this PR sends that score exactly once into a single observability "bus" so all monitoring/exporter integrations (Langfuse, Laminar, Braintrust, Datadog, LangSmith, etc.) automatically receive it — avoiding exporter-specific wiring and duplicate deliveries.

Overview

Routes Mastra evaluation scores through the unified observability ScoreEvent bus (mastra.observability.addScore) so any exporter implementing onScoreEvent receives them. MastraScorer.run() remains the single producer of ScoreEvent; the core scorer hook no longer iterates exporters to call addScoreToTrace. Per-exporter addScoreToTrace/_addScoreToTrace behaviors are preserved as @deprecated wrappers that forward to shared submit helpers where applicable. Fixes a bug in buildScoreEvent so scorerName and targetEntityType from ScoreInput are forwarded into emitted ExportedScore.

Changes by Component

Core (@mastra/core)

  • Removed exporter-iteration and addScoreToTrace republishing from createOnScorerHook; the hook now only persists scores to the legacy scores store.
  • Ensures MastraScorer.run() is the single producer of ScoreEvent.
  • Registers static workflow-step scorers synchronously during addWorkflow and resolves dynamic scorers asynchronously so scorers are registered before use.
  • Updated core test to assert the hook does not emit ScoreEvent.

Observability (@mastra/observability)

  • Fix: buildScoreEvent (observability/mastra/src/recorded.ts) now includes scorerName and targetEntityType in emitted ScoreEvent.
  • Bus routing: route-event now prefers onScoreEvent; if an exporter lacks onScoreEvent but implements deprecated addScoreToTrace, the bus maps the ScoreEvent into the legacy addScoreToTrace payload and calls it (avoids duplicate delivery when both exist).
  • Added unit tests for score routing behavior and the buildScoreEvent regression.

Exporter Integrations (new onScoreEvent handlers + tests)

  • Langfuse (@mastra/langfuse)

    • Added onScoreEvent → LangfuseClient.score.create with a shared submitScore helper and centralized error logging.
    • Preserved addScoreToTrace as a deprecated wrapper forwarding to submitScore.
    • Tests updated to cover onScoreEvent and mark addScoreToTrace deprecated.
  • Laminar (@mastra/laminar)

    • Added onScoreEvent delegating to a private submitScore that POSTS to /v1/evaluators/score.
    • submitScore converts trace/span IDs to Laminar/OTel UUIDs, merges metadata/reason, enforces timeouts via AbortController, and logs non-OK/timed-out responses.
    • Preserved _addScoreToTrace as a @deprecated wrapper forwarding to submitScore.
    • Tests verify POST payload mapping and wrapper behavior.
  • Braintrust (@mastra/braintrust)

    • Added onScoreEvent that calls logger.logFeedback using spanId (falling back to traceId) and skips when neither is present.
    • Maps scorer name/ID, reason, and metadata into the feedback payload and logs failures.
    • Tests cover span/trace fallback and skip behavior.
  • Datadog (@mastra/datadog)

    • Added onScoreEvent: validates traceId/spanId, looks up exported dd-trace span context (traceState), and calls tracer.llmobs.submitEvaluation with mapped fields (label/value/metricType/mlApp/timestamp/reason/metadata).
    • Drops scores when identifiers or exported span context are missing and warns when scores arrive before the span is exported.
    • Tests cover successful submission, unknown IDs (no-op), and dropping out-of-order scores.
  • LangSmith (@mastra/langsmith)

    • Added onScoreEvent that submits Client.createFeedback when a spanId→LangSmith runId mapping exists.
    • Introduced a configurable bounded LRU spanId→runId cache (runIdCacheMaxEntries) populated from RunTree building; evicts oldest entries when full and is cleared on shutdown.
    • Ensures authoritative scorerId/scoreSource are preserved in sourceInfo and ignores events when span/run mapping is absent.
    • Tests cover mapping semantics, eviction, metadata protection, and skipping unseen spans.

Tests

  • Core hook test rewritten to assert the hook does not publish ScoreEvent.
  • Per-exporter onScoreEvent test suites added for Langfuse, Laminar, Braintrust, Datadog, and LangSmith covering field mapping, identifier fallbacks, ordering/dropping, timeout handling, and cache eviction.
  • Bus tests added to assert onScoreEvent preferred and legacy addScoreToTrace fallback behavior.
  • Regression test added to ensure buildScoreEvent forwards scorerName and targetEntityType.

Backwards Compatibility

  • Deprecated addScoreToTrace/_addScoreToTrace preserved (where present) as wrappers that forward to the new onScoreEvent/submit helpers to avoid breaking existing callers.
  • Observability interface docs updated to deprecate addScoreToTrace in favor of onScoreEvent/ScoreEvent bus.
  • No breaking public API surface changes beyond documented additions and deprecations.

Commits & Release Notes

  • Seven package-scoped changesets created for @mastra/core, @mastra/observability, @mastra/langfuse, @mastra/laminar, @mastra/langsmith, @mastra/datadog, and @mastra/braintrust documenting onScoreEvent support and the buildScoreEvent fix.
  • Exporter-specific notes documented (Datadog ordering caveat; Laminar timeout/error logging; LangSmith runId cache behavior).

Related Issues

Fixes #10896 and unblocks #11097.

Implements mastra-ai#10896. Removes the deprecated per-exporter addScoreToTrace
side-channel from the scorer hook so MastraScorer.run() (already on main)
remains the sole producer of ScoreEvents. Adds onScoreEvent on Langfuse,
Laminar, Braintrust, Datadog, and LangSmith so eval scores reach each
backend through the unified observability bus. Also unblocks mastra-ai#11097.

- Fixes buildScoreEvent to forward scorerName and targetEntityType from
  ScoreInput onto ExportedScore.
- Preserves Langfuse addScoreToTrace and Laminar _addScoreToTrace as
  @deprecated wrappers for backwards compatibility.
- One single-package changeset per touched workspace.
@vercel

vercel Bot commented May 4, 2026

Copy link
Copy Markdown

@intojhanurag is attempting to deploy a commit to the Mastra Team on Vercel.

A member of the Team first needs to authorize it.

@changeset-bot

changeset-bot Bot commented May 4, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 2503c14

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 35 packages
Name Type
@mastra/langfuse Minor
@mastra/core Minor
@mastra/langsmith Minor
@mastra/laminar Minor
@mastra/observability Patch
@mastra/braintrust Minor
@mastra/datadog Minor
mastracode Patch
@mastra/mcp-docs-server Patch
@internal/playground Patch
@mastra/client-js Patch
@mastra/opencode Patch
@mastra/longmemeval Patch
@mastra/express Patch
@mastra/fastify Patch
@mastra/hono Patch
@mastra/koa Patch
@mastra/nestjs Patch
@mastra/otel-bridge Patch
@mastra/otel-exporter Patch
@mastra/posthog Patch
@mastra/sentry Patch
mastra Patch
@mastra/deployer-cloud Minor
@mastra/deployer-vercel Patch
@mastra/playground-ui Patch
@mastra/react Patch
@mastra/arize Patch
@mastra/arthur Patch
@mastra/server Minor
@mastra/deployer Minor
create-mastra Patch
@mastra/deployer-cloudflare Patch
@mastra/deployer-netlify Patch
@mastra/temporal Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@coderabbitai

coderabbitai Bot commented May 4, 2026

Copy link
Copy Markdown
Contributor

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review

Walkthrough

Centralizes evaluator score emission onto a ScoreEvent bus: scorer runs call mastra.observability.addScore(), buildScoreEvent now includes scorerName and targetEntityType, hooks no longer republish scores to exporters, and exporters implement new onScoreEvent handlers while preserving deprecated addScoreToTrace/_addScoreToTrace wrappers.

Changes

Unified Score Event Pipeline

Layer / File(s) Summary
Data Shape
observability/mastra/src/recorded.ts, observability/mastra/src/tracing.test.ts, packages/core/src/storage/domains/observability/record-builders.ts, .../record-builders.test.ts
buildScoreEvent now forwards scorerName and targetEntityType; persisted score records gain top-level scorerName (removed from metadata); tests updated.
Core hook & registration
packages/core/src/mastra/hooks.ts, packages/core/src/mastra/hook.test.ts, packages/core/src/mastra/index.ts, packages/core/src/mastra/scorer-registration.test.ts, packages/core/src/workflows/handlers/step.ts
Removed legacy exporter republishing from scorer hook; hook now only saves to legacy scores store. Workflow registration synchronously registers static step scorers and asynchronously lists dynamic scorers; scorer execution registers scorers with engine.mastra; tests added for registration timing and hook behavior.
Score record shape / storage
packages/core/src/storage/domains/observability/record-builders.ts, .../record-builders.test.ts
Score records now include top-level `scorerName: string
Bus routing / backward-compat
observability/mastra/src/bus/route-event.ts, observability/mastra/src/bus/observability-bus.test.ts
ScoreEvent routing prefers onScoreEvent; when absent, routes to legacy addScoreToTrace with mapped fields; tests cover precedence and fallback.
LangSmith exporter
observability/langsmith/src/tracing.ts, observability/langsmith/src/tracing.test.ts
Added bounded LRU spanId → langSmith runId cache with config runIdCacheMaxEntries; populate on RunTree build; implement onScoreEvent to createFeedback when mapping exists; clear cache on shutdown; tests for happy path, missing spans, and cache eviction.
Langfuse exporter
observability/langfuse/src/tracing.ts, observability/langfuse/src/tracing.test.ts
Added private submitScore() and public onScoreEvent(event: ScoreEvent) delegating to it; refactored addScoreToTrace to call submitScore (deprecated wrapper); tests renamed/added for deprecated and new behavior.
Laminar exporter
observability/laminar/src/tracing.ts, observability/laminar/src/tracing.test.ts
Introduced submitScore() helper (builds payload, converts IDs, enforces timeout, logs non-OK/errors); added onScoreEvent delegating to it; replaced _addScoreToTrace with deprecated wrapper; tests validate HTTP payload and UUID conversion.
Datadog exporter
observability/datadog/src/tracing.ts, observability/datadog/src/tracing.test.ts
Added onScoreEvent(event: ScoreEvent) that validates traceId/spanId, looks up exported dd-trace context, and uses tracer.llmobs.submitEvaluation with mapped options; tests cover exported-span lookup, unknown spans, and early-event dropping.
Braintrust exporter
observability/braintrust/src/tracing.ts, observability/braintrust/src/tracing.test.ts
Added onScoreEvent(event: ScoreEvent) to call logger.logFeedback using spanId or traceId as id; skips when identifiers are missing; tests cover id selection and no-op behavior.
Changesets / releases & docs
.changeset/*, docs/src/content/en/reference/observability/tracing/interfaces.mdx
Added/updated changeset entries documenting minor/patch releases across @mastra/* packages describing the new onScoreEvent integrations and the buildScoreEvent fix; docs annotated addScoreToTrace as deprecated.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~55 minutes

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 40.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The PR title clearly describes the main change: routing eval scores through the ScoreEvent bus. It is concise, specific, and uses imperative mood with proper capitalization.
Linked Issues check ✅ Passed The PR fully addresses issue #10896 by implementing eval scoring support across all exporters through the unified observability pipeline via ScoreEvent bus and onScoreEvent handlers.
Out of Scope Changes check ✅ Passed All changes are directly related to implementing eval score routing: exporter onScoreEvent implementations, ScoreEvent bus routing, deprecated addScoreToTrace wrappers, scorer registration for workflows, and supporting infrastructure. No unrelated changes detected.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Tip

💬 Introducing Slack Agent: The best way for teams to turn conversations into code.

Slack Agent is built on CodeRabbit's deep understanding of your code, so your team can collaborate across the entire SDLC without losing context.

  • Generate code and open pull requests
  • Plan features and break down work
  • Investigate incidents and troubleshoot customer tickets together
  • Automate recurring tasks and respond to alerts with triggers
  • Summarize progress and report instantly

Built for teams:

  • Shared memory across your entire org—no repeating context
  • Per-thread sandboxes to safely plan and execute work
  • Governance built-in—scoped access, auditability, and budget controls

One agent for your entire SDLC. Right inside Slack.

👉 Get started


Comment @coderabbitai help to get the list of available commands and usage tips.

@intojhanurag
intojhanurag marked this pull request as draft May 4, 2026 19:06

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 6

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In @.changeset/five-stars-smile.md:
- Around line 5-38: The changeset text currently includes exporter-side
migration details (mentions of onScoreEvent, addScoreToTrace deprecation, and
usage examples) that apply beyond `@mastra/core`; edit the
.changeset/five-stars-smile.md content so it only describes the core change
(MastraScorer.run() now producing score events via
mastra.observability.addScore() and that ScoreEvent is the unified pipeline) and
remove exporter migration examples and deprecation notes (onScoreEvent,
addScoreToTrace usage) from this file; if you want to keep exporter migration
guidance, create separate changeset(s) in the exporter package(s) with those
details and ensure the frontmatter only lists `@mastra/core`.

In @.changeset/giant-apes-accept.md:
- Line 5: Rewrite the changeset line to state the user-facing outcome instead of
implementation details: replace the sentence that mentions onScoreEvent,
mastra.observability.addScore(), and Client.createFeedback with a short
description of what `@mastra/langsmith` users will experience (e.g., "Evaluation
scores are automatically submitted to LangSmith and associated with the
corresponding run for easier tracking and review"), keeping the focus on the
product behavior and benefit rather than the internal function names.

In @.changeset/ready-socks-leave.md:
- Line 5: Edit the changeset description so it only refers to the package listed
in the frontmatter (`@mastra/observability`): remove or reword any mentions of
other exporter packages (Langfuse, Braintrust, LangSmith, Laminar) and the
broader behavior implications, and instead state the scoped change (forwarding
scorerName and targetEntityType onto ExportedScore) as applying solely to
`@mastra/observability`; keep the technical detail but restrict the impact
language to that single package.

In `@observability/datadog/src/tracing.test.ts`:
- Around line 1575-1620: The test suite is missing a case where a score arrives
before SPAN_ENDED and currently onScoreEvent drops such scores because it checks
this.traceState.get(score.traceId)?.contexts.get(score.spanId) and returns if
not exported; add a unit test that calls exporter.onScoreEvent(...) with a score
for an active span before emitting a SPAN_ENDED (use
createMockSpan/createTracingEvent to create the span), assert the current
behavior fails, then choose one remediation: either implement buffering inside
onScoreEvent (store pending scores keyed by traceId/spanId and flush them when
the SPAN_ENDED path sets exported in traceState) or add a clear comment and
runtime warning in the exporter code documenting that scores must only be sent
after SPAN_ENDED (update onScoreEvent to log/throw when a score is received for
an unexported span); update tests to reflect the chosen behavior.

In `@observability/laminar/src/tracing.ts`:
- Around line 392-396: The POST to /v1/evaluators/score should use an
AbortSignal with the existing timeoutMillis to avoid hanging; update the code
around the fetch call in tracing.ts (where score is submitted) to create an
AbortController, pass controller.signal into fetch options, start a setTimeout
that calls controller.abort() after timeoutMillis (or
this.config.timeoutMillis), and clear the timer once fetch resolves or throws so
you don't leak timers; ensure any AbortError is handled/propagated consistently
by the surrounding function (e.g., the score submission method).

In `@observability/langsmith/src/tracing.test.ts`:
- Around line 1410-1451: The test reveals feedback uses Mastra span/trace IDs
that LangSmith doesn't know about; update buildRunTreePayload and span creation
so LangSmith run IDs are preserved or recorded, and make onScoreEvent look up
the LangSmith runId before calling client.createFeedback. Specifically, either
include span.id as the id field when instantiating new RunTree (and when calling
createChild) so LangSmith uses the Mastra span.id as its runId, or modify the
span creation flow to capture and store the LangSmith-allocated runId into
traceData (e.g., traceData[traceId].runId or traceData.spanRunIds[spanId]) and
then change onScoreEvent to read that stored LangSmith runId (instead of using
spanId/traceId directly) before invoking mockClient.createFeedback /
client.createFeedback.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 43c69fbf-759b-4c7e-87cf-87b800c6ca89

📥 Commits

Reviewing files that changed from the base of the PR and between 7679a63 and f6f07f4.

📒 Files selected for processing (21)
  • .changeset/dark-rooms-build.md
  • .changeset/five-stars-smile.md
  • .changeset/giant-apes-accept.md
  • .changeset/large-yaks-battle.md
  • .changeset/ready-socks-leave.md
  • .changeset/tiny-doodles-hang.md
  • .changeset/violet-rules-design.md
  • observability/braintrust/src/tracing.test.ts
  • observability/braintrust/src/tracing.ts
  • observability/datadog/src/tracing.test.ts
  • observability/datadog/src/tracing.ts
  • observability/laminar/src/tracing.test.ts
  • observability/laminar/src/tracing.ts
  • observability/langfuse/src/tracing.test.ts
  • observability/langfuse/src/tracing.ts
  • observability/langsmith/src/tracing.test.ts
  • observability/langsmith/src/tracing.ts
  • observability/mastra/src/recorded.ts
  • observability/mastra/src/tracing.test.ts
  • packages/core/src/mastra/hook.test.ts
  • packages/core/src/mastra/hooks.ts

Comment thread .changeset/five-stars-smile.md Outdated
Comment thread .changeset/giant-apes-accept.md Outdated
Comment thread .changeset/ready-socks-leave.md Outdated
Comment thread observability/datadog/src/tracing.test.ts
Comment thread observability/laminar/src/tracing.ts
Comment thread observability/langsmith/src/tracing.test.ts Outdated
Per review: exporter-side migration guidance (onScoreEvent example,
addScoreToTrace deprecation) belongs in the per-exporter changesets,
not the @mastra/core entry.
Per review: replaced implementation-detail wording (onScoreEvent,
mastra.observability.addScore, Client.createFeedback) with a description
of the product behavior — eval scores now appear in the LangSmith UI
attached to the matching run.
Per review: removed cross-package impact language (Langfuse, Braintrust,
LangSmith, Laminar) from the @mastra/observability entry. The technical
detail about scorerName/targetEntityType being forwarded is preserved;
downstream exporter behavior is described in their own changesets.
…an_ended

Datadog's onScoreEvent silently dropped scores when the target span had
not yet been emitted to dd-trace (i.e. before its SPAN_ENDED was
processed and the buffered trace tree was flushed). On Mastra's normal
scoring path this never fires — scorer hooks run after the scored
entity completes — but a manual addScore() for an in-progress span
could trigger it.

- Promoted the silent debug log to a warning so the misuse is observable.
- Documented the ordering constraint on the JSDoc.
- Added a unit test asserting the behavior for the SPAN_STARTED-only case.
The POST to /v1/evaluators/score had no abort signal, so a hung
connection or unresponsive server could block indefinitely while the
exporter already supports a configured timeoutMillis for OTLP.

- Wired the existing this.config.timeoutMillis into the score fetch via
  AbortController + setTimeout.
- Cleared the timer in a finally block so it never leaks on success or
  error.
- AbortError is surfaced as `timedOut: true` in the existing error log
  so timeouts are distinguishable from other failures.
The previous onScoreEvent passed Mastra's span.id as runId to
client.createFeedback, but LangSmith assigns its own UUID to each
RunTree on construction (see langsmith run_trees.js: when id is omitted,
it auto-generates via uuid7FromTime). Mastra's span.id was never known
to LangSmith, so feedback would not match the run.

- Capture langSmithSpan.id into a Map<spanId, runId> inside _buildSpan,
  populated whenever a RunTree is created (or a child span is added).
- onScoreEvent now reads that map to resolve the LangSmith runId and
  uses it for createFeedback.
- Trace-level scoring (no spanId) is skipped with a warning until we
  build a proper trace-level mapping.
- Map is cleared in _postShutdown so long-running processes don't
  accumulate entries.
- Updated tests: mockRunTree now exposes an id; tests assert the
  LangSmith runId is forwarded, not the Mastra spanId; added a test
  for unseen spans.
@intojhanurag
intojhanurag marked this pull request as ready for review May 5, 2026 11:07
@intojhanurag
intojhanurag marked this pull request as draft May 5, 2026 11:07

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In @.changeset/five-stars-smile.md:
- Around line 5-7: Rewrite the changeset entry to lead with the user-facing
outcome: explain that scores are now emitted once via the unified observability
pipeline (mastra.observability.addScore()) and therefore exporters no longer
receive duplicate deliveries; remove implementation-focused references to
createOnScorerHook and the exporter iteration, but keep a short parenthetical
note that MastraScorer.run() is the single source of score events if extra
clarity is needed.

In `@observability/langsmith/src/tracing.ts`:
- Line 74: The private Map field `#langsmithRunIdBySpanId` (and the other similar
maps referenced) is unbounded and can cause memory leaks; replace it with a
bounded cache (e.g., an LRU or size-limited Map with eviction) or add TTL
eviction to cap growth, and update all usages (put/get/delete in the methods
that reference `#langsmithRunIdBySpanId` and the other two maps mentioned) to use
the new cache API so entries are evicted automatically when the capacity/age
limit is reached; ensure construction of the cache is configurable (max size /
TTL) so tests or shutdown logic can still clear it explicitly.
- Around line 142-146: The sourceInfo construction currently spreads
score.metadata last which allows metadata to overwrite reserved fields; in
tracing.ts change the order or sanitize metadata so reserved keys remain
authoritative — either spread ...(score.metadata ?? {}) first and then add
scorerId: score.scorerId and ...(score.scoreSource ? { scoreSource:
score.scoreSource } : {}), or prefilter score.metadata to remove "scorerId" and
"scoreSource" before spreading; update the sourceInfo creation to use one of
these approaches so scorerId and scoreSource in sourceInfo cannot be
overwritten.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: bc1f59ce-f282-4965-aaf6-4ed680440413

📥 Commits

Reviewing files that changed from the base of the PR and between f6f07f4 and 7d3db72.

📒 Files selected for processing (8)
  • .changeset/five-stars-smile.md
  • .changeset/giant-apes-accept.md
  • .changeset/ready-socks-leave.md
  • observability/datadog/src/tracing.test.ts
  • observability/datadog/src/tracing.ts
  • observability/laminar/src/tracing.ts
  • observability/langsmith/src/tracing.test.ts
  • observability/langsmith/src/tracing.ts
✅ Files skipped from review due to trivial changes (3)
  • .changeset/giant-apes-accept.md
  • observability/datadog/src/tracing.test.ts
  • observability/langsmith/src/tracing.test.ts
🚧 Files skipped from review as they are similar to previous changes (2)
  • .changeset/ready-socks-leave.md
  • observability/laminar/src/tracing.ts

Comment thread .changeset/five-stars-smile.md Outdated
Comment thread observability/langsmith/src/tracing.ts
Comment thread observability/langsmith/src/tracing.ts
Per review: rewrote the entry to lead with what users see (scores
emitted once, no duplicate exporter deliveries) and dropped the
implementation walkthrough of createOnScorerHook and the legacy
exporter iteration. Kept a one-line parenthetical noting that
MastraScorer.run() is the single source for clarity.
The spanId → langsmithRunId map was unbounded — only cleared on
shutdown — which would leak memory in long-running processes
emitting many spans.

- Replaced the plain Map access with two private helpers that
  implement an LRU policy (re-insert on read, FIFO-evict the oldest
  entry when size exceeds the cap).
- Cap defaults to 10000 entries (~1 MB). Configurable per-instance
  via the new LangSmithExporterConfig.runIdCacheMaxEntries option,
  primarily for tests.
- _postShutdown still clears the map.
- Added a test that drives a 2-entry cap and asserts the oldest
  entry is evicted when a third span is registered.
The sourceInfo object spread score.metadata last, so a user could
clobber the authoritative scorerId / scoreSource fields by putting
keys of those names inside metadata.

- Reordered the sourceInfo construction so user metadata is spread
  first; the reserved scorerId and scoreSource fields are added
  afterwards and therefore win.
- Added a regression test that passes evil { scorerId: 'evil',
  scoreSource: 'evil' } in metadata and asserts the authoritative
  values survive.
@intojhanurag
intojhanurag marked this pull request as ready for review May 5, 2026 11:27
@intojhanurag
intojhanurag marked this pull request as draft May 5, 2026 11:27
…data

Found during Phase 1A E2E testing of PR mastra-ai#16185.

The score record schema (scoreRecordSchema) already declares a top-level
scorerName column. buildScoreRecord was working around an older gap
where ScoreEvent did not forward scorerName, so it stuffed scorerName
into metadata as a fallback. With buildScoreEvent now forwarding
scorerName onto ExportedScore (earlier commit on this branch), the
workaround is wrong — scorerName should land on its own column so
queries and UIs can read it directly.

- buildScoreRecord now writes scorerName: s.scorerName ?? null and
  passes user metadata through unchanged.
- Updated record-builder tests that asserted the old metadata-pollution
  shape.

Verified end-to-end via .score-event-test/phase1a-direct-addScore.ts
against InMemoryStore: scorerName lands as a top-level field, metadata
is clean.
Found during Phase 10 E2E testing of PR mastra-ai#16185.

Mastra already walks agent scorers and calls __registerMastra on each
(see addAgent path around line 1690). The same registration was missing
for workflow step scorers. As a result, MastraScorer.run() never saw a
Mastra instance and the score event was never published to the bus —
exporters didn't receive scores from workflow step scorers via
onScoreEvent at all (the legacy validateAndSaveScore path still wrote
to the scores table because it doesn't gate on __registerMastra).

- addWorkflow now calls workflow.listScorers() and registers each
  step-level scorer via addScorer, mirroring the agent path.
- Verified end-to-end: workflow step scorer now produces one row in the
  observability scores domain with entityType: 'workflow_step',
  scorerName populated, score value correct.
…er interface

Found during Phase 11 docs audit of PR mastra-ai#16185.

The reference page for observability interfaces listed addScoreToTrace
without any deprecation notice, while the runtime path has been
deprecated in favor of onScoreEvent (already documented elsewhere in the
same file). Added a JSDoc-style @deprecated note pointing readers at
onScoreEvent and explaining that the method is kept on the interface
for backwards compatibility only.
@github-actions github-actions Bot added the Documentation Improvements or additions to documentation label May 6, 2026
Per review: the PR had three separate @mastra/core changesets — one
minor for the main producer-side alignment and two patches for the
follow-on fixes (scorerName-as-its-own-column, workflow-step-scorer
registration). Three changesets for one package in a single release
clutters the changelog and is unusual for one PR.

- Folded the scorerName and workflow-step-scorer fixes into the main
  five-stars-smile.md @mastra/core entry as bullet points.
- Removed score-name-top-level-column.md.
- Removed workflow-step-scorers-auto-register.md.

Bump level stays minor (the highest of the three).
@intojhanurag
intojhanurag marked this pull request as ready for review May 6, 2026 11:01
@intojhanurag
intojhanurag marked this pull request as draft May 6, 2026 16:54
Route ScoreEvent payloads to deprecated addScoreToTrace only when an exporter does not implement onScoreEvent, avoiding duplicate score delivery while preserving old exporter integrations.

Register workflow step scorers synchronously for static configs and at execution time for dynamic configs so workflow scorer publishes have a Mastra instance before running.
@intojhanurag
intojhanurag marked this pull request as ready for review May 7, 2026 09:37

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
observability/mastra/src/bus/observability-bus.test.ts (1)

265-281: ⚡ Quick win

Strengthen fallback payload coverage with explicit non-default fields.

This test validates fallback dispatch, but it currently uses default score fixtures where scorerName, spanId, and metadata are unset. Setting them explicitly here would better protect against mapping regressions.

Proposed test hardening
-      const event = createScoreEvent();
+      const event: ScoreEvent = {
+        type: 'score',
+        score: {
+          ...createScoreEvent().score,
+          spanId: 'span-legacy-1',
+          scorerName: 'relevance-v2',
+          metadata: { source: 'eval-suite' },
+        },
+      };
       bus.emit(event);

       expect(addScoreToTrace).toHaveBeenCalledWith({
         traceId: event.score.traceId,
         spanId: event.score.spanId,
         score: event.score.score,
         reason: event.score.reason,
         scorerName: event.score.scorerName ?? event.score.scorerId,
         metadata: event.score.metadata,
       });
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@observability/mastra/src/bus/observability-bus.test.ts` around lines 265 -
281, The test "should fall back to deprecated addScoreToTrace for exporters
without onScoreEvent" uses default fixture values so update the created event
from createScoreEvent() to set explicit non-default fields (e.g., provide a
concrete scorerName, a non-empty spanId, and non-empty metadata) before calling
bus.emit(event); then assert addScoreToTrace was called with the mapped payload
containing those explicit values (ensure the scorerName vs scorerId selection
logic is exercised) so the fallback mapping in
createMockExporter/addScoreToTrace is robustly covered.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@observability/mastra/src/bus/observability-bus.test.ts`:
- Around line 265-281: The test "should fall back to deprecated addScoreToTrace
for exporters without onScoreEvent" uses default fixture values so update the
created event from createScoreEvent() to set explicit non-default fields (e.g.,
provide a concrete scorerName, a non-empty spanId, and non-empty metadata)
before calling bus.emit(event); then assert addScoreToTrace was called with the
mapped payload containing those explicit values (ensure the scorerName vs
scorerId selection logic is exercised) so the fallback mapping in
createMockExporter/addScoreToTrace is robustly covered.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: c13bcff5-6a1b-4d96-b2e4-bf73b286b8c5

📥 Commits

Reviewing files that changed from the base of the PR and between b62e978 and 1183a1f.

📒 Files selected for processing (6)
  • .changeset/ready-socks-leave.md
  • observability/mastra/src/bus/observability-bus.test.ts
  • observability/mastra/src/bus/route-event.ts
  • packages/core/src/mastra/index.ts
  • packages/core/src/mastra/scorer-registration.test.ts
  • packages/core/src/workflows/handlers/step.ts
🚧 Files skipped from review as they are similar to previous changes (2)
  • .changeset/ready-socks-leave.md
  • packages/core/src/mastra/index.ts

Comment thread .changeset/giant-apes-accept.md Outdated
Comment thread packages/core/src/mastra/index.ts Outdated
@intojhanurag
intojhanurag merged commit 7b0ad1f into mastra-ai:main May 8, 2026
44 of 46 checks passed
@intojhanurag
intojhanurag deleted the feat/observability-score-events branch May 8, 2026 14:39
abhiaiyer91 pushed a commit that referenced this pull request May 10, 2026
## Description

Registers workflow step scorers with the owning Mastra instance so
workflow scorer runs can emit scores through the observability score
pipeline.

This keeps static workflow scorers discoverable after workflow
registration and registers dynamic workflow scorers at execution time,
when request context is available.

## Related Issue(s)

Follow-up to #16185.

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Code refactoring
- [ ] Performance improvement
- [x] Test update

## Checklist

- [x] I have made corresponding changes to the documentation (if
applicable)
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] I have addressed all Coderabbit comments on this PR

## Testing

- pnpm --filter ./packages/core test
src/mastra/scorer-registration.test.ts
- pnpm --filter ./packages/core check
- git diff --check

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## ELI5

Workflow step scorers are little checks that rate each step of a
process. This PR makes sure those checkers are registered with Mastra
(either when the workflow is added or at execution time) so their scores
are emitted into the observability pipeline and tracked.

## Changes

**Changeset Addition** (`.changeset/workflow-scorers-register.md`)
- Adds a changelog entry (minor) for `@mastra/core` noting that
configured workflow step scorers now automatically emit scores through
the observability score pipeline.

**Core Registration Logic** (`packages/core/src/mastra/index.ts`)
- Added a private helper to register static workflow step scorers found
on workflow steps.
- `Mastra.addWorkflow()` now invokes that helper after
committing/registering a workflow so static (non-function)
`step.scorers` are registered with source `'code'`.
- Removed the previous declarative schedule auto-registration
side-effect (it no longer sets `#hasScheduledWorkflow` or triggers
scheduler bootstrap/registration based on collected schedules).
- Documented `addScorer()` behavior regarding duplicate keys.

**Workflow Step Execution**
(`packages/core/src/workflows/handlers/step.ts`)
- `runScorersForStep()` now ensures dynamic scorers are registered with
`engine.mastra` before execution by invoking the scorer's internal
register helper and calling `mastra.addScorer(...)`.
- When emitting scores, scorer identification now uses
`scorerObject.scorer.id` (instead of the prior `scorerObject.name`).

**Tests** (`packages/core/src/mastra/scorer-registration.test.ts`)
- Expanded tests to cover workflow step scorer registration (static and
dynamic) in addition to existing agent-level tests.
- New/updated test cases verify:
- Static workflow step scorers are registered after `addWorkflow()`
returns and are findable via `getScorerById`.
  - Shared static scorers are not duplicated (one registration per id).
  - Dynamic workflow step scorers are registered before they run.
- Score events are emitted through `mastra.observability.addScore` with
expected scorer id/name, `targetEntityType: 'workflow_run'`, and numeric
score values.

## Impact

Fixes a bug where workflow step scorers (both static and dynamic) could
be undiscoverable by Mastra; ensures they are registered so their
emitted scores flow through the observability score pipeline and are
captured for reporting and analysis.

[![Review Change
Stack](https://storage.googleapis.com/coderabbit_public_assets/review-stack-in-coderabbit-ui.svg)](https://app.coderabbit.ai/change-stack/mastra-ai/mastra/pull/16371)
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
mbenhamd added a commit to mbenhamd/mastra that referenced this pull request May 14, 2026
* chore: regenerate providers and docs [skip ci]

* fix(playground): stabilize streaming message rendering (#16331)

This fixes a couple Studio rendering issues that showed up during fast
streaming output.

Reasoning chunks now stay separated when a model interleaves reasoning,
tool calls, and text instead of appending everything back into the first
reasoning block.

Before:
```tsx
reasoning: "first thought second thought third thought"
tool call
text
```

After:
```tsx
reasoning: "first thought"
tool call
reasoning: "second thought"
text
reasoning: "third thought"
```

Autoscroll also keeps following rapid output from parallel tool calls
unless the user intentionally scrolls away, so content growth alone does
not disable follow mode.


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## ELI5

This PR fixes how the Studio chat displays AI responses that stream in
very quickly. When an AI thinks, then uses a tool, then replies with
text, the fix keeps each "thought" separate instead of mixing them all
together. It also improves the auto-scroll feature so the chat keeps
showing new messages at the bottom, but stops automatically scrolling if
you manually scroll up.

## Changes Overview

This pull request addresses streaming render stability in the Studio
playground by improving how interleaved reasoning chunks are handled and
refining autoscroll behavior during rapid output.

### Reasoning Chunk Handling

The `toUIMessage` utility now preserves separate reasoning blocks when
`reasoning-delta` chunks arrive interleaved with other message parts
(tool calls, text). Previously, all reasoning content was merged into
the first reasoning block regardless of where it appeared in the message
sequence. The new behavior:
- Appends new reasoning-delta text to the most recent trailing reasoning
part if one exists
- Creates a new separate reasoning part if the last part is not
reasoning, preventing merged reasoning blocks

A new test case validates this behavior, ensuring reasoning parts remain
distinct and properly sequenced.

### Autoscroll Behavior Improvements

The `useAutoscroll` hook now better handles user interaction during
rapid streaming:
- Detects user scroll intent via wheel, touch, pointer, and keyboard
events
- Temporarily disables auto-follow when explicit user scroll-up is
detected
- Cancels pending animation frames to prevent forced viewport
repositioning after user scrolling
- Uses `requestAnimationFrame` for scroll animations instead of smooth
scroll behavior
- Introduces a `SCROLL_END_THRESHOLD` constant (8px) to determine
whether content should trigger auto-scrolling
- Expands scroll triggers from mutation observation to include both
`MutationObserver` (content changes) and `ResizeObserver` (size changes)

### Files Modified

- `.changeset/quiet-geese-scroll.md` - Changelog entry describing the
fixes for `@mastra/react` and `@mastra/playground-ui`
- `client-sdks/react/src/lib/ai-sdk/utils/toUIMessage.ts` - Updated
reasoning-delta handling logic
- `client-sdks/react/src/lib/ai-sdk/utils/toUIMessage.test.ts` - Added
test for interleaved reasoning preservation
- `packages/playground-ui/src/hooks/use-autoscroll.tsx` - Enhanced
autoscroll with user intent detection and improved animation handling

<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: Mastra Code (openai/gpt-5.5) <noreply@mastra.ai>

* chore: version packages

* chore: regenerate providers and docs [skip ci]

* ci(e2e): migrate remaining matrix jobs to starsling-ubuntu-24.04 (#16330)

Co-authored-by: starslingdev[bot] <248995740+starslingdev[bot]@users.noreply.github.com>

* fix(core): fix requestContext serialization in persistStepUpdate (#16082)

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: caleb <caleb@mastra.ai>
Co-authored-by: Caleb Barnes <caleb.d.barnes@gmail.com>

* fix(observability): resolve OpenRouter "vendor/model" ids in pricing lookup (#16206)

## Description

Adds two more model-id variants to the pricing lookup so OpenRouter ids
in `vendor/model` form resolve:

- slash-flattened (`xiaomi/mimo-v2-pro` → `xiaomi-mimo-v2-pro`)
- vendor-prefix-dropped (`openai/gpt-5-mini` → `gpt-5-mini`)

Date stripping is applied to each variant; the variant set is deduped so
non-prefixed inputs don't pay for redundant lookups. Native provider
lookups are unchanged.

Follows up on #14959 (dot-to-dash) and #15349 (date suffixes).

## Related Issue(s)

Closes #16205

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Code refactoring
- [ ] Performance improvement
- [ ] Test update

## Checklist

- [x] I have made corresponding changes to the documentation (if
applicable)
- [x] I have added tests that prove my fix is effective or that my
feature works
- [ ] I have addressed all Coderabbit comments on this PR

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## ELI5
This PR fixes how the system finds prices for OpenRouter models that use
names like "vendor/model" by trying additional name variants (like
replacing "/" with "-" and dropping the vendor prefix) so the correct
cost is found and shown.

## Changes

### Core Logic (observability/.../pricing-registry.ts)
- Rewrote getModelVariants to produce a prioritized, deduplicated set of
lookup variants for a given model id.
- New variants generated:
- Slash-to-dash normalization (e.g., `xiaomi/mimo-v2-pro` →
`xiaomi-mimo-v2-pro`)
- Vendor-prefix-dropped for slash-containing ids (e.g.,
`openai/gpt-5-mini` → `gpt-5-mini`)
  - Dot-to-dash normalization (existing behavior retained)
- Date-suffix stripping is applied to every variant.
- Variants are deduplicated to avoid redundant lookups.
- Doc comment expanded to explain variant strategy and ordering.
- Native provider lookups are unchanged.

### Tests (observability/.../estimator.test.ts)
- Added two Vitest cases to validate OpenRouter model normalization and
pricing resolution:
1. Resolves `xiaomi/mimo-v2-pro-...` to `xiaomi-mimo-v2-pro` when
pricing keeps the vendor prefix.
2. Resolves `openai/gpt-5-mini-...` to `gpt-5-mini` when pricing omits
the vendor prefix.
- Ensures pricing_id/tier_index and cost calculations match
expectations.

### Test Fixtures (observability/.../pricing-data-test.jsonl)
- Added JSONL fixture entries for OpenRouter models used in tests:
- `openrouter-xiaomi-mimo-v2-pro` with USD pricing under
model_pricing/v1
  - `openrouter-gpt-5-mini` with USD pricing under model_pricing/v1

### Changelog (.changeset/gentle-maps-rest.md)
- Patch entry for @mastra/observability documenting the fix: Model Usage
& Cost now shows costs for OpenRouter IDs in `vendor/model` form that
previously appeared empty.

## Impact
- Fixes observation/metrics cost lookup failures for OpenRouter models
using `vendor/model` IDs by expanding normalization rules and applying
date-stripping to all variants.
- Backward-compatible: native provider lookups unchanged.
- Tests and fixtures included; related issue closed: #16205
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

* fix(playground-ui): unbreak storybook build (codemirror lang-jinja missing peer deps) (#16346)

## Summary

- Vercel preview's `pnpm build-storybook` for `@mastra/playground-ui`
started failing on every PR with `[vite]: Rollup failed to resolve
import "@codemirror/view" from "@codemirror/lang-jinja"`.
- Root cause: upstream bug in `@codemirror/lang-jinja@6.0.0` — it
imports `@codemirror/view` and `@codemirror/state` but does not declare
them as deps. Pulled in transitively via `@codemirror/language-data`
lazy import (added to playground-ui in #9302).
- Latent until #16245 (`chore(deps): lock file maintenance`) regenerated
the pnpm graph (bumped `@codemirror/view` 6.41.0 → 6.41.1,
`@codemirror/search` 6.6.0 → 6.7.0). New hoisting layout no longer
placed `@codemirror/view` next to `lang-jinja`, so Rollup strict
resolution failed. Vite dev hides it via pre-bundling; Rollup prod build
does not.
- Fix: use `pnpm.packageExtensions` to inject the missing deps into
`lang-jinja`. Standard pnpm mechanism for upstream peer-dep gaps; no
revert needed.

## Test plan

- [x] `pnpm install` — lock regenerated cleanly
- [x] Verified
`node_modules/.pnpm/@codemirror+lang-jinja@6.0.0/node_modules/@codemirror/`
now contains `state` and `view`
- [x] `pnpm --filter @mastra/playground-ui build-storybook` — build
green locally
- [ ] Vercel preview build green on this PR

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## ELI5

The package that needed two helper tools (@codemirror/view and
@codemirror/state) was fixed upstream, and this PR tells our package
manager to use that fixed version so Storybook can build again.

## Problem

Vercel preview builds for @mastra/playground-ui Storybook failed during
Rollup bundling because @codemirror/lang-jinja@6.0.0 imported
@codemirror/view and @codemirror/state but did not declare them as
dependencies. Lockfile maintenance (#16245) changed pnpm hoisting and
revealed the missing deps (Vite dev pre-bundling had hidden the issue).

## Solution

Use pnpm overrides to pin @codemirror/lang-jinja to ^6.0.1, which
upstream released with @codemirror/view and @codemirror/state properly
declared, removing the previous packageExtensions workaround.

## Changes

- package.json: added pnpm.overrides entry for @codemirror/lang-jinja ->
^6.0.1 (net +2/-1 lines)

## Verification

- `pnpm install` — lockfile regenerated cleanly
- Confirmed node_modules contains @codemirror/state and @codemirror/view
under @codemirror/lang-jinja@6.0.1
- `pnpm --filter @mastra/playground-ui build-storybook` — local build
succeeds
- Vercel preview build: pending
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

* refactor(playground): redesign composer and polish DS interactions (#16269)

## Summary
- **Composer redesign**: split into a pill-shaped provider/model
switcher, an action row that wraps cleanly when space is tight
(`flex-wrap-reverse`, no breakpoint magic), and a dedicated send button.
Attachments now float above the composer. Smaller remove icon (`X` in
`Button` + `icon-sm`).
- **DS Combobox / Select**: added an `active:bg-surface4` state so
triggers feel like buttons, and fixed value truncation when a leading
badge (e.g. provider connection-status dot) is rendered — the truncate
now sits on the inner span so absolutely-positioned children stay
visible.
- **DS Threads**: dropped the default `bg-surface2` background so
consumers control the surface.
- **DS PanelSeparator**: enlarged hit area, refined hover/active fills,
added a focus-visible accent.
- **Agents layout polish**: removed the header bottom border, removed
`bg-surface2 px-4` from the page tabs, removed `py-4` from the main
panel; `LLMProvider`'s logo gets `shrink-0` + min size to keep the
connection-status dot anchored inside the new pill.

Before:
<img width="1731" height="993" alt="CleanShot 2026-05-06 at 17 06 37"
src="https://github.com/user-attachments/assets/86ad5f91-bdec-4f0f-9dad-5691ed2f1907"
/>

After:
<img width="1731" height="993" alt="CleanShot 2026-05-06 at 17 04 24"
src="https://github.com/user-attachments/assets/a4c0968c-f5f8-4ced-98a5-b9df15aeb1de"
/>


## Why
The composer felt cramped at narrow widths, the model switcher squashed
the provider logo and clipped its connection-status dot, and the
surrounding agent layout had stacked grey backgrounds (`Threads` + tabs
+ panel) that competed visually. This PR consolidates the chat surface,
gives the switcher a proper pill shape, and tightens DS triggers so they
read as buttons.

## Test plan
- [ ] Composer: type a message, attach files, verify attachments render
above the composer and remove (`X`) works.
- [ ] Composer model switcher: verify provider + model pills share a
single border, hover/active fills clip to the pill outline,
connection-status dot remains visible at all widths.
- [ ] Narrow viewport: verify the action row wraps so the switcher drops
below the buttons (`flex-wrap-reverse`) without overflow.
- [ ] Missing-API-key warning: select a non-connected provider — banner
should render above the action row, not inside the switcher pill.
- [ ] DS Combobox / Select: hover + active states show new fill; opening
triggers no longer flash; truncation works for long values.
- [ ] PanelSeparator: hover, drag, and focus-visible all render the new
affordance.
- [ ] `Threads` consumers (`chat-threads`, `workflow-run-list`): verify
the new transparent default still looks correct in their parents.

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## ELI5

This PR redesigns how the message composer and related interface
elements look and work in Mastra Studio. The composer is split into
simpler, more compact pieces that rearrange cleanly on smaller screens,
attachments now float above the composer instead of being cramped inside
it, and various design system components get visual feedback
improvements (like buttons lighting up when clicked) and better text
handling.

---

## Composer Redesign & Layout

- **Composer structure overhaul**: Split into a pill-shaped
provider/model switcher, an action row that wraps cleanly without
breakpoint-specific hacks (using `flex-wrap-reverse`), and a dedicated
send button
- **Floating attachments**: Moved to absolute positioning above the
composer input, with removal controls reduced to a smaller X icon
(changed from CircleXIcon to X, size downgraded from icon-md to icon-sm)
- **ComposerActionRow**: New component consolidates model switcher,
attachment button, dictation input, and send/cancel controls with
cleaner conditional rendering
- **Missing API key banner**: Now positioned above the action row
instead of inside the switcher pill

---

## Design System Component Updates

**Combobox & Select**:
- Added `active:bg-surface4` state to trigger elements so they feel like
interactive buttons with visual feedback
- Fixed value truncation when a leading badge (e.g., provider
connection-status dot) is rendered by moving truncation logic to an
inner span wrapper, keeping absolutely-positioned children visible

**Threads**:
- Removed default `bg-surface2` background from root nav element,
allowing consumers to control the surface color

**PanelSeparator**:
- Enlarged hit area for dragging
- Refined hover and active fill states
- Added focus-visible accent for keyboard navigation affordance
- Now renders a resizing handle with improved structural markup

---

## Agent Layout Refinements

- **Agent header**: Added `border={false}` prop to Header component to
remove bottom border
- **Page tabs**: Removed `bg-surface2` and `px-4` from the wrapper,
leaving a plain flex container
- **Main panel**: Removed vertical padding (`py-4`) from the layout
- **Provider logo**: Added `shrink-0` and minimum size constraints to
keep the connection-status dot anchored inside the provider/model pill
at narrow widths; improved fallback UI with consistent sizing

---

## Testing Considerations

- Verify attachments float above composer and removal works as expected
- Confirm provider+model pill shares a single border and
connection-status dot remains visible at narrow widths
- Validate action row wraps with `flex-wrap-reverse` without content
overflow
- Ensure missing-API-key banner renders above the action row, not inside
the switcher pill
- Check Combobox/Select visual feedback (hover/active states) and
truncation behavior with badges
- Validate PanelSeparator hover, drag, and focus-visible affordances are
clear
- Confirm Threads consumers render correctly with the
transparent/controllable default background
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

* Add MODEL_INFERENCE span type for measuring model provider latency (#16267)

## Description

This PR introduces a new `MODEL_INFERENCE` span type that wraps the
model provider call within a `MODEL_STEP`, enabling precise measurement
of model latency separately from input/output processing and tool
executions.

### Key Changes

- **New Span Type**: Added `MODEL_INFERENCE` to `SpanType` enum with
corresponding attributes interface (`ModelInferenceAttributes`)
- **Span Hierarchy**: Updated from `MODEL_GENERATION -> MODEL_STEP ->
MODEL_CHUNK` to `MODEL_GENERATION -> MODEL_STEP -> MODEL_INFERENCE ->
MODEL_CHUNK`
- **Processor/Tool Handling**: Processors and tool executions remain as
siblings of `MODEL_INFERENCE` under `MODEL_STEP`, ensuring
`MODEL_INFERENCE` measures only pure model time
- **Feature Gate**: Implemented behind `model-inference-span` feature
flag for gradual rollout
- **Snapshot Updates**: Updated all test snapshots to reflect the new
span hierarchy

### Implementation Details

- Modified `ModelSpanTracker` to create `MODEL_INFERENCE` spans when the
feature is enabled
- Added `ModelInferenceContext` type for tracking inference-specific
data
- Updated span hierarchy in trace generation and visualization
- Added comprehensive test coverage for the new span type

### Files Modified

- `packages/core/src/observability/types/tracing.ts`: Added
`MODEL_INFERENCE` span type and attributes
- `observability/mastra/src/model-tracing.ts`: Implemented inference
span creation and tracking
- `observability/mastra/src/model-tracing.test.ts`: Added tests for
inference span behavior
- `packages/core/src/features/index.ts`: Added feature flag
- Multiple snapshot files: Updated to reflect new span hierarchy
- Supporting files: Updated span type handling in base spans, evals, and
processor tracing

## Type of Change

- [x] New feature (non-breaking change that adds functionality)

## Testing

Added comprehensive unit tests in `model-tracing.test.ts` covering:
- Inference span creation as child of MODEL_STEP
- Chunk spans properly parented under inference spans
- Inference span attributes (stepIndex, model, provider, streaming,
parameters, availableTools)
- Multiple inference spans within a single step
- Feature flag behavior

All existing tests pass with updated snapshots reflecting the new span
hierarchy.

https://claude.ai/code/session_01GyZ9wem6FgwPECXpnMQsSK

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## ELI5

This PR adds a separate "MODEL_INFERENCE" timing span so we can measure
only the time the model provider spends generating responses—separate
from other work like tool calls or input/output handling.

## Summary

Introduces a new SpanType MODEL_INFERENCE and supporting types/logic to
isolate model provider latency by changing the span hierarchy to
MODEL_GENERATION → MODEL_STEP → MODEL_INFERENCE → MODEL_CHUNK. The
feature is behind a `model-inference-span` feature flag for gradual
rollout; when disabled, behavior falls back to the previous MODEL_STEP →
MODEL_CHUNK parenting.

### Key Changes

- Types & API
- Added SpanType.MODEL_INFERENCE and ModelInferenceAttributes,
ModelInferenceContext.
- Extended IModelSpanTracker with optional setInferenceContext(context).
- Added observabilityFeatures export containing 'model-inference-span'.
  - Added 'model_inference' route-name literal to client-js route types.

- Core implementation
- ModelSpanTracker creates MODEL_INFERENCE spans under MODEL_STEP when
the feature flag is enabled and parents MODEL_CHUNK spans to the active
inference span.
- New setInferenceContext persists request-side parameters/provider
options/availableTools/toolChoice/responseFormat for subsequent
inference spans.
- wrapStream and chunk parent selection centralized to respect STEP →
INFERENCE → CHUNK lifecycle.
- Inference spans are ended eagerly on step finish (including
durable/deferred step flows) so later tool work doesn't inflate
inference duration.
- Usage and finishReason are duplicated onto both MODEL_INFERENCE and
MODEL_STEP for compatibility.
- Span internal classification and trajectory conversion updated to
treat MODEL_INFERENCE as internal / skipped type where appropriate.

- Integrations
- MastraLLMVNext.stream and durable LLM execution wiring call
setInferenceContext to propagate request context (including
structured-output responseFormat when applicable).
- Durable LLM execution adds inference context setup to ensure correct
attributes for persisted runs.

- Testing & snapshots
- Comprehensive unit tests added/updated to validate inference span
creation, parenting, attributes, context propagation, multiple
inferences per step, durable-mode behavior, and feature-flag fallback.
- ~29 observability snapshot files updated to reflect new span hierarchy
and adjusted metadata ordering (environment before modelMetadata).
  - Test exporter gained getModelInferenceSpans() helper.

- Backward compatibility & rollout
- Feature gated by core feature flag 'model-inference-span'
(coreFeatures and observabilityFeatures).
- If the flag is absent (older packages), traces revert to prior
structure (chunks under MODEL_STEP).
  - Changeset bumps for @mastra/core and @mastra/observability (minor).

Files of note modified:
packages/core/src/observability/types/tracing.ts,
observability/mastra/src/model-tracing.ts (+tests),
packages/core/src/features/index.ts, Mastra LLM loop & durable execution
files, many observability snapshots and test helpers.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: Claude <noreply@anthropic.com>

* feat(core,server,cli,redis-streams): standalone workers + push-capable PubSub + Redis Streams broker (#16309)

## Summary

Introduces a first-class **MastraWorker** abstraction so a Mastra
deployment can split the workflow orchestrator, scheduler, and
background-task worker into separate processes deployed independently
from the HTTP server. Also adds a production Redis Streams PubSub
implementation, a push-capable PubSub interface, and the supporting
routes / CLI / docs.

The default (single-process) deployment is unchanged: when nothing is
configured, Mastra auto-creates the three workers in-process and runs as
before.

## What's in this PR

### Worker abstraction (packages/core, packages/cli, packages/server)

- `MastraWorker` base class with `OrchestrationWorker`,
`SchedulerWorker`, `BackgroundTaskWorker` concrete implementations.
- Mastra auto-creates the three workers by default; opt out per-worker
via `MASTRA_WORKERS` env (e.g. `MASTRA_WORKERS=orchestration,scheduler`
runs only those; `MASTRA_WORKERS=false` disables event processing
entirely).
- New `mastra worker` CLI command for running a worker process from a
bundled Mastra entry file. `WorkerBundler` reuses the existing build
pipeline.
- Two step-execution strategies for orchestrators that don't run in the
server process:
  - `InProcessStrategy`: default, executes steps locally.
- `HttpRemoteStrategy`: ships step execution to the server's `POST
/api/workflows/:id/runs/:runId/steps/execute` route. Auth is delegated
to the framework's existing `experimental_auth` provider via standard
`Authorization: Bearer` headers — no separate worker-secret machinery.
- `BackgroundTaskWorker` reuses Mastra's `BackgroundTaskManager`
instance when present so cross-process workers share the same dispatch /
result / recovery semantics. When dispatch is received in a worker
process that doesn't hold the producer's per-task closure, executor
lookup falls back to a static `toolName → executor` registry populated
from the registered tools on Mastra.

### Push-capable PubSub (packages/core, packages/server)

- `PubSub` gains a `supportedModes: ('pull' | 'push')[]` declaration.
Existing third-party implementations default to `['pull']` for backwards
compatibility.
- `EventEmitterPubSub` declares `['pull', 'push']` (in-process, supports
both).
- `mastra.handleWorkflowEvent(event)` is the new public hook for push
delivery. When configured with a push-only PubSub, Mastra wires the
in-process subscription to call it directly and skips auto-creating an
`OrchestrationWorker`.
- New `POST /api/workers/events` route receives workflow events from
push-mode brokers (e.g. GCP Pub/Sub push subscriptions, EventBridge HTTP
targets) and forwards them to `handleWorkflowEvent`. This route is the
symmetric counterpart to the step-execute route and is gated by the same
auth provider mechanism.

### Redis Streams PubSub (pubsub/redis-streams)

New `@mastra/redis-streams` package implementing a production-grade
pull-mode broker:

- Consumer groups created at stream start (`'0'`) so late-joining
workers see historical events.
- `XAUTOCLAIM` reclaim loop for pending messages whose consumer died.
- Configurable `maxDeliveryAttempts` cap on `nack` (poison-pill
protection).
- Configurable `MAXLEN ~` stream trim on publish.
- Topic-scoped `unsubscribe` + clean shutdown.
- Structured logger pass-through; no silent `catch {}` blocks.

### Cross-process integration tests (pubsub/redis-streams/src)

Real multi-process tests using the bundled CLI worker artifact + Redis:

- `cross-process.test.ts` — server + standalone orchestrator
- `scheduler-cross-process.test.ts` — standalone scheduler
- `background-cross-process.test.ts` — standalone background-task worker
(proves cross-process executor resolution via the static-tool registry)
- `all-workers-split.test.ts` — server + all three workers in separate
processes
- `cli-bundler.test.ts` — `mastra worker` bundled artifact boots and
processes events
- `resilience.test.ts` — competing consumers, replica failover,
late-joining workers
- `auth-e2e.test.ts` — confirms `Authorization: Bearer` tokens are
validated by the framework auth provider on both `steps/execute` and
`workers/events` routes

### Docs

- `deployment/standalone-workers.mdx` — covers `MASTRA_WORKERS`, remote
step execution via `MASTRA_STEP_EXECUTION_URL` +
`MASTRA_WORKER_AUTH_TOKEN`, push vs pull pubsub, example server-side
`MastraAuthConfig`.
- `deployment/overview.mdx` — new section linking to standalone workers.

## What's NOT in this PR

- GCP Pub/Sub / SNS / EventBridge / SQS adapters — push-mode receive
route exists, but the broker-specific envelope decoders are
intentionally out of scope.
- `mastra worker temporal` — the existing `@mastra/temporal` package
stays as a thin library; no new worker subclass.

## Test plan

- `pnpm --filter ./packages/core check` (typecheck)
- `pnpm test:core`
- `pnpm --filter ./packages/server test`
- `pnpm --filter ./pubsub/redis-streams test` (against local Redis)


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## ELI5

This PR lets Mastra split its work into separate programs: one that
schedules jobs, one that decides what to run, and one that runs
background tasks. It adds a CLI to build/run those workers, a
Redis‑Streams message broker to coordinate them, and a way for remote
workers to ask the server to execute individual workflow steps over HTTP
(with authentication).

---

## Overview

Adds first-class standalone worker support and a Redis Streams PubSub
broker so Mastra can run orchestration, scheduler, and background-task
workers as separate processes while preserving single-process defaults.
Includes worker lifecycle APIs, two step-execution strategies
(in-process and HTTP remote), CLI bundling/launcher, push-capable PubSub
hooks, a production-grade Redis Streams implementation, multi-process
integration tests, and deployment docs.

---

## Key Features & Changes

- Worker framework
- New MastraWorker base and concrete OrchestrationWorker,
SchedulerWorker, BackgroundTaskWorker.
- Auto-creation of workers by default; controllable via config.workers
and MASTRA_WORKERS (comma-separated selection or MASTRA_WORKERS=false).
- Mastra exposes workers getter and getWorker(name); startWorkers(name?)
/ stopWorkers(); legacy startEventEngine/stopEventEngine kept as
deprecated aliases.

- Step execution strategies
  - InProcessStrategy (default) executes steps in-process.
- HttpRemoteStrategy posts to POST
/api/workflows/:id/runs/:runId/steps/execute with merged abort/timeout
semantics and bearer auth from MASTRA_WORKER_AUTH_TOKEN or explicit
config.
- Server adds the step-execution route with payload validation and
permissive JSON response.

- CLI & bundling
- mastra worker [name] command and WorkerBundler to emit/run bundled
worker artifacts, with dotenv handling, platform selection, and
signal-safe shutdown.

- PubSub delivery modes & push support
- PubSub now has supportedModes: ('pull'|'push')[] (default ['pull']);
EventEmitterPubSub supports ['pull','push'].
- Mastra.handleWorkflowEvent(event) public hook and POST
/api/workers/events for push-mode deliveries (auth gated).
- Orchestration worker creation skipped when pubsub lacks 'pull';
startWorkers wires push subscriptions when appropriate.

- Redis Streams broker (@mastra/redis-streams)
- RedisStreamsPubSub implementing pull-only delivery with consumer
groups (anchored at '0'), XAUTOCLAIM reclaim loop, configurable reclaim
intervals, optional MAXLEN trimming, and topic-scoped unsubscribe.
- Delivery attempt tracking with maxDeliveryAttempts (validation:
negative/NaN rejected; 0 treated as Infinity with a one-time warning;
Infinity disables cap).
- Ack via XACK (no xdel) so other groups can replay; nack
republish-redelivers with incremented deliveryAttempt; events dropped
and warned when attempts exhausted.
- Per-subscriber reader clients, flush/close semantics, clean shutdown,
structured logger passthrough, tests and docker-compose for Redis.

- Background tasks cross-process
- BackgroundTaskManager static executor registry
(registerStaticExecutor/unregisterStaticExecutor/getStaticExecutor) for
cross-process dispatch.
- BackgroundTaskWorker reuses Mastra's manager when present (preserves
per-task contexts) or owns a manager otherwise; subscribes on start();
start() throws if init() wasn't called.

- Transport & routing
- Worker transport interfaces (WorkerTransport, EventRouter) and
PullTransport implementation that subscribes grouped pull consumers and
nacks on handler promise rejections with injected logger support.

- WorkflowEventProcessor improvements
  - Pluggable StepExecutionStrategy injection.
- New handle(event) returning { ok:true } | { ok:false, retry:boolean };
process(event, ack?) preserved as wrapper.

- Robustness and infra fixes
- Idempotent lifecycle operations, stricter validations, improved
logging, proper nacking on handler failures, unsubscribe scoping fixes,
CLI spawn error surfacing, test flake fixes, and related hardening.

---

## API Additions (high level)

- Core
- Mastra.handleWorkflowEvent(event), startWorkers, stopWorkers, workers
getter, getWorker(name).
- MastraWorker, OrchestrationWorker, SchedulerWorker,
BackgroundTaskWorker (+ configs).
- StepExecutionStrategy, StepExecutionParams, InProcessStrategy,
HttpRemoteStrategy, StepExecutionError.
  - WorkerTransport/EventRouter interfaces and PullTransport.
  - PubSubDeliveryMode type and supportedModes getter on PubSub.

- Server
- POST /api/workflows/:workflowId/runs/:runId/steps/execute (requires
auth when an auth provider is configured).
  - POST /api/workers/events for push-mode broker deliveries.

- CLI
  - mastra worker [name] command and WorkerBundler.

- New package
- @mastra/redis-streams: RedisStreamsPubSub and RedisStreamsPubSubConfig
plus test/build configs.

---

## Testing

Extensive Vitest integration tests exercise:
- Multi-process topologies (server + orchestration + scheduler +
background) against real Redis and shared storage.
- Cross-process step execution and auth (auth-e2e).
- Background-task cross-process dispatch and failure cases.
- Scheduler cross-process behavior and resilience (late-join recovery,
competing consumers, reclaim/XAUTOCLAIM).
- RedisStreams PubSub behaviors (fan-out, groups, ack/nack, redelivery
caps, unsubscribe scoping).
- CLI bundler output check (skipped-by-default gated test).

---

## Docs

- deployment/standalone-workers.mdx and deployment/overview.mdx
- explorations/orchestrator-report.md
- Changeset documenting push-capable PubSub, startWorkers usage, and
operational notes.

---

## Breaking Changes

No functional breaking changes—single-process defaults preserved.
startEventEngine/stopEventEngine are deprecated aliases for
startWorkers/stopWorkers.

---

## Migration / Usage Notes

- To run workers as separate processes: configure a Redis Streams
broker, build bundles with mastra worker, and run desired workers
(orchestration, scheduler, backgroundTasks) or use MASTRA_WORKERS to
select.
- For remote step execution: set MASTRA_STEP_EXECUTION_URL on workers
and provide MASTRA_WORKER_AUTH_TOKEN (or explicit HttpRemoteStrategy
auth) to authenticate server calls.
- For push-only PubSub providers, Mastra will call handleWorkflowEvent
for push delivery and skip auto-creating orchestration workers.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: Mastra Code (anthropic/claude-opus-4-7) <noreply@mastra.ai>

* feat(observability): expose dropped observability events (#16111)

Adds structured drop events for observability data that
`DefaultExporter` cannot persist. Exporters and bridges can implement
`onDroppedEvent` to observe unsupported storage drops and retries being
exceeded with signal, reason, count, storage, exporter, timestamp, and
sanitized error details.

Before, unsupported logs or metrics could be dropped after a local
warning, and retry exhaustion inside the event buffer was not visible to
other observability sinks.

After:

```ts
class DropAlertExporter extends BaseExporter {
  name = 'drop-alerts'

  async onDroppedEvent(event: ObservabilityDropEvent) {
    await fetch('https://monitoring.example.com/observability-drops', {
      method: 'POST',
      headers: { 'content-type': 'application/json' },
      body: JSON.stringify({
        count: event.count,
        signal: event.signal,
        reason: event.reason,
        exporterName: event.exporterName,
      }),
    })
  }
}
```

Includes focused tests for bus fan-out, `DefaultExporter`
unsupported-storage and retry-exhausted paths, and `EventBuffer`
dropped-event return values.

Fixes #16040


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## ELI5

When observability systems (which collect logs, metrics, and traces from
applications) can't store or send data, they usually just silently fail.
This PR adds a way for exporters to be notified about these failures so
that external monitoring systems can alert when observability is only
partially working—for example, when traces are being captured but logs
aren't.

---

## Overview

This PR introduces a structured event notification system for dropped
observability data. When the observability pipeline cannot persist
events (due to unsupported storage or exhausted retries), it now emits
standardized `ObservabilityDropEvent` objects to exporters and bridges
implementing the new `onDroppedEvent` handler, enabling external
monitoring and alerting on partial observability setups.

---

## Key Changes

### New Type Definitions

Four new types are introduced in
`packages/core/src/observability/types/core.ts`:
- `ObservabilityDropSignal`: Identifies the signal type (`'tracing' |
'log' | 'metric' | 'score' | 'feedback'`)
- `ObservabilityDropReason`: Indicates why the event was dropped
(`'unsupported-storage' | 'retry-exhausted'`)
- `ObservabilityDropError`: Sanitized error details (`id`, `domain`,
`message`)
- `ObservabilityDropEvent`: Complete drop notification with signal,
reason, count, timestamp, exporter/storage names, and optional error

### API Extensions

- `ObservabilityExporter` interface now includes optional
`onDroppedEvent?(event: ObservabilityDropEvent): void | Promise<void>`
handler
- `InitExporterOptions` now includes optional `emitDropEvent` callback
for exporters to emit drop events

### Bus & Routing

- `ObservabilityBus` adds `emitDropEvent(event: ObservabilityDropEvent):
void` method to broadcast drop events to all registered exporters and
bridges
- New `routeDropToHandler` function in `route-event.ts` safely routes
drop events to exporter handlers with error handling and async support

### DefaultExporter Enhancements

`DefaultExporter` now:
- Tracks unsupported signals per storage instance using typed
`ObservabilityDropSignal`
- Emits drop events with `reason: 'unsupported-storage'` when a storage
adapter doesn't support a signal
- Emits drop events with `reason: 'retry-exhausted'` when flush retries
exceed the configured maximum
- Includes sanitized `MastraError` details in drop events for debugging
- Threads drop emission through create-flush and span update/end paths
- Uses singular signal names (`log`, `metric`, `score`) instead of
plural in drop events

### EventBuffer Updates

`EventBuffer.reAddCreates()` and `EventBuffer.reAddUpdates()` now return
`BufferedEvent[]` of events that exceeded `maxRetries` instead of
silently dropping them. This allows callers (DefaultExporter) to
explicitly emit drop events for exhausted retries.

### Documentation

- **Configuration examples** section added to DefaultExporter
documentation with a new "Dropped observability events" subsection
explaining the two drop reasons and providing a `DropAlertExporter`
example that forwards drop events to an external monitoring endpoint
- Reference documentation updated to describe the new drop event types,
callback signature, and metadata included in drop notifications

### Testing

- `ObservabilityBus` tests verify drop event routing to exporters and
bridges, async handler handling, and safe handling of missing
`onDroppedEvent` implementations
- `DefaultExporter` tests extend coverage to verify drop events for
unsupported-storage and retry-exhausted scenarios, including error
details and correct signal/reason values
- `EventBuffer` tests verify that `reAddCreates` and `reAddUpdates`
properly return dropped events when retry limits are exceeded

---

## Documentation Updates

Two changesets document the additions:
- `@mastra/observability`: DefaultExporter now emits drop-event
telemetry to custom exporters and integrations
- `@mastra/core`: New structured drop-event types and `onDroppedEvent`
hook for bridge/exporter integrations

<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: Mohamed BEN HAMDOUNE <mbenhamd@Mohameds-MacBook-Pro.local>

* Fix and feat: add metadata filtering support to semantic recall (Issue #8610) (#9256)

Co-authored-by: Hunter Hagedorn <hunterhagedorn@Hunters-MacBook-Pro-2.local>
Co-authored-by: wardpeet <ward@coding-tech.com>
Co-authored-by: Mastra Code (openai/gpt-5.5) <noreply@mastra.ai>

* chore: dedupe and clean up external dependencies (#16351)

Co-authored-by: Mastra Code (openai/gpt-5.5) <noreply@mastra.ai>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>

* Fix cache write tokens not being set in AI SDK v6 usage (#16354)

## Description

Fixed an issue where `inputTokenDetails.cacheWriteTokens` was always
`undefined` in the AI SDK v6 usage object. The field now correctly
reflects the prompt cache creation tokens reported by the provider (via
`cacheCreationInputTokens`). Previously, this value was only accessible
through `providerMetadata.anthropic.cacheCreationInputTokens`.

## Related Issue(s)

<!-- Link to the issue(s) this PR addresses, using hashtag notation:
Fixes #123 -->

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Code refactoring
- [ ] Performance improvement
- [ ] Test update

## Changes

- Updated `normalizeV6Usage()` helper in
`client-sdks/ai-sdk/src/helpers.ts` to map
`usage.cacheCreationInputTokens` to `inputTokenDetails.cacheWriteTokens`
instead of leaving it undefined

## Checklist

- [x] I have made corresponding changes to the documentation (if
applicable)
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] I have addressed all Coderabbit comments on this PR

https://claude.ai/code/session_01Rhzjr9j1dvvo3qein2yeR9

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## ELI5

This PR fixes a bug where the AI SDK was losing track of how many tokens
were used for cache writing. Instead of that information disappearing,
it now properly captures and stores it in the right place so developers
can see it easily.

## Overview

Fixes cache write token tracking in AI SDK v6 usage objects. The
`inputTokenDetails.cacheWriteTokens` field was previously undefined and
unavailable through the standard API surface; cache token information
was only accessible via the provider-specific metadata. This PR maps the
provider-reported prompt cache creation tokens
(`usage.cacheCreationInputTokens`) directly into
`inputTokenDetails.cacheWriteTokens`.

## Changes

- **Updated `normalizeV6Usage()` in
`client-sdks/ai-sdk/src/helpers.ts`**: Added mapping from
`usage.cacheCreationInputTokens` to `inputTokenDetails.cacheWriteTokens`
to ensure cache write tokens are properly exposed in the standard usage
object.

- **Added changeset**: Documents the fix for the `@mastra/ai-sdk`
package (patch version bump).

## Scope

- 1 line changed in the main code file (`helpers.ts`)
- Metadata update in changeset documentation
- No public API changes

<!-- end of auto-generated comment: release notes by coderabbit.ai -->

Co-authored-by: Claude <noreply@anthropic.com>

* Add MAPPING span tracing for tool toModelOutput transforms (#16347)

## Description

This PR adds observability tracing for tool `toModelOutput`
transformations by emitting a `MAPPING` span type. When a tool defines a
`toModelOutput` function and it executes successfully, a child span is
created to track the transformation of the tool result before it's
passed to the model.

### Changes Made

1. **New Span Type**: Added `MAPPING` span type to `SpanType` enum in
`observability/types/tracing.ts` to represent inline data transforms
between pipeline stages.

2. **New Attributes Interface**: Created `MappingAttributes` interface
to capture mapping-specific metadata:
   - `mappingType`: Identifier of the mapping (e.g., `toModelOutput`)
- `toolCallId`: Associated tool call ID when mapping operates on a tool
result

3. **Span Emission Logic**: Updated `llm-mapping-step.ts` to:
- Create a child `MAPPING` span when `toModelOutput` is defined and the
tool result is non-null
- Capture the tool result as span input and the transformed output as
span output
- Mark the span as errored if `toModelOutput` throws, then re-throw the
error
- Skip span creation if the tool has no `toModelOutput` or if the result
is null/undefined

4. **Comprehensive Tests**: Added four test cases covering:
   - Successful mapping span emission with correct attributes and output
   - No span emission when tool lacks `toModelOutput`
   - No span emission when tool result is null/undefined
   - Error handling when `toModelOutput` throws

## Related Issue(s)

Fixes #15486

## Type of Change

- [ ] Bug fix (non-breaking change that fixes an issue)
- [x] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Code refactoring
- [ ] Performance improvement
- [x] Test update

## Checklist

- [x] I have made corresponding changes to the documentation (if
applicable)
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] I have addressed all Coderabbit comments on this PR

https://claude.ai/code/session_018bEDqEtGS2XkdsqBjye6JC

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## ELI5

When a tool changes its result before the AI model sees it, the system
now records that transformation in traces so you can see both the
original tool output and what the model actually received.

## Overview

This PR adds observability for tool toModelOutput transformations by
introducing a MAPPING span type and emitting a child MAPPING span when a
tool defines toModelOutput and the tool result is non-null. The span
records the raw tool result as input and the transformed model-facing
output as span output, and marks the span errored if the transform
throws. No mapping span is created when toModelOutput is absent or the
tool result is null/undefined.

## Changes

- Added SpanType.MAPPING = 'mapping' and exported MappingAttributes in
packages/core/src/observability/types/tracing.ts to represent inline
data transforms and carry mapping-specific attributes (mappingType,
toolCallId).
- Updated
packages/core/src/loop/workflows/agentic-execution/llm-mapping-step.ts
to:
- create a child MAPPING span (EntityType.TOOL) when tool.toModelOutput
exists and the tool result is non-null,
- set span input to the raw tool result and end the span with the
transformed modelOutput,
- mark the span errored (endSpan: true) and re-throw if toModelOutput
throws,
- avoid creating a span when toModelOutput is undefined or tool result
is null/undefined,
- attach modelOutput into providerMetadata.mastra.modelOutput when
present.
- Added tests in
packages/core/src/loop/workflows/agentic-execution/llm-mapping-step.test.ts
covering:
- successful MAPPING span emission with expected attributes and output,
  - no span when the tool lacks toModelOutput,
  - no span when tool result is null/undefined,
- error handling when toModelOutput throws (span errored and error
re-thrown).
- Updated client SDK route types
(client-sdks/client-js/src/route-types.generated.ts) to include
'mapping' in relevant union types.
- Documented the change in the changeset (.changeset/loud-ads-double.md)
and regenerated client types as part of the commit.

## Issue Resolution

Addresses issue #15486 by capturing toModelOutput return values in
tracing so traces can distinguish "not invoked" vs "no-op" vs
"transformed" for tool-to-model mappings.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: Claude <noreply@anthropic.com>

* fix(core): use toJSON for requestContext serialization in all workflow paths (#12573)

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Co-authored-by: Ward Peeters <ward@coding-tech.com>

* Evented workflow foreach fix (#16358)

## Description

<!-- Provide a brief description of the changes in this PR -->

- Fix evented workflow foreach timing out when payload is an empty array
- Write better test for foreach concurrency

## Related Issue(s)

<!-- Link to the issue(s) this PR addresses, using hashtag notation:
Fixes #123 -->

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Code refactoring
- [ ] Performance improvement
- [x] Test update

## Checklist

- [ ] I have made corresponding changes to the documentation (if
applicable)
- [ ] I have added tests that prove my fix is effective or that my
feature works
- [ ] I have addressed all Coderabbit comments on this PR


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## ELI5

This PR fixes a bug where evented workflows could get stuck when asked
to loop over an empty list, making the workflow properly treat an empty
list as "done" and continue. It also improves the foreach concurrency
test to verify parallelism reliably.

## Changes Overview

### Bug Fix: Empty Array Handling in Foreach Loops

- File:
packages/core/src/workflows/evented/workflow-event-processor/loop.ts
- processWorkflowForEach now treats a previous result with an empty
output array (prevResult.output.length === 0) as a completed foreach. It
constructs and persists a successful empty result, updates stepResults,
and advances to the next step. The workflowsStore acquisition was moved
earlier to support this flow. This prevents timeouts when the foreach
payload is an empty array.

### Test Updates

- File: packages/core/src/workflows/evented/evented-workflow.test.ts  
- Removed skip entries for foreachPartialConcurrencyTiming and
emptyForeach so those tests run for the evented engine.

- File: workflows/_test-utils/src/domains/foreach.ts  
- Reworked the "foreach-partial-concurrency" test setup to use a runtime
concurrency tracker (increment/decrement + peakActive) inside the mock
map step instead of wall-clock timing assertions. The test now asserts
concurrencyTracker.peakActive === 2 and keeps output verification;
resetMocks now resets the tracker.

### Changelog

- File: .changeset/silver-hounds-sink.md  
- Added a changeset for @mastra/core (patch) documenting the foreach fix
for empty array payloads.

## Type of change

- Bug fix
- Test update

## Checklist (from PR)

- Documentation changes: not marked complete  
- Tests added: not marked complete  
- Addressed Coderabbit comments: not marked complete
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

* Fix MODEL_INFERENCE span timing to exclude processor work (#16357)

## Description

Fixed `MODEL_INFERENCE` span timing so it accurately measures pure model
latency, excluding input processor and `prepareStep` work. Previously,
the inference span opened from `startStep()`, which the agentic loop
calls before processors run, inflating the span's duration.

The span now opens immediately before the model call via a new
`startInference()` method on the tracker. Chunk-arrival auto-creation
remains as a fallback for callers that don't explicitly call it.

Additionally, `setInferenceContext()` is now applied per-step after
input processors finalize the tool set, so `availableTools` and
`toolChoice` on `MODEL_INFERENCE` spans reflect per-step mutations
(input processors, `prepareStep`, `activeTools` filtering) instead of
the agent-run-level snapshot.

### Changes

- **ModelSpanTracker**: Removed automatic inference span creation from
`startStep()`. Added public `startInference()` method to open the span
immediately before model invocation, snapshotting the latest
`#inferenceContext`.
- **Auto-creation safety net**: Inference spans are auto-created on
first chunk arrival (or step-start chunk) if the caller didn't
explicitly call `startInference()`, ensuring chunks always parent under
`MODEL_INFERENCE` rather than `MODEL_STEP`.
- **LLM execution steps**: Both agentic and durable execution steps now
call `setInferenceContext()` + `startInference()` immediately before the
model call, after input processors and `prepareStep` have completed.
- **Utility function**: Added `getStepAvailableToolNames()` to compute
the post-processor tool set, applying `activeTools` filtering when
present.

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [x] Code refactoring

## Checklist

- [x] I have added tests that prove my fix is effective or that my
feature works
- Added 4 new test cases covering: span timing exclusion, inference
context snapshotting, auto-creation on chunk arrival, and explicit
`startInference()` behavior
  - Added unit tests for `getStepAvailableToolNames()` utility function
- [x] Existing tests pass

https://claude.ai/code/session_01TURPcy3KRBWnmRV7WuQqun

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## ELI5

This PR makes the AI model timer more accurate by starting the
model-timing span immediately before the model runs (after input
processing), so measured latency reflects only the model call itself.

---

## Overview

Refactors MODEL_INFERENCE span lifecycle so inference spans are opened
and contextualized right before the model invocation (excluding input
processors and prepareStep work). Introduces an explicit
startInference() API, auto-creation safety-net for streaming chunks,
per-step inference-context snapshotting, and a utility to compute
available tools for a step.

---

## Key Changes

- ModelSpanTracker behavior
  - startStep() now opens only MODEL_STEP.
- New public startInference(payload?) opens MODEL_INFERENCE immediately
before model invocation and snapshots the latest inferenceContext.
- Auto-creation safety-net: if chunks (including step-start chunks)
arrive before startInference() is called, the tracker auto-creates
missing MODEL_STEP and MODEL_INFERENCE so chunk spans parent under
MODEL_INFERENCE.

- Inference-context timing
- Agentic and durable execution paths now call setInferenceContext() and
startInference() immediately before the model call (after input
processors and prepareStep), so availableTools, toolChoice, and
responseFormat reflect post-processor mutations.
- Durable execution: responseFormat for inference context is derived
from the actual structuredOutput payload sent to execute(), avoiding
incorrect reporting when structuringModelConfig is used.

- Utility
  - Added getStepAvailableToolNames(tools?, activeTools?) which:
- Returns a shallow copy of activeTools whenever activeTools is provided
(including an explicit empty array, allowing explicit disabling of
tools),
- Otherwise returns Object.keys(tools) when tools is provided (may be
[]),
    - Otherwise returns undefined.

- Safety & consolidation
- Extracted ensureStepAndInference helper to consolidate auto-create
logic used by chunk handlers and wrapStream (which treats step-start as
a point to open inference if not yet opened).

---

## Public API Changes

- IModelSpanTracker (packages/core/src/observability/types/tracing.ts)
- Added optional startInference?(payload?: StepStartPayload): void with
docs noting it should be called immediately before invoking the model
and that it snapshots inference context and supports fallback
auto-creation.

- ModelSpanTracker (observability/mastra/src/model-tracing.ts)
- Added public startInference(payload?: StepStartPayload): void and
moved related logic into ensureStepAndInference helper.

- New exported utility
- getStepAvailableToolNames(tools?: Record<string, unknown>,
activeTools?: readonly string[]): string[] | undefined

---

## Tests

- Added four new tests in model-tracing.test.ts:
  1. startStep() does not create MODEL_INFERENCE
2. startInference() snapshots latest inference context into the emitted
span
3. MODEL_INFERENCE.startTime is recorded at startInference() (excluding
earlier delays)
4. Receiving chunks auto-creates MODEL_INFERENCE if caller never called
startInference()

- Added unit tests for getStepAvailableToolNames covering activeTools
precedence (including explicit []), fallback to tools keys, empty-tool
cases, and undefined behavior.

---

## Files Modified

- .changeset/solid-snakes-double.md
- observability/mastra/src/model-tracing.ts
- observability/mastra/src/model-tracing.test.ts
- packages/core/src/observability/types/tracing.ts
- packages/core/src/observability/utils.ts
- packages/core/src/observability/utils.test.ts
- packages/core/src/agent/durable/workflows/steps/llm-execution.ts
-
packages/core/src/loop/workflows/agentic-execution/llm-execution-step.ts
- packages/core/src/llm/model/model.loop.ts
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: Claude <noreply@anthropic.com>

* feat(observability): route eval scores through ScoreEvent bus (#16185)

## Description

Wires Mastra eval scores through the unified observability bus so every
exporter implementing `onScoreEvent` automatically receives them.
Replaces the deprecated per-exporter `addScoreToTrace` side-channel that
bypassed the pipeline. Also unblocks Mastra spans appearing under
Braintrust experiment eval spans (#11097) as a side effect of the
unified path.

- Producer alignment: `packages/core/src/mastra/hooks.ts` no longer
iterates exporters with `addScoreToTrace`. `MastraScorer.run()` (already
on main from PR #14920) remains the single producer of `ScoreEvent`s
through `mastra.observability.addScore()`.
- Consumer side: implemented `onScoreEvent` on Langfuse
(`LangfuseClient.score.create`), Laminar (`/v1/evaluators/score`),
Braintrust (`logger.logFeedback`), Datadog
(`tracer.llmobs.submitEvaluation`), and LangSmith
(`Client.createFeedback`).
- Backwards compatibility: Langfuse `addScoreToTrace` and Laminar
`_addScoreToTrace` are preserved as `@deprecated` wrappers that forward
to a shared private `submitScore` helper.
- Bug fix in `observability/mastra/src/recorded.ts`: `buildScoreEvent`
now forwards `scorerName` and `targetEntityType` from `ScoreInput` to
`ExportedScore` (without this, exporters silently fell back to
`scorerId`).
- Tests: rewrote core hook test to assert the hook no longer publishes a
`ScoreEvent`; added per-exporter `onScoreEvent` tests; added a
regression test for `buildScoreEvent` field forwarding.
- Seven single-package changesets created (`@mastra/core`,
`@mastra/observability`, `@mastra/langfuse`, `@mastra/laminar`,
`@mastra/braintrust`, `@mastra/datadog`, `@mastra/langsmith`) per the
project's changeset style guide.

## Related Issue(s)

Fixes #10896 (also unblocks #11097)

## Type of Change

- [ ] Bug fix (non-breaking change that fixes an issue)
- [x] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Code refactoring
- [ ] Performance improvement
- [ ] Test update

## Checklist

- [ ] I have made corresponding changes to the documentation (if
applicable)
- [x] I have added tests that prove my fix is effective or that my
feature works
- [ ] I have addressed all Coderabbit comments on this PR

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## ELI5

When Mastra assigns an evaluation score, this PR sends that score
exactly once into a single observability "bus" so all
monitoring/exporter integrations (Langfuse, Laminar, Braintrust,
Datadog, LangSmith, etc.) automatically receive it — avoiding
exporter-specific wiring and duplicate deliveries.

## Overview

Routes Mastra evaluation scores through the unified observability
ScoreEvent bus (mastra.observability.addScore) so any exporter
implementing onScoreEvent receives them. MastraScorer.run() remains the
single producer of ScoreEvent; the core scorer hook no longer iterates
exporters to call addScoreToTrace. Per-exporter
addScoreToTrace/_addScoreToTrace behaviors are preserved as @deprecated
wrappers that forward to shared submit helpers where applicable. Fixes a
bug in buildScoreEvent so scorerName and targetEntityType from
ScoreInput are forwarded into emitted ExportedScore.

## Changes by Component

### Core (@mastra/core)
- Removed exporter-iteration and addScoreToTrace republishing from
createOnScorerHook; the hook now only persists scores to the legacy
scores store.
- Ensures MastraScorer.run() is the single producer of ScoreEvent.
- Registers static workflow-step scorers synchronously during
addWorkflow and resolves dynamic scorers asynchronously so scorers are
registered before use.
- Updated core test to assert the hook does not emit ScoreEvent.

### Observability (@mastra/observability)
- Fix: buildScoreEvent (observability/mastra/src/recorded.ts) now
includes scorerName and targetEntityType in emitted ScoreEvent.
- Bus routing: route-event now prefers onScoreEvent; if an exporter
lacks onScoreEvent but implements deprecated addScoreToTrace, the bus
maps the ScoreEvent into the legacy addScoreToTrace payload and calls it
(avoids duplicate delivery when both exist).
- Added unit tests for score routing behavior and the buildScoreEvent
regression.

### Exporter Integrations (new onScoreEvent handlers + tests)
- Langfuse (@mastra/langfuse)
- Added onScoreEvent → LangfuseClient.score.create with a shared
submitScore helper and centralized error logging.
- Preserved addScoreToTrace as a deprecated wrapper forwarding to
submitScore.
- Tests updated to cover onScoreEvent and mark addScoreToTrace
deprecated.

- Laminar (@mastra/laminar)
- Added onScoreEvent delegating to a private submitScore that POSTS to
/v1/evaluators/score.
- submitScore converts trace/span IDs to Laminar/OTel UUIDs, merges
metadata/reason, enforces timeouts via AbortController, and logs
non-OK/timed-out responses.
- Preserved _addScoreToTrace as a @deprecated wrapper forwarding to
submitScore.
  - Tests verify POST payload mapping and wrapper behavior.

- Braintrust (@mastra/braintrust)
- Added onScoreEvent that calls logger.logFeedback using spanId (falling
back to traceId) and skips when neither is present.
- Maps scorer name/ID, reason, and metadata into the feedback payload
and logs failures.
  - Tests cover span/trace fallback and skip behavior.

- Datadog (@mastra/datadog)
- Added onScoreEvent: validates traceId/spanId, looks up exported
dd-trace span context (traceState), and calls
tracer.llmobs.submitEvaluation with mapped fields
(label/value/metricType/mlApp/timestamp/reason/metadata).
- Drops scores when identifiers or exported span context are missing and
warns when scores arrive before the span is exported.
- Tests cover successful submission, unknown IDs (no-op), and dropping
out-of-order scores.

- LangSmith (@mastra/langsmith)
- Added onScoreEvent that submits Client.createFeedback when a
spanId→LangSmith runId mapping exists.
- Introduced a configurable bounded LRU spanId→runId cache
(runIdCacheMaxEntries) populated from RunTree building; evicts oldest
entries when full and is cleared on shutdown.
- Ensures authoritative scorerId/scoreSource are preserved in sourceInfo
and ignores events when span/run mapping is absent.
- Tests cover mapping semantics, eviction, metadata protection, and
skipping unseen spans.

## Tests

- Core hook test rewritten to assert the hook does not publish
ScoreEvent.
- Per-exporter onScoreEvent test suites added for Langfuse, Laminar,
Braintrust, Datadog, and LangSmith covering field mapping, identifier
fallbacks, ordering/dropping, timeout handling, and cache eviction.
- Bus tests added to assert onScoreEvent preferred and legacy
addScoreToTrace fallback behavior.
- Regression test added to ensure buildScoreEvent forwards scorerName
and targetEntityType.

## Backwards Compatibility

- Deprecated addScoreToTrace/_addScoreToTrace preserved (where present)
as wrappers that forward to the new onScoreEvent/submit helpers to avoid
breaking existing callers.
- Observability interface docs updated to deprecate addScoreToTrace in
favor of onScoreEvent/ScoreEvent bus.
- No breaking public API surface changes beyond documented additions and
deprecations.

## Commits & Release Notes

- Seven package-scoped changesets created for @mastra/core,
@mastra/observability, @mastra/langfuse, @mastra/laminar,
@mastra/langsmith, @mastra/datadog, and @mastra/braintrust documenting
onScoreEvent support and the buildScoreEvent fix.
- Exporter-specific notes documented (Datadog ordering caveat; Laminar
timeout/error logging; LangSmith runId cache behavior).

## Related Issues

Fixes #10896 and unblocks #11097.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

* Fix flaky test by sorting thread IDs in memory handler test (#16360)

## Description

This PR fixes a flaky test in the memory handlers test suite. The test
was asserting thread IDs in a specific order without guaranteeing that
order from the API response. By sorting the thread IDs before
comparison, the test now reliably passes regardless of the order in
which threads are returned.

no changset required.

## Related Issue(s)

<!-- Link to the issue(s) this PR addresses, if applicable -->

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Code refactoring
- [ ] Performance improvement
- [ ] Test update

## Checklist

- [ ] I have made corresponding changes to the documentation (if
applicable)
- [x] I have added tests that prove my fix is effective or that my
feature works
- [ ] I have addressed all Coderabbit comments on this PR

## Test Plan

The existing test in
`packages/server/src/server/handlers/memory.test.ts` now passes
reliably. The change only affects test assertion order and does not
modify any production code behavior.

https://claude.ai/code/session_01RGNgHJmAGDZM8xEsSA7GW1

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## ELI5

A test was failing unpredictably because it expected thread IDs in a
specific order, but the API doesn't guarantee a consistent order. The
fix sorts the thread IDs before comparing them, making the test
reliable.

## Changes

Updated the `Authorization - Reserved Context Keys` test in the memory
handlers test suite to make thread ID assertions order-independent by
sorting the returned thread IDs before comparison. This eliminates
flakiness caused by the `filterAccessible` FGA filter not guaranteeing
stable return order.

**Files changed:**
- `packages/server/src/server/hand…
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Documentation Improvements or additions to documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[FEATURE] Add eval scoring support to all exporters

2 participants