Skip to content
This repository was archived by the owner on Oct 16, 2026. It is now read-only.
This repository was archived by the owner on Oct 16, 2026. It is now read-only.

feat: compute costUsd from token usage via model pricing table #635

Closed as not planned

Description

@christso

Status Update (2026-04-05)

This issue remains a good core capability issue.

The Studio-specific dashboard ambitions are no longer urgent. If this work is done, the important part is still the underlying cost computation so cost data is present in result artifacts and available to any consumer (CLI, CI, Studio, or external tooling).

Revised Scope

In scope

  • pricing table for models that do not report cost directly
  • cost computation from token usage
  • clear precedence for provider-reported costUsd
  • user overrides/extensions for pricing data

Deprioritized

  • large Studio cost-attribution dashboard work
  • budget-control / optimization-center UI in Studio
  • making Studio the primary consumer of this feature

Acceptance Signals

  • costUsd is computed when token usage exists and provider cost is absent
  • provider-reported cost is never overridden
  • unknown models remain undefined rather than guessed
  • cost data is available in result artifacts for downstream consumers

Activity

  1. christso commented on Mar 27, 2026

    @christso
    CollaboratorAuthor

    Extension: Cost Attribution Views in Dashboard

    This issue currently defines the core cost computation engine. The following dashboard surface should be added:

    Dashboard capabilities for cost analytics

    • Cost attribution: Break down cost by evaluator (LLM-judge vs rubric vs code-grader), test category, target, and campaign
    • Cost efficiency metric: Cost per point of score improvement — helps prioritize which evaluators give best ROI
    • Budget tracking: Campaign-level budget with progress bar and projected total based on burn rate
    • Cost trends: Historical cost per run/campaign, identify cost spikes
    • Optimization suggestions: Flag expensive evaluators that could be replaced with cheaper alternatives (e.g., LLM-judge → rubric for simple checks)

    How this connects

    Research source

    • melagiri/code-insights — cost breakdowns by project and model, activity charts, multi-tool usage visualization
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions