This repository was archived by the owner on Oct 16, 2026. It is now read-only.

Repository navigation

feat(evals): set default targets so all evals work out of the box - #898

Merged

christso merged 40 commits into

mainfrom

feat/default-targets

Apr 2, 2026

Merged

feat(evals): set default targets so all evals work out of the box#898
christso merged 40 commits into
mainfrom
feat/default-targets

Commits on Apr 1, 2026
- feat(evals): set default targets so all evals work out of the box
  
  christso
  and
  claude
  committed
  1026e00
- feat(evals): set default targets so all evals work out of the box
  
  christso
  and
  claude
  committed
  ebe1688
- feat(evals): make default target env-var-driven for out-of-box evals
  
  christso
  and
  claude
  committed
  f74fb09
- fix(ci): use explicit include patterns instead of negated globs
  
  christso
  and
  claude
  committed
  d2102dc
- feat(cli): support negation patterns (!glob) in eval path resolution
  
  christso
  and
  claude
  committed
  37a526c
- fix(ci): remove --targets override so per-example targets auto-discover
  
  christso
  and
  claude
  committed
  71d77a5
- fix: remove deprecated workspace_template from mock target configs
  
  christso
  and
  claude
  committed
  df3a765
- fix(ci): add Gemini credentials to workflow .env
  
  christso
  and
  claude
  committed
  1191250
- feat(evals): add llm target and classify all evals as llm or agent
  
  christso
  and
  claude
  committed
  03f5503
- fix(evals): use default (copilot) instead of pi-cli for agent evals
  
  christso
  and
  claude
  committed
  b2c6a78
- chore(ci): increase eval workers from 1 to 3
  
  christso
  and
  claude
  committed
  0b04cf9
- fix(ci): exclude evals with local script providers from CI
  
  christso
  and
  claude
  committed
  5c53635
- fix(ci): add missing echo provider and install uv for local script evals
  
  christso
  and
  claude
  committed
  f3870d6
- fix(evals): make LLM eval assertions pass with generic models
  
  christso
  and
  claude
  committed
  d081bd6
- fix(evals): switch llm and grader targets to OpenRouter
  
  christso
  and
  claude
  committed
  f8d8e94
- fix(evals): switch per-example grader targets from azure to root grader
  
  christso
  and
  claude
  committed
  2a9f1c3
- feat(core): add target alias support for single-env-var provider switching
  
  christso
  and
  claude
  committed
  2185c65
- feat(core): add use_target for target delegation
  
  christso
  and
  claude
  committed
  6438c23
- refactor(targets): use use_target for llm and grader targets
  
  christso
  and
  claude
  committed
  6936380
- refactor(core): make provider optional when use_target is set
  
  christso
  and
  claude
  committed
  a076d4e
- fix(core): allow provider to be omitted when use_target is set
  
  christso
  and
  claude
  committed
  fddd943
- fix(core): allow use_target in targets-file.ts parser
  
  christso
  and
  claude
  committed
  3c39f70
- fix(ci): exclude copilot-log-eval from CI
  
  christso
  and
  claude
  committed
  7650b51
- fix(cli): catch before_all failures per eval file instead of aborting
  
  christso
  and
  claude
  committed
  3441f91
- fix(core): resolve use_target chains in orchestrator for grader targets
  
  christso
  and
  claude
  committed
  0dd936a
- fix(evals): restore workspace.template for mock agent evals
  
  christso
  and
  claude
  committed
  50eef93
- fix(ci): exclude evals with pre-existing workspace/batch bugs
  
  christso
  and
  claude
  committed
  595fc16
- fix(evals): fix remaining CI failures
  
  christso
  and
  claude
  committed
  41d1fad
- fix(ci): remove --verbose to reduce log size, make JUnit step non-fatal
  
  christso
  and
  claude
  committed
  64f9b40
- fix(ci): use --output instead of -o for JUnit path
  
  christso
  and
  claude
  committed
  2cf1004
- feat(ci): add eval results summary to GitHub Actions step summary
  
  christso
  and
  claude
  committed
  2844421
Commits on Apr 2, 2026
- fix: remove unused grader targets from offline-grader-benchmark
  
  christso
  and
  claude
  committed
  29ea7c1
- fix(ci): use npm package for copilot CLI instead of curl installer
  
  christso
  and
  claude
  committed
  a852e0b
- fix(ci): add Node 22 for copilot CLI compatibility
  
  christso
  and
  claude
  committed
  d8c9f8d
- debug(ci): remove tee pipe and limit to 2 eval sets for debugging
  
  christso
  and
  claude
  committed
  17431c2
- fix(evals): fix csv-analyzer rubrics criteria format
  
  christso
  and
  claude
  committed
  99c2f33
- fix(evals): keep skill-trigger assertions required, tag for exclusion
  
  christso
  and
  claude
  committed
  7e60324
- fix(evals): add csv-analyzer skill to workspace and set workspace template
  
  christso
  and
  claude
  committed
  8e91cdf
- fix(ci): include copilot logs in artifacts for debugging
  
  christso
  and
  claude
  committed
  61c1b74
- fix(evals): make csv-analyzer skill essential with proprietary formula
  
  christso
  and
  claude
  committed
  707761b

Commits

Commits on Apr 1, 2026

Commits on Apr 2, 2026