Skip to content

Require pushed environments for eval uploads#612

Open
d42me wants to merge 1 commit intomainfrom
improvement/eval-upload-push-env-prompt
Open

Require pushed environments for eval uploads#612
d42me wants to merge 1 commit intomainfrom
improvement/eval-upload-push-env-prompt

Conversation

@d42me
Copy link
Copy Markdown
Contributor

@d42me d42me commented May 7, 2026

Summary

  • Block prime eval push from creating uploads against bare local environment names.
  • Prompt users to run prime env push <env> and retry with an owner-qualified environment slug before uploading.
  • Update eval push help/tests to reflect that uploads require a published environment.

Validation

  • uv run pytest packages/prime/tests/test_eval_push.py -q
  • uv run pytest packages/prime/tests/test_eval_help.py packages/prime/tests/test_eval_push.py -q
  • uv run ruff check packages/prime/src/prime_cli/commands/evals.py packages/prime/tests/test_eval_push.py
  • uv run ty check packages/prime/src/prime_cli/commands/evals.py packages/prime/tests/test_eval_push.py

Note

Medium Risk
Changes prime eval push behavior to hard-fail when given an unqualified environment name, which may break existing workflows/scripts that relied on local env names.

Overview
prime eval push now requires a published environment slug (owner/name) when associating an environment with an uploaded evaluation; passing a bare env name triggers a clear error prompting prime env push and a retry with an owner-qualified slug.

CLI help text, examples, and error tips were updated accordingly, and tests were adjusted/added to assert the new enforcement and that the API receives environments=[{"slug": ...}].

Reviewed by Cursor Bugbot for commit 854710b. Bugbot is set up for automated code reviews on this repo. Configure here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant