Skip to content

Add JSONL traces for agentic search streams - #1798

Open
RitwijParmar wants to merge 1 commit into
airweave-ai:mainfrom
RitwijParmar:codex/airweave-agentic-search-jsonl
Open

Add JSONL traces for agentic search streams#1798
RitwijParmar wants to merge 1 commit into
airweave-ai:mainfrom
RitwijParmar:codex/airweave-agentic-search-jsonl

Conversation

@RitwijParmar

@RitwijParmar RitwijParmar commented May 27, 2026

Copy link
Copy Markdown

Summary

Adds optional JSONL trace capture to scripts/agentic_search_stream.py:

  • new --jsonl <path> flag
  • writes request metadata, ordered raw stream events, per-event elapsed time, and a final summary
  • preserves the existing terminal renderer behavior
  • creates parent directories for trace files automatically

Why

Airweave's agentic search stream viewer is useful for interactive debugging, but once an agentic retrieval run scrolls by there is no replayable artifact for evals, regression analysis, or retrieval-quality review.

The JSONL output makes the same streamed planning/searching/evaluation/done events easy to archive and compare across prompts, filters, embedding configs, or retrieval strategy changes. That gives teams a lightweight path toward retrieval eval harnesses without changing the API surface.

Example:

python scripts/agentic_search_stream.py my-collection "find deployment docs" \
  --mode agentic \
  --jsonl traces/deployment-docs.jsonl

Verification

  • python3 -m py_compile scripts/agentic_search_stream.py
  • python3 scripts/agentic_search_stream.py --help
  • git diff --check

AI disclosure

I used an AI coding assistant to help draft and verify this contribution. I reviewed the patch before opening the PR.


Summary by cubic

Adds optional JSONL trace output to scripts/agentic_search_stream.py so agentic search streams are replayable for evals and regression checks. Default behavior is unchanged; traces are written only when --jsonl is provided.

  • New Features
    • Add --jsonl <path> to write stream events to a JSONL file.
    • Record request metadata, ordered events with sequence and elapsed_ms, plus a final summary with elapsed_s.
    • Auto-create parent directories and print the trace path at start and end.
    • Preserve the existing terminal renderer.

Written for commit 4dae56c. Summary will update on new commits. Review in cubic

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 1 file

Re-trigger cubic

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant