Add opt-in anonymous product analytics provider #210

Problem

We need a privacy-conscious way to understand which Agent Code features people actually use.

The immediate example is command usage: if commands are never run, we should know whether they should be deprecated, hidden, renamed, improved, or better documented. Without usage data, the app can only grow by adding features, not by pruning or educating around features that are confusing or unused.

This should be separate from the existing performance telemetry. Performance is primarily local/dev-oriented tracing and timing. Product analytics has a different purpose: anonymous, opt-in usage signals that help keep the software lean.

Desired Behavior

Add a shared analytics provider/service, similar in spirit to the existing performance service but with a product-analytics contract.

The system should be:

• opt-in only
• anonymous by default
• transparent about what is collected
• easy to disable
• safe to leave on for normal users
• structured enough to answer product questions over time

The first concrete event family should be command usage.

First Event Family: Command Usage

Track when a user runs a command palette command.

At minimum, record:

• command id
• command category/source if available
• timestamp
• app version/build channel if available
• coarse provider/session kind context when relevant, e.g. claude, codex, terminal, or none
• broad surface/context when relevant, e.g. Grid, Dispatch, Reader, Global Editor

Do not record:

• prompt text
• command arguments containing user content
• file paths
• cwd
• provider session ids
• Agent Code session ids
• transcript content
• terminal output
• raw clipboard contents

Privacy / Consent Requirements

Analytics must be opt-in.

Acceptance criteria around consent:

• Analytics are off by default.
• The user can enable/disable analytics in settings.
• The setting text clearly says this is anonymous product usage analytics, not performance tracing.
• The user can inspect what kind of events are collected, at least at a documented schema level.
• No sensitive content is collected.
• No stable personally identifying id is created unless we explicitly decide to add an anonymous install id, and that decision should be documented.
• If an anonymous install id is used, it must be resettable.

Architecture Notes

Create a shared analytics abstraction/provider rather than sprinkling ad hoc calls throughout the UI.

Possible shape:

• src/shared/analytics/ for event types and schema
• src/main/analytics/AnalyticsService.ts for persistence/upload/batching
• src/renderer/src/analytics/client.ts or similar renderer client
• IPC boundary for renderer-originated analytics events
• settings state for analytics opt-in

This should be conceptually parallel to performance telemetry, but separate from it:

• performance = timing/tracing/debuggability
• analytics = anonymous product usage decisions

Do not reuse performance event schemas if that makes privacy semantics ambiguous.

Storage / Transport

TBD. First implementation can be conservative.

Options to evaluate:

• local-only JSONL queue first, with no upload until policy is finalized
• batched upload to a hosted endpoint after opt-in
• explicit “export analytics sample” dev tool for validation

If remote upload is implemented, it must be batched, failure-tolerant, and never block the UI.

Product Questions This Should Answer

Command usage should eventually let us ask:

• Which commands are heavily used?
• Which commands are never used?
• Which commands are used only by power users/dev mode?
• Which features need better discoverability?
• Which commands should be deprecated, hidden, merged, or renamed?
• Which surfaces are most used: Grid, Dispatch, Reader, Global Editor, etc.?

Acceptance Criteria

• Add an opt-in analytics setting, default off.
• Add a shared analytics event schema for product usage events.
• Add a main-process analytics service/client boundary.
• Track command execution events through one centralized command execution path where practical.
• Command events include command id and coarse context only.
• Command events do not include user content, paths, prompts, cwd, transcripts, provider ids, or session ids.
• Analytics calls are no-op when disabled.
• Analytics failures do not affect command execution.
• Add focused tests for event sanitization and disabled/enabled behavior.
• Add thick WHY comments explaining why analytics is separate from performance and why content/path/session identifiers are excluded.

Open Questions

• Should v1 be local-only until the remote/privacy policy is fully settled?
• Do we need an anonymous install id, or can aggregate event counts work without one?
• If we use an install id, how is it reset?
• Where should analytics opt-in live in Settings?
• Should dev builds have a visible analytics debug panel/log tail?
• What backend, if any, should receive uploaded events?