Invisible token usage tracker for AI CLI tools
Built in Rust. Daemon-powered. 5 MB idle. Reports in 7 ms. Your workflow never notices.
toki = token inspector — sounds like tokki (토끼, rabbit in Korean). Fast and light, just like one.
Not just vibe-coded — carefully architected by a professional developer.
Looking for a GUI? Toki Monitor is a macOS menu bar app with a real-time dashboard, animated rabbit indicator, and anomaly alerts — built on the toki daemon.
Multi-device sync? Toki Sync Server aggregates token usage across all your devices. Self-hosted, works with the built-in
toki settings synccommands.
- Quick Start
- Who is this for?
- How It Works
- Performance
- Privacy & Security
- Commands
- Multi-Device Sync
- Cost Calculation
- Supported Providers
- Planned Features
- Sponsor
- License
# Install (macOS)
brew tap korjwl1/tap
brew install toki
# toki auto-detects ~/.claude and ~/.codex. No config needed.
# 1. Start the daemon
toki daemon start
# 2. Real-time event stream (in another terminal)
toki trace
# 3. Reports
toki report daily --since 20260301
toki report --provider claude_code
toki report monthly
# 4. PromQL-style queries (instant)
toki query 'sum by (model)(toki_tokens_total[1h])'
toki query -z Asia/Seoul 'sum by (model)(toki_tokens_total[1d])'
# 5. Remote query (via sync server)
toki query --remote 'sum by (model)(toki_tokens_total[1h])'
# 6. Range query (with time window — via report)
toki report --since 20260301 --until 20260331 query 'sum(usage{since="20260301"}[1d]) by (project)'
toki report query 'events{since="20260320"}'-
Your terminal freezes on every token report? toki is 14x faster on cold start and 1,700x faster on reports. Even 2 GB of data comes back in 7 ms.
-
Need more than "total tokens"? Per-model, per-session, per-project, per-day breakdowns with PromQL-style queries. Filter by time range, group by any dimension, track costs — all in one command.
-
Don't want to set up OpenTelemetry? No collector, no config files, no environment variables. Install toki, run it, done. It reads your existing session files directly — including months of history from before you installed it.
-
Using multiple AI CLI tools? toki tracks Claude Code and Codex CLI in a single unified view. Filter by
--providerwhen you need per-tool breakdown.
Docker-like daemon/client architecture:
toki daemon start # always-on server (≈ dockerd)
toki trace # real-time stream (≈ docker logs -f)
toki report # instant TSDB query (≈ docker ps)
- daemon — watches session logs from configured providers (Claude Code, Codex CLI), parses events, writes to per-provider embedded TSDBs (fjall). 4 base threads + 2 per connected trace client. Zero overhead when no trace clients are connected.
- trace — connects to the daemon over UDS for real-time JSONL event streaming. Supports multiple sinks (
--sink uds://,--sink http://) for relaying to other services. - report — sends a query to the daemon, gets merged results from all provider TSDBs. Always fast, always indexed. Filter by
--providerto query a single provider.
toki sits at 5 MB idle, near-zero CPU, and answers any report in 7 ms. Most alternatives re-read everything from scratch on every invocation — toki indexes once, then gets out of your way.
Benchmarked against ccusage (Node.js) and zzusage (Zig) on the same dataset, disk cache purged before each run.
14x faster than ccusage, similar speed to zzusage but with 93% less memory.
In normal operation, toki resumes from its last checkpoint — only new data gets indexed.
Cold Start detailed data
| Data Size | toki | ccusage | zzusage | toki vs ccusage |
|---|---|---|---|---|
| 100 MB | 0.11 s | 2.38 s | 0.13 s | 21x faster |
| 200 MB | 0.16 s | 3.09 s | 0.18 s | 19x faster |
| 300 MB | 0.27 s | 4.47 s | 0.27 s | 16x faster |
| 400 MB | 0.31 s | 5.07 s | 0.32 s | 16x faster |
| 500 MB | 0.39 s | 6.06 s | 0.40 s | 15x faster |
| 1 GB | 0.78 s | 10.88 s | 0.76 s | 14x faster |
| 2 GB | 1.54 s | 21.53 s | 1.41 s | 14x faster |
| Data Size | toki | ccusage | zzusage |
|---|---|---|---|
| 100 MB | 37 MB | 126 MB | 165 MB |
| 200 MB | 38 MB | 127 MB | 246 MB |
| 300 MB | 67 MB | 127 MB | 421 MB |
| 400 MB | 69 MB | 127 MB | 492 MB |
| 500 MB | 71 MB | 126 MB | 615 MB |
| 1 GB | 119 MB | 127 MB | 1,209 MB |
| 2 GB | 166 MB | 126 MB | 2,311 MB |
Why does matching zzusage matter? toki does strictly more work per line — TSDB writes, rollup aggregation, checkpoint persistence, and schema validation. zzusage skips all of this. Despite the extra workload, toki matches zzusage in wall-clock time.
~7 ms regardless of data size — 1,742x faster than ccusage at 2 GB.
Report detailed data
| Data Size | toki (warm) | toki (cold disk) | ccusage | zzusage | warm vs ccusage | warm vs zzusage |
|---|---|---|---|---|---|---|
| 100 MB | 0.007 s | 0.16 s | 2.38 s | 0.13 s | 358x | 20x |
| 200 MB | 0.007 s | 0.15 s | 3.09 s | 0.18 s | 435x | 25x |
| 300 MB | 0.007 s | 0.15 s | 4.47 s | 0.27 s | 602x | 37x |
| 400 MB | 0.008 s | 0.14 s | 5.07 s | 0.32 s | 658x | 41x |
| 500 MB | 0.008 s | 0.16 s | 6.06 s | 0.40 s | 785x | 51x |
| 1 GB | 0.009 s | 0.15 s | 10.88 s | 0.76 s | 1,153x | 81x |
| 2 GB | 0.012 s | 0.17 s | 21.53 s | 1.41 s | 1,742x | 114x |
| Data Size | toki (warm) | toki (cold disk) | ccusage | zzusage |
|---|---|---|---|---|
| 100 MB | 5 MB | 8 MB | 126 MB | 165 MB |
| 500 MB | 5 MB | 8 MB | 126 MB | 615 MB |
| 1 GB | 5 MB | 8 MB | 127 MB | 1,209 MB |
| 2 GB | 10 MB | 10 MB | 126 MB | 2,311 MB |
| Data Size | toki (warm) | toki (cold disk) | ccusage | zzusage |
|---|---|---|---|---|
| 100 MB | 0% | 14% | 101% | 20% |
| 500 MB | 0% | 18% | 100% | 76% |
| 1 GB | 1% | 18% | 100% | 102% |
| 2 GB | 0% | 12% | 101% | 122% |
After cold start, toki drops to background-level resource usage.
| CPU | Memory | DB Size |
|---|---|---|
| ~0% | 5 MB | ~3% of source data (2 GB sessions → 64 MB TSDB) |
toki is the only tool here with a persistent idle state. The others pay full resource cost on every invocation.
Measured on Apple M1 MacBook Air (8 GB RAM), macOS, power saving off. Reproduce:
sudo -v && python3 benches/benchmark.py run --purge --tool all
toki is privacy-safe by architecture, not by policy.
- No prompt access: the JSONL parser only deserializes token counts and model name from
"assistant"lines. Prompts, responses, file contents, and thinking blocks are never loaded into memory — serde skips them without allocation. - No network transmission of your data: all processing is local. The only outbound request is an optional pricing fetch from the public LiteLLM repo (
--no-costto disable). - No conversation logging: the TSDB stores only timestamp, model name, session ID, source file path, project name, and token count integers.
- Read-only access: toki only reads session files. It never writes to or modifies any CLI tool's data.
toki daemon start # Start (background)
toki daemon start --foreground # Foreground (for debug)
toki daemon stop # Stop
toki daemon restart # Restart (reload settings)
toki daemon status # Check status
toki daemon reset # Wipe DB + reinitialize# Summary
toki report
toki report --provider claude_code
toki report --since 20260301 --until 20260331
# Time grouping
toki report daily --since 20260301
toki report weekly --start-of-week tue
toki report monthly
# Session/project filters
toki report --group-by-session
toki report --project toki
# Range queries (with --since/--until time window)
toki report --since 20260301 --until 20260331 query 'sum(usage[1d]) by (project)'
toki report query 'events{since="20260320"}'
toki report query 'usage[1d] offset 7d'# Instant PromQL query (top-level command, no --since/--until)
toki query 'sum by (model)(toki_tokens_total[1h])'
toki query -z Asia/Seoul 'sum by (model)(toki_tokens_total[1d])'
# type filter and regex operator
toki query 'sum(toki_tokens_total{type=~"input|output"}[1h])'
# Remote (via sync server)
toki query --remote 'sum by (model)(toki_tokens_total[1h])'
# Output format and options
toki query -w 'sum by (model)(toki_tokens_total[1d])' # wide table
toki query --output-format json 'toki_tokens_total[1h]'
toki query --no-cost 'toki_tokens_total[1h]'
toki report querystill works for range queries with--since/--untiltime windows.
For the full command reference, query syntax, and settings options, see the Usage Guide.
toki trace # JSONL stream to stdout
toki trace --sink uds:///tmp/toki.sock # Relay to UDS
toki trace --sink http://localhost:8080/events # Relay via HTTPtoki settings # Open TUI
toki settings set providers --add codex # Add a provider
toki settings list # List alltoki settings sync enable --server <host> # Opens browser for authentication (device code flow)
toki settings sync disable # Prompts to delete remote data
toki settings sync disable --delete # Delete this device's data from server
toki settings sync disable --keep # Keep remote data, only disable locally
toki settings sync status # Connection info
toki settings sync devices # Registered devices
toki settings sync rename <new-name> # Rename this deviceSync token usage across multiple machines to a central toki-sync server. All your devices' data in one place — queryable via PromQL, visible in the web dashboard or Toki Monitor.
# Connect to your sync server (opens browser for authentication)
toki settings sync enable --server sync.example.com
# For self-signed TLS (IP-only servers)
toki settings sync enable --server 1.2.3.4 --insecure
# Check status
toki settings sync status
# List registered devices
toki settings sync devices
# Query server data from CLI
toki query --remote 'sum by (model)(toki_tokens_total)'
# Disable sync
toki settings sync disable # Prompts to delete remote data
toki settings sync disable --delete # Delete this device's data from server
toki settings sync disable --keep # Keep remote data, only disable locally- Daemon sync thread connects to toki-sync server via TLS TCP (persistent connection)
- Events are batched (1,000/batch), zstd-compressed (≥100 items), and sent with ACK-based flow control
- On disconnect: events accumulate locally in fjall DB, delta-synced on reconnect
- JWT auto-refresh, exponential backoff (2s→300s cap), wake detection
- Settings hot-reload:
toki settings sync enabletakes effect without daemon restart
Sync is opt-in and off by default. When enabled, only token counts and metadata (model, session ID, project name) are transmitted — never prompts or responses. TLS encrypts all traffic. Each user's data is isolated on the server via label injection.
All outputs include estimated cost (USD) per model, sourced from LiteLLM community pricing.
- First run: downloads LiteLLM JSON → filters by
litellm_provider(Anthropic, OpenAI, Gemini) → caches to~/.config/toki/pricing.json - Subsequent runs: HTTP ETag conditional request → 304 if unchanged (~50 ms, no body)
- Offline: uses cached data; if no cache, cost column is omitted
--no-cost: skips price fetch entirely
| Provider | CLI Tool | Data Format | Status |
|---|---|---|---|
claude_code |
Claude Code | JSONL (append-only) | Supported |
codex |
Codex CLI | JSONL (append-only) | Supported |
| (gemini) | Gemini CLI | JSON (full rewrite) | Planned |
Each provider gets its own isolated database (~/.config/toki/<provider>.fjall). Reports merge results across all enabled providers by default, or filter to a single provider with --provider.
| Feature | Description | Status |
|---|---|---|
| Gemini CLI | Google Gemini CLI provider support | Planned |
toki-sync |
Multi-device support — sync usage data across machines | Available |
Have a feature request or found a bug? Open an issue.
| Document | Description |
|---|---|
| Architecture & Design | Daemon threads, TSDB schema, rollup strategy, checkpoint recovery, data flow |
| Usage Guide | Detailed command reference, output formats, library API, examples |
| JSONL Format Reference | Claude Code JSONL structure, line types, parsing optimizations |
| Benchmark Details | Full comparison methodology, architecture analysis, scaling predictions |
| Codex CLI Analysis | Codex CLI local data format, token structure, parsing strategy |
| Gemini CLI Analysis | Gemini CLI local data format analysis (future provider) |
| Why Not OpenTelemetry? | Why toki parses local files instead of receiving OTEL data |
| OTEL Comparison | OpenTelemetry implementation details: Claude Code vs Gemini CLI vs toki |
| Purpose | Choice | Rationale |
|---|---|---|
| Database | fjall 3.x | Pure Rust LSM-tree, fits TSDB keyspace model |
| Concurrency | std::thread + crossbeam-channel | No async runtime conflicts, library-safe |
| Parallel scan | rayon | Cold start parallel file processing |
| File watching | notify 6.x | FSEvents (macOS), inotify (Linux), polling fallback per provider |
| Serialization | bincode (DB), serde_json (JSONL) | Minimal binary overhead |
| Hashing | xxhash-rust 0.8 (xxh3) | Checkpoint line identification (30 GB/s) |
| HTTP | ureq 2.x | Synchronous, ETag conditional requests |
| CLI | clap 4.x | Subcommands, global options |
| Tables | comfy-table 7.1 | Unicode table rendering |
| Sync protocol | toki-sync-protocol (shared crate) | Wire-compatible types, bincode serialization |
| TLS | native-tls 0.2 | Platform TLS for sync connections |
| IPC | Unix Domain Socket | Daemon-client NDJSON streaming |
src/
├── lib.rs # Public API: start(), Handle
├── main.rs # CLI binary (clap)
├── config.rs # Config + file-based settings
├── db.rs # fjall wrapper (7 keyspaces)
├── engine.rs # TrackerEngine: cold_start + watch_loop
├── writer.rs # DB writer thread (DbOp channel)
├── query.rs # TSDB query engine (report)
├── query_parser.rs # PromQL-style query parser
├── retention.rs # Data retention policy
├── checkpoint.rs # Reverse-scan, xxHash3 matching
├── pricing.rs # LiteLLM price fetch, ETag caching
├── settings.rs # Cursive TUI settings
├── common/
│ ├── types.rs # Shared types & traits
│ └── time.rs # Fast timestamp parser (0.1µs)
├── daemon/ # Daemon server components
│ ├── broadcast.rs # BroadcastSink (zero-overhead fan-out)
│ ├── listener.rs # UDS accept loop + multi-DB query merge
│ └── pidfile.rs # PID file management
├── sink/ # Output abstraction (Sink trait)
│ ├── print.rs # PrintSink (table/json → stdout)
│ ├── uds.rs # UdsSink (NDJSON → UDS)
│ └── http.rs # HttpSink (JSON POST)
├── providers/ # Per-provider parsers (Provider trait)
│ ├── mod.rs # Provider trait, FileParser trait, registry
│ ├── claude_code/ # Claude Code JSONL parser
│ │ ├── mod.rs # ClaudeCodeProvider impl
│ │ └── parser.rs # Session discovery + line parsing
│ └── codex/ # Codex CLI JSONL parser
│ ├── mod.rs # CodexProvider impl
│ └── parser.rs # Stateful parser (model tracking)
├── sync/ # Multi-device sync
│ ├── thread.rs # Sync loop, SyncToggle, wake detection
│ ├── client.rs # TCP+TLS client, auth, batch send
│ ├── protocol.rs # Re-exports from toki-sync-protocol
│ ├── backoff.rs # Exponential backoff (2s→300s)
│ └── credentials.rs # Keychain (macOS) / sync.json (Linux)
└── platform/mod.rs # FSEvents watcher + per-provider polling strategy
If toki is useful to you, consider sponsoring to support development.
For commercial use in paid products, please sponsor or reach out.