LyreBot

A local-first, scientific AI agent for bioacoustics with a chat UI and inline widgets. Use LyreBot to analyze your audio recordings with BirdNET and ask questions about the results in natural language.

This is more than a chat interface - LyreBot uses an agent-based architecture where the LLM plans analysis steps, Python executes them, and the results are the source of truth. You can attach audio files or folders, and ask questions like:

"What species were detected?"
"Show me the species distribution"
"Filter for common species in Germany"
"Generate a spectrogram for the first blackbird detection"
"Let me hear a detection of the robin"

Why is this helpful?

AI-powered bioacoustics tools can play a critical role in conservation by automating the analysis of large-scale audio datasets. Despite the availability of many open tools with graphical interfaces (e.g., the BirdNET Analyzer), end-to-end processing pipelines often remain complex and difficult to use. Agents such as LyreBot add an additional layer of abstraction by orchestrating these tools and can lower the barrier to entry, making advanced bioacoustic analysis more accessible to researchers and conservation practitioners without extensive technical expertise.

Read more about the motivation and vision for this project in our companion memo: Agentic AI and the Next Phase of Tool-Assisted Conservation.

⚠️ Disclaimer

This project is a proof-of-concept and research prototype that is still very rough around the edges. However, we are releasing it to the public to gather feedback and understand how such a tool can best serve the bioacoustics community.

Note: No data is sent to any external servers except for the LLM API calls (Anthropic). All audio analysis is done locally on your machine. See Architecture & Workflow for details.

Features

Chat Interface: Natural language queries about bird detections
BirdNET Integration: Analyze audio files locally using BirdNET
Real-time Streaming: WebSocket-based updates show analysis progress as it happens
Inline Widgets: Tables, plots (bar, box, scatter, histogram), spectrograms, audio playback, and downloads
Markdown Responses: Rich formatted responses with proper styling
CSV Export: Download detection results for use in spreadsheets or research
Dark/Light Mode: Toggle between themes in settings
Local-First: Your data stays on your machine
Agent-Based: LLM plans, Python executes - results are source of truth

Interface Preview

Prerequisites

Node.js 18+ and npm
Python 3.10+
Rust (for Tauri) - Install Rust
Anthropic API Key - Get one here

Installation

Quick Setup (macOS/Linux)

git clone https://github.com/birdnet-team/lyrebot.git && cd lyrebot && ./setup.sh

Windows Setup

Windows users should follow the Manual Setup below. Make sure you have:

Python 3.10+ (check "Add to PATH" during install)
Node.js 18+
Rust (for Tauri desktop app, optional for web-only use)

Manual Setup (All Platforms)

1. Clone the repository

git clone https://github.com/birdnet-team/lyrebot.git
cd lyrebot

2. Set up the Python backend

cd backend

# Create virtual environment
python3 -m venv .venv          # On Windows: python -m venv .venv
source .venv/bin/activate      # On Windows: .venv\Scripts\activate

# Install dependencies
pip install -r requirements.txt

3. Set up the frontend

cd ../app

# Install dependencies
npm install

Running the Application

Quick Start (macOS/Linux, After Setup)

./run-all.sh

Then open http://localhost:1420 in your browser.

Development Mode (All Platforms)

Terminal 1 - Start the backend:

cd backend
source .venv/bin/activate      # On Windows: .venv\Scripts\activate
python run.py

Terminal 2 - Start the frontend:

cd app
npm run dev

Then open http://localhost:1420 in your browser.

With Tauri (Desktop App)

cd app
npm run tauri dev

This will:

Start the Vite dev server
Build and launch the Tauri desktop application
Attempt to auto-start the Python backend

Configuration

In-App Settings

Click the gear icon in the top-right corner
Enter your Anthropic API Key
Set the Allowed Data Root (directory where your audio files are)
Toggle Dark/Light mode as preferred
Click Save Settings

Environment Variables (Optional)

You can also configure via environment variables:

# In backend/.env
ANTHROPIC_API_KEY=your-key-here
ALLOWED_DATA_ROOT=/path/to/your/audio/files

Usage

Analyzing Audio Files

Click the paperclip icon to attach files
Select audio files (WAV, MP3, FLAC, OGG, M4A) or a folder
Ask a question like:
- "Analyze these recordings"
- "What species were detected?"
- "Show me the species distribution"
- "Generate a spectrogram for the first blackbird detection"
- "Export results to CSV"
- "Let me hear a detection of the robin"

General Questions

You can also ask general bioacoustics questions without attachments:

"Tell me about blue tits"
"What does confidence mean in BirdNET?"
"Explain spectrograms"

Architecture & Workflow

LyreBot operates as a closed-loop scientific agent. Instead of just "chatting" about birds, it uses a multi-step orchestration process to ensure that all answers are backed by verifiable data analysis performed locally on your machine.

  ┌────────────────┐      ┌───────────────────────────────────┐      ┌─────────────┐
  │  Desktop App   │      │       FastAPI Backend             │      │ Local Data  │
  │ (React/Tauri)  │ <~~> │ - Agent Loop (LLM Planner/Writer) │ <──> │ - DuckDB    │
  │ UI & Widgets   │  WS  │ - Scientific Tools (BirdNET/SQL)  │      │ - Audio     │
  └────────────────┘      └───────────────────────────────────┘      └─────────────┘

Planner: The LLM (Anthropic Claude) receives your query and decides which tools it needs to use (e.g., "Run BirdNET analysis on this folder", "Query the database for robin detections").
Execute: The Python backend carries out these tasks locally via WebSocket streaming, providing real-time progress updates. It runs the BirdNET model, stores results in a local DuckDB database, and uses scientific libraries to process audio or create plots.
Writer: The LLM takes the raw data from the tools and translates it into a human-readable scientific summary, attaching interactive widgets (plots, tables, audio clips) directly to the chat bubble.

The JSON Protocol

LyreBot uses a structured JSON exchange to bridge the gap between natural language and scientific execution. Here is a simplified look at the data flow:

1. Plan (LLM → API)

When you ask a question, the LLM generates a structured plan of tool calls:

{
  "steps": [
    {
      "tool": "query_results",
      "arguments": { "species": "Common Blackbird", "limit": 5 }
    },
    {
      "tool": "get_summary",
      "arguments": { "run_id": "run_20240204_1200" }
    }
  ],
  "assistant_text": "I'll check the detections for blackbirds and summarize the overall run stats."
}

2. Execute & Render (API → UI)

After the backend executes the tools, it sends back a response containing both the natural language answer and the UI components:

{
  "response": {
    "response_text": "I found 5 detections of the Common Blackbird.",
    "widgets": [
      {
        "type": "plot",
        "title": "Species Distribution",
        "series": [{ "name": "Detections", "x": ["Robin", "Blackbird"], "y": [12, 5] }]
      }
    ]
  }
}

This ensures the UI can render rich scientific components regardless of which LLM is used, keeping the "intelligence" in the planning and the "rendering" in the frontend.

Available Tools

The LLM has access to these tools for analysis:

Tool	Description
`get_species_at_location`	Get expected species for a GPS location (for filtering)
`register_dataset`	Register audio files for analysis
`create_run`	Create a new analysis run with settings
`run_analysis`	Execute BirdNET analysis
`job_status`	Check analysis progress
`get_summary`	Get aggregated analysis statistics
`query_results`	Query specific detection results
`get_plot_data`	Get data for visualizations (bar charts, box plots, scatter plots, histograms, timelines)
`get_spectrogram`	Generate spectrogram images
`get_audio_clip`	Extract audio segments for playback
`generate_report`	Create markdown reports
`export_csv`	Export detections to CSV for download

Building for Production

Build the Backend

The backend runs as a standalone Python process. For distribution, consider using PyInstaller:

cd backend
pip install pyinstaller
pyinstaller --onefile run.py

Build the Desktop App

cd app
npm run tauri build

The built application will be in app/src-tauri/target/release/.

Project Structure

lyrebot/
├── app/                     # Tauri + React frontend
│   ├── src/
│   │   ├── components/      # React components
│   │   ├── App.tsx          # Main application
│   │   ├── api.ts           # Backend API client
│   │   └── types.ts         # TypeScript types
│   ├── src-tauri/           # Tauri configuration
│   └── package.json
│
├── backend/                 # FastAPI backend
│   ├── app/
│   │   ├── main.py          # FastAPI app
│   │   ├── models.py        # Pydantic models
│   │   ├── config.py        # Configuration
│   │   ├── database.py      # DuckDB operations
│   │   ├── birdnet.py       # BirdNET wrapper
│   │   ├── tools.py         # LLM tools
│   │   └── llm.py           # Agent loop
│   └── requirements.txt
│
└── README.md

Troubleshooting

Backend won't start

Ensure Python 3.10+ is installed
Check if port 8765 is available
Verify all dependencies are installed

Can't connect to backend

Make sure the backend is running on http://127.0.0.1:8765
Check the browser console for CORS errors
Try restarting both frontend and backend

BirdNET analysis fails

Ensure audio files are in a supported format (WAV, MP3, FLAC, OGG, M4A)
Check file paths are under the allowed data root
Try reinstalling dependencies: pip install -r requirements.txt

API key issues

Verify the key is correctly pasted in Settings
Ensure there are no extra spaces
Check your Anthropic account has available credits

Contributing

Contributions are welcome! Please feel free to submit issues and pull requests.

Areas for improvement include:

adding more analysis tools
add other LLM providers
add a local LLM option
improving the UI/UX

This project was largely co-developed with AI tools, so don't be afriad to leverage them in your contributions as well!

If you open a pull request, please make sure to address a specific issue or feature request instead of many changes at once.

License

Source Code: The source code for this project is licensed under the MIT License.
Models: The models used in this project are licensed under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License (CC BY-NC-SA 4.0).

Please ensure you review and adhere to the specific license terms provided with each model.

Please note that all educational and research purposes are considered non-commercial use and it is therefore freely permitted to use BirdNET models in any way.

Funding

Our work in the K. Lisa Yang Center for Conservation Bioacoustics is made possible by the generosity of K. Lisa Yang to advance innovative conservation technologies to inspire and inform the conservation of wildlife and habitats.

The development of BirdNET is supported by the German Federal Ministry of Research, Technology and Space (FKZ 01|S22072), the German Federal Ministry for the Environment, Climate Action, Nature Conservation and Nuclear Safety (FKZ 67KI31040E), the German Federal Ministry of Economic Affairs and Energy (FKZ 16KN095550), the Deutsche Bundesstiftung Umwelt (project 39263/01) and the European Social Fund.

Partners

BirdNET is a joint effort of partners from academia and industry. Without these partnerships, this project would not have been possible. Thank you!

Name		Name	Last commit message	Last commit date
Latest commit History 37 Commits
app		app
backend		backend
samples		samples
.gitignore		.gitignore
LICENSE		LICENSE
README.md		README.md
READMORE.md		READMORE.md
run-all.sh		run-all.sh
run-backend.sh		run-backend.sh
run-frontend.sh		run-frontend.sh
setup.sh		setup.sh
test_chat.py		test_chat.py

Folders and files

Latest commit

History

Repository files navigation

LyreBot

Why is this helpful?

⚠️ Disclaimer

Table of Contents

Features

Interface Preview

Prerequisites

Installation

Quick Setup (macOS/Linux)

Windows Setup

Manual Setup (All Platforms)

1. Clone the repository

2. Set up the Python backend

3. Set up the frontend

Running the Application

Quick Start (macOS/Linux, After Setup)

Development Mode (All Platforms)

With Tauri (Desktop App)

Configuration

In-App Settings

Environment Variables (Optional)

Usage

Analyzing Audio Files

General Questions

Architecture & Workflow

The JSON Protocol

1. Plan (LLM → API)

2. Execute & Render (API → UI)

Available Tools

Building for Production

Build the Backend

Build the Desktop App

Project Structure

Troubleshooting

Backend won't start

Can't connect to backend

BirdNET analysis fails

API key issues

Contributing

License

Funding

Partners

About

Resources

License

Uh oh!

Stars

Watchers

Forks

Releases

Contributors 1

Languages