Skip to content

CLAUDE.md

Latest commit

 

History

History
273 lines (201 loc) · 11 KB

CLAUDE.md

File metadata and controls

273 lines (201 loc) · 11 KB

CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

Makefile Commands

The project includes a Makefile with common development tasks. Run make help to see all available commands:

make setup         # Install project with all development dependencies
make install       # Install project with minimal dependencies
make test          # Run all tests
make test-cov      # Run tests with coverage report
make system-deps   # Install external system dependencies (ffmpeg, etc.)
make check         # Run all code checks (format + lint + typecheck)
make format        # Format code with black and isort
make lint          # Run flake8 linter
make typecheck     # Run mypy type checker
make build         # Build package
make clean         # Clean build artifacts
make publish       # Bump version, commit, tag, and push (triggers PyPI release)
make publish-patch # Publish patch version (same day increment)

Build & Development Commands

# Install for development (with all optional dependencies)
pip install -e ".[dev,rag,whisper,documents,tables,videos,audio,remote,tasks]"

# Format code
isort .
black .

# Lint and type check
flake8 .
mypy src/litemind

# Run all tests
pytest src/

# Run a single test file
pytest src/litemind/apis/tests/test_openai_api.py

# Run tests matching a pattern
pytest -k "tools" src/

# Run tests with coverage report
pytest --cov=src --cov-report=html:reports/coverage src/

# Build package
hatch clean && hatch build

Publishing to PyPI

Publishing is handled via GitHub Actions triggered by git tags. Use the Makefile commands:

# For a new day's release (bumps version to YYYY.M.D format)
make publish

# For same-day patch releases (bumps to YYYY.M.D.N format)
make publish-patch

These commands will:

  1. Update the version in src/litemind/__init__.py
  2. Commit the version bump
  3. Create a git tag (e.g., v2026.2.4)
  4. Push to origin with tags
  5. GitHub Actions then automatically publishes to PyPI via OIDC trusted publishing

CLI Tools

# Export repository to single file
litemind export --folder-path . --output-file exported.txt

# Validate model registry against live APIs
litemind validate --api openai gemini --models gpt-4o models/gemini-1.5-pro

# Discover features for unknown models
litemind discover --api openai --models new-model-name

Architecture Overview

Litemind is a unified API wrapper and agentic AI framework supporting multiple LLM providers (OpenAI, Anthropic, Google Gemini, Ollama).

Core Layers

API Layer (src/litemind/apis/):

  • BaseApi → DefaultApi → Provider implementations (OpenAIApi, AnthropicApi, GeminiApi, OllamaApi)
  • CombinedApi: Chains multiple APIs for fallback
  • Feature-based model selection via ModelFeatures enum and get_best_model(features=[...])
  • Provider-specific code lives in providers/<provider>/ with standard methods: convert_messages, format_tools, process_response

Agent Layer (src/litemind/agent/):

  • Agent: Main orchestrator managing conversation state, tool execution, and augmentations
  • Conversation: Manages system and user messages
  • Message/MessageBlock: Multimodal message containers
  • ToolSet: Collection of tools (FunctionTool, AgentTool, BuiltinTool)
  • AugmentationSet: RAG augmentations including vector databases

Media Layer (src/litemind/media/):

  • MediaBase subclasses: Text, Code, Json, Image, Audio, Video, Table, Document, NdImage
  • MediaConverter: Automatic format conversion pipeline for model compatibility
  • Conversion utilities in media/conversion/

Augmentations (src/litemind/agent/augmentations/):

  • BaseVectorDB → InMemoryVectorDatabase, QdrantVectorDatabase
  • Information: Knowledge units with metadata for RAG

Key Patterns

  • Abstract base classes define interfaces (BaseApi, BaseTool, AugmentationBase, MediaBase)
  • Callback system throughout (ApiCallbackManager, ToolCallbackManager)
  • Feature matrices describe model capabilities for intelligent selection
  • Tests mirror source structure (e.g., apis/tests/, agent/tests/, media/tests/)

Model Registry Architecture

Model feature data is curated in YAML files (src/litemind/apis/model_registry/*.yaml) rather than auto-discovered via live API calls. This design decision was made because:

  • Auto-discovery (the old scanner approach) produced false negatives and brittle tests
  • Curated YAML provides accurate feature data, token limits, aliases, and metadata
  • Each provider has a registry file: registry_openai.yaml, registry_anthropic.yaml, registry_gemini.yaml, registry_ollama.yaml
  • Use ModelRegistry.supports_feature(), get_model_info(), and resolve_alias() for feature lookups
  • The litemind validate CLI can validate registry accuracy against live APIs when needed

Data Flow

User Message → Agent → Augmentation retrieval → Message formatting →
API call → Tool execution (if needed) → Response → Conversation history

Environment Variables

  • OPENAI_API_KEY - OpenAI API
  • ANTHROPIC_API_KEY - Anthropic/Claude API
  • GOOGLE_GEMINI_API_KEY - Google Gemini API
  • Ollama: No key required (local server)

Code Style

  • Use Black for formatting, isort for imports
  • Numpy-style docstrings
  • Type hints throughout
  • Conventional commits: feat:, fix:, docs:, refactor:, test:, chore:

Testing Guidelines

  • Always guard bug fixes with unit tests. When fixing a bug, add a regression test that would have caught the bug. Place tests in the corresponding tests/ directory next to the source (e.g., media/tests/, agent/tests/). Use unittest.mock for unit tests that would otherwise require live API calls.

Dependency Management

Running Tests and Python Commands

IMPORTANT: Always use hatch to run tests, Python commands, and other development tasks.

The system Python environment is externally managed and pip cannot install packages directly. Use hatch for all Python operations:

# Run all tests
hatch test

# Run specific test file
hatch test -- src/litemind/agent/tests/test_agent.py -v

# Run tests matching a pattern
hatch test -- -k "tools" src/

# Run arbitrary Python code
hatch run python -c "from litemind import Agent; print('Works!')"

# Run a Python script
hatch run python myscript.py

# Install additional packages (into hatch environment)
hatch run pip install package-name

# Run linting tools
hatch run pip install flake8 && hatch run flake8 src/

Key Dependency Constraints

Docling compatibility: Docling is an important library for document conversion. It requires pandas>=2.1.4,<3.0.0 and has specific numpy constraints. Do NOT upgrade pandas to 3.x or numpy beyond what docling supports without checking compatibility first.

pytest-md-report: This package doesn't support pytest 9.x yet. It has been removed from the test dependencies until it's updated.

Lazy-loaded Optional Dependencies

Some converters use lazy loading to avoid import-time dependency conflicts:

  • document_converter_docling.py - Docling converter is initialized lazily via _get_docling_converter()

This pattern allows the library to work even when optional dependencies aren't installed, while still supporting them when available.

Documentation Conventions

Sub-package README Files

Each major sub-package should have a README.md file explaining its purpose, structure, and usage. Current sub-packages with READMEs:

  • src/litemind/agent/README.md - Agentic AI framework (Agent, Messages, Tools, Augmentations)
  • src/litemind/apis/README.md - API abstraction layer (providers, model registry, features)
  • src/litemind/media/README.md - Multimodal media types and conversion system
  • src/litemind/utils/README.md - Utility functions (file handling, embeddings, etc.)
  • src/litemind/tools/README.md - CLI commands (export, validate)
  • src/litemind/workflow/README.md - Task workflow system
  • src/litemind/remote/README.md - Remote agent execution via RPyC

README Content Guidelines

Each sub-package README should include:

  1. Package Structure - Directory tree showing key files
  2. Core Components - Classes and functions with brief descriptions
  3. Usage Examples - Code snippets demonstrating common patterns
  4. Docstring Coverage - Summary table of documentation completeness
  5. Dependencies - External libraries required (if applicable)

Main README Generation

When updating the main README.md, use Claude Code directly. The README should follow this structure:

README Sections

  1. Summary - Enthusiastic and complete description of the library, its purpose, and philosophy
  2. Features - List of main features including unique or standout capabilities
  3. Installation - How to install litemind
  4. Basic Usage - Several illustrative examples of the agent-level API (with multimodal inputs):
    • Basic agent usage
    • Agent with tools
    • Agent with tools and augmentation (RAG)
    • Complex example with multimodal inputs, tools and augmentations
  5. Concepts - Main classes, their purpose, and interactions. Explain the difference between the API wrapper layer versus the agentic API
  6. Multi-modality - Multimodal capabilities, ModelFeatures, how to request features, Media classes
  7. More Examples - Advanced use cases:
    • CombinedAPI usage
    • Agent that can execute Python code
    • Wrapper API for structured outputs
    • Agent with tool that generates images
  8. Command Line Tools - CLI tools with example command lines
  9. Caveats and Limitations - Known issues or areas for improvement
  10. Code Health - Unit test results from test_report.md (total tests, passed, failed, assessment)
  11. API Keys - How to obtain and configure API keys for each provider (OpenAI, Claude, Gemini). Which environment variables to set for each OS
  12. Roadmap - Use contents of TODO.md as starting point
  13. Contributing - Refer to CONTRIBUTING.md
  14. License

Code Examples Guidelines

When writing code examples in the README:

  • Use markdown code blocks
  • Keep different examples in separate code blocks with text preambles
  • Examples must be complete and runnable as-is
  • Include all required imports
  • Include expected output as comments where possible
  • Make examples didactic, well-commented, and easy to understand
  • Follow PEP-8 formatting
  • Code examples using specific features must request models that have those features
  • Use real URLs of files that exist (e.g., from the tests)
  • Show that ModelFeatures can be provided as strings instead of enums (see normalisation code)

Important Notes

  • Add badges for: PyPI, license, pepy.tech stats, GitHub stars
  • The Pepy.tech badge image: https://static.pepy.tech/badge/litemind
  • Explain that logging uses Arbol (http://github.com/royerlab/arbol) and can be deactivated with Arbol.passthrough = True
  • Avoid hallucinating - Only document features that exist in the repository
  • Litemind does NOT have a ReActAgent class or anything 'react' related - the Agent class itself has all ReAct framework features
  • If unsure about something, don't make it up