Skip to content

Add LocalAI integration - #3273

Open
lbarrosby wants to merge 1 commit into
DataDog:masterfrom
lbarrosby:add-localai-integration
Open

lbarrosby wants to merge 1 commit into
DataDog:masterfrom
lbarrosby:add-localai-integration

Conversation

@lbarrosby

Copy link
Copy Markdown

What does this PR do?

Adds a new Agent-based integration for LocalAI, the open source OpenAI-compatible server for running LLM, embedding, image and audio models on your own hardware.

The check scrapes the Prometheus endpoint LocalAI serves at /metrics and maps the metrics to localai.*:

  • API request latency by route and HTTP method (api_call histogram, in seconds).
  • Prompt and completion tokens by served model and user, billed requests by model and route, and requests that finished without a usage record.
  • Model failover chains: health of each target and switches between targets.
  • Chat context compression: attempts by result, original-to-compressed token ratio, and duration.

The endpoint label of the usage metrics is renamed to api_endpoint, since endpoint is the tag the Agent sets for the scraped URL, and the otel_scope_* labels added by the OpenTelemetry exporter are dropped.

Motivation

LocalAI is a popular self-hosted alternative to the OpenAI API, and there is no integration for it today. The vLLM, SGLang and TGI integrations cover single inference engines; LocalAI sits in front of several backends, so token usage and failover are the useful signals here.

I'm happy to be listed as the maintainer of this integration.

Review checklist

  • PR has a meaningful title
  • Feature or bugfix has tests
  • Git history is clean
  • If PR impacts documentation, docs team has been notified or an issue has been opened on the documentation repo
  • If this PR includes a log pipeline, please add a description describing the remappers and processors.

Additional Notes

  • The fixture in tests/fixtures/metrics.txt is a real scrape from LocalAI v4.12.1 (darwin-arm64 release, llama-cpp backend) serving a small GGUF model, with a failover chain whose first target is broken and a model with context compression enabled, so every mapped metric shows up.
  • When the client accepts gzip, LocalAI compresses the /metrics response twice (the HTTP compression middleware wraps the already compressed promhttp output), which breaks parsing. The check sends Accept-Encoding: identity by default; there is a unit test for it.
  • localai_cost_usd_total is not mapped: it is only emitted when a pricing version is attached to the usage record, and nothing sets one yet. The agent, PII and motion counters are left out for now as well, since they belong to separate features.
  • Tested locally with ddev test localai (unit), ddev test --lint, and ddev validate (config, models, metadata, service-checks, readmes, openmetrics, typos, package). I also ran the check against the live server and all 15 metrics plus the health service check came through.

Collect the Prometheus metrics exposed by LocalAI: API request latency, token usage and billed requests by model, requests without a usage record, model failover target health and switches, and chat context compression.

Signed-off-by: Lucas Barros <lucasbarros.kam@gmail.com>
@lbarrosby
lbarrosby requested review from a team as code owners November 6, 2026 02:55
@git-thuerk-done git-thuerk-done added the editorial review Waiting on a more in-depth review from a docs team editor label Nov 6, 2026
@git-thuerk-done

git-thuerk-done commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

Opened DOCS-16335 to assign a Docs writer and follow up with editorial review.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

editorial review Waiting on a more in-depth review from a docs team editor

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants