Skip to content

Add Hugging Face TEI integration - #3270

Open
lbarrosby wants to merge 1 commit into
DataDog:masterfrom
lbarrosby:add-hugging-face-tei-integration
Open

lbarrosby wants to merge 1 commit into
DataDog:masterfrom
lbarrosby:add-hugging-face-tei-integration

Conversation

@lbarrosby

Copy link
Copy Markdown

What does this PR do?

Adds a new Agent-based integration for Hugging Face Text Embeddings Inference (TEI).

The check scrapes the Prometheus endpoint TEI serves at /metrics and maps the te_* metrics to hugging_face_tei.*: request counts, successes and failures (the err label is renamed to error_type, as in the TGI integration), end-to-end, tokenization, queue and inference latency histograms, queue size, batching, and the per-input embed (embedding models) and predict (re-ranker and classification models) families.

Motivation

TEI is the embedding and re-ranking backend in many RAG deployments, often running next to TGI or vLLM. The hugging_face_tgi integration in integrations-core covers text generation, but there is no integration for TEI. Metric names follow the same layout as hugging_face_tgi so both can be used side by side.

I'm happy to be listed as the maintainer of this integration.

Review checklist

  • PR has a meaningful title
  • Feature or bugfix has tests
  • Git history is clean
  • If PR impacts documentation, docs team has been notified or an issue has been opened on the documentation repo
  • If this PR includes a log pipeline, please add a description describing the remappers and processors.

Additional Notes

  • Both fixtures are real scrapes from TEI 1.9.4: metrics.txt from an embedding model (sentence-transformers/all-MiniLM-L6-v2) and metrics_rerank.txt from a re-ranker (cross-encoder/ms-marco-MiniLM-L-6-v2). The embed and predict families are exclusive to each model type, so the metadata symmetry test runs the check against both.
  • Tested locally with ddev test hugging_face_tei (unit), ddev test --lint, and ddev validate (config, models, metadata, service-checks, readmes, openmetrics, typos). I also ran the check against a live TEI server.
  • /metrics is a public route in TEI, so it does not need the API key.

Collect the Prometheus metrics exposed by Hugging Face Text Embeddings Inference: request counts and failures, end-to-end, tokenization, queue and inference latency, queue depth, batching, and the per-input embed and predict metrics.

Signed-off-by: Lucas Barros <lucasbarros.kam@gmail.com>
@lbarrosby
lbarrosby requested review from a team as code owners November 5, 2026 16:37
@joepeeples joepeeples self-assigned this Nov 5, 2026
@joepeeples joepeeples removed their assignment Nov 5, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants