model: support for LlamaBidirectionalModel architecture

[sfallah]

This PR adds support for LlamaBidirectionalModel architectures, resolving feature request #17478.

The implementation enables bidirectional LLaMA embedding models—such as:

• nvidia/llama-embed-nemotron-8b
• nvidia/llama-nemotron-embed-1b-v2

while keeping changes minimal by reusing/copying the existing implementation of LLAMA wherever possible.

---

Key Points

• Support for LlamaBidirectionalModel
• Minimal changes: the implementation largely copies the existing LLaMA architecture already present
• this copies and adapts the existiting model implementation in src/models/llama.cpp to handle bidirectional attention required for embeddings

---

Validation

• Successfully tested with:
  • nvidia/llama-embed-nemotron-8b
  • nvidia/llama-nemotron-embed-1b-v2
• Embedding outputs have the expected shape and value ranges compared to reference implementations

---

GGUF Models

sabafallah/llama-nemotron-embed-1b-v2-GGUF
sabafallah/llama-embed-nemotron-8b-GGUF

Tests

llama-nemotron-embed-1b-v2

curl http://localhost:8080/v1/embeddings \
-H "Content-Type: application/json" \
-H "Authorization: Bearer no-key" \
-d '{
  "model": "llama-nemotron-embed-1b-v2-Q4_K_M.gguf",
  "input": [
    "query: How do neural networks learn patterns from examples?",
    "passage: Deep learning models adjust their weights through backpropagation, using gradient descent to minimize error on training data and improve predictions over time.",
    "passage: Market prices are determined by the relationship between how much people want to buy a product and how much is available for sale, with scarcity driving prices up and abundance driving them down."
  ]
}'

llama-nemotron-embed-1b-v2

curl http://localhost:8080/v1/embeddings \
-H "Content-Type: application/json" \
-H "Authorization: Bearer no-key" \
-d '{
  "model": "llama-embed-nemotron-8b-Q4_K_M.gguf",
  "input": [
    "Instruct: Given a question, retrieve passages that answer the question\nQuery: How do neural networks learn patterns from examples?",
    "Deep learning models adjust their weights through backpropagation, using gradient descent to minimize error on training data and improve predictions over time.",
    "Market prices are determined by the relationship between how much people want to buy a product and how much is available for sale, with scarcity driving prices up and abundance driving them down."
  ]
}'

[CISC]

Instead of duplicating the model code, perhaps just template it as for non/swa models, just with/out cache and output.

[CISC]

Have you tested if adding LlamaBidirectionalForSequenceClassification too just works?

[sfallah]

I have not tested it, but I have had a look at nvidia/llama-nemotron-rerank-1b-v2.
I think the classifier tensor (score.weight) need to be added for it to work.