Add the endpoints /api/tags and /api/chat#13659
Add the endpoints /api/tags and /api/chat#13659ngxson merged 3 commits intoggml-org:masterfrom R-Dson:master
Conversation
Add the endpoints /api/tags and /api/chat, and improved the model metadata response
tools/server/server.cpp
Outdated
| } | ||
| }, | ||
| {"modelfile", ""}, // Specific to ollama and does not seem to be needed | ||
| {"parameters", ""}, // TODO: add parameters |
There was a problem hiding this comment.
Does the API client expect this to be a string or an object?
There was a problem hiding this comment.
modelfile is a string. Parameters i am not sure, but it seems to be enough as is.
|
I think this is the minimal code that we need for copilot to work with it. |
| res.status = 503; | ||
| } else if (req.path == "/models" || req.path == "/v1/models") { | ||
| } else if (req.path == "/models" || req.path == "/v1/models" || req.path == "/api/tags") { | ||
| // allow the models endpoint to be accessed during loading |
There was a problem hiding this comment.
during loading, common_chat_templates_source call will fail
There was a problem hiding this comment.
Do you want me to remove that endpoint from the if-case?
Edit:
The current code does not use common_chat_templates_source in that endpoint after the latest changes.
There was a problem hiding this comment.
hmm I think it's ok to keep this then
* Add the endpoints /api/tags and /api/chat Add the endpoints /api/tags and /api/chat, and improved the model metadata response * Remove trailing whitespaces * Removed code that is not needed for copilot to work.
|
Just a gotcha if you are using llama-swap: it does not support or map /api/tags so use the actual endpoint/port for the specific model started by llama.cpp |
4 participants
Add the endpoints /api/tags and /api/chat, and improved the model metadata response.
These changes made llama-server work with Copilot Chat again for me. both /api/tags and /api/chat are added because of a discussion on open-webui's page where they point out that both need to be handled. The old json values are kept as is to keep backwards compatibility, in case anyone uses them.
This issue was also pointed out in the comments of #12896.