Feature Request: allow the web UI to be used with other backends

[jerkstorecaller]

Prerequisites

• I am running the latest code. Mention the version if possible as well.
• I carefully followed the README.md.
• I searched using keywords relevant to my issue to make sure that I am creating a new issue that is not already open (or closed).
• I reviewed the Discussions, and have a new and useful enhancement to share.

Feature Description

I wrote about this in #16938 , and a llama.cpp collaborator asked me to create a Feature Request issue for it, so here I am.

I think it would be nice if the llama-server web UI was available to use with any backend, not just llama-server.
You can either separate it into its own app, or simply keep it in llama-server (with no model loaded, it uses very little RAM according). Either way, allow the user to configure it to point at a different Chat Completions backend.

Motivation

I think it would be a gain to the ecosystem if we could just fire up the llama-server docker image that uses a tiny amount of RAM and have a working general-purposeweb UI.

llama-server hits a certain sweetspot that other apps don't.

1. It's fast, elegant and tiny
2. Has the features most casual users need, including chat history, model switching, system prompt, multi-modal
3. Shows useful speed stats which is useful in quick experiments

I use other chat apps. The polished ones are all big bloated multi-user enterprise ChatGPT replacements, with one Docker container for the DB, one for RAG, memory, often complicated configuration...or they are really basic unpolished vibecoded things that barely work.

Possible Implementation

My suggestion: keep everything as is, but...

1. On the Settings page, add a Server section
2. By default, it shows 2 radio buttons, [o] Use this llama-server, and [] Remote Server. Switching to Remote Server enables further configuration as seen below:

Server endpoint: ______
API key (optional): ______
Ignore SSL Errors: [x]
Test Connection: button that tries to hit http://serverurl/v1/models

3. Now the webUI is using this remote server, unless the user returns to Settings -> Server and changes from Remote Server to Use This Llama-server.
4. The model list + model ID to use is same as before, via the dropdown in the main UI.

[martinengelke]

Great description, you nailed down my concerns about most platforms out there. May I add a suggestion:

Imagine you are in a conversation and you feel like your local model isn't capable of the task. At this point you may want to follow up with the remote server. Since there is already a conversation branching feature we may continue with a setup remote server? In my opinion this is the sweet spot of efficiently handling tasks locally and remotely.

But that's maybe a total new feature (branch and follow up with different models). Maybe I should open an extra Feature request, what do you say?

[jerkstorecaller]

@martinengelke That's already the current behavior in llama-server's web UI. You can change model at any time and the new model will be used on your next turn, continuing from where the previous model left off. Fork or no fork.

Whoever implements this won't be clearing your conversation just because you changed servers. The conversations are local on your own PC.

[martinengelke]

Phenomenal!

[allozaur]

Hey, I'd love to implement that shortly after introducing the models management UI/UX

[evil617]

Thanks for this feature request. I recently switched from llama.cpp to vLLM because of the performance improvements, and one thing I really miss is the UI from llama.

[allozaur]

Thanks for this feature request. I recently switched from llama.cpp to vLLM because of the performance improvements, and one thing I really miss is the UI from llama.

Soon, soon 😉

[niutech]

I've made the PR #24383 providing support for calling remote API endpoints configured at build time by optionally setting the env variables. You can build the standalone web ui using npm run build and upload it to any static web hosting.

[allozaur]

I've made the PR #24383 providing support for calling remote API endpoints configured at build time by optionally setting the env variables. You can build the standalone web ui using npm run build and upload it to any static web hosting.

Thanks! I will take a look at it :) i still had some certain idea for how we want to handle this UX-wise, so let's see!

[niutech]

@allozaur I've updated my PR #24383 to dynamically set both API Base and API Key in the Web UI Settings.