toggle to remove buffering from token streaming

[Rose22]

id like the option to have direct token-by-token streaming instead of kobold's current buffering method, it makes things feel much faster and more responsive

my idea is to have it as a setting you can toggle, between kobold's buffering method and the more direct method (which is what llamacpp does)

also you told me in voice call to tell you to investigate if theres a way to set how many tokens to buffer per chunk

[esolithe]

I'll take a look at this eventually, but letting copilot have a go at it to see what it does.

[esolithe]

@Rose22 I investigated this. As far as I can see, there's not a blocker on the tokens unless antislop is in use. Tokens are set at under a second delay with the default config (unless streaming cannot be used at all).

As such, I don't see how this could be implemented given it appears streaming is done per token when possible. If smoother buffering is needed, then it can be done with a client side queue.