Kokoro failed with ImportError: cannot import name 'KPipeline' from 'kokoro' (/backends/cuda12-kokoro/kokoro.py) #6115
Description
LocalAI version:
Docker image: localai/localai:latest-gpu-nvidia-cuda12
Environment, CPU architecture, OS, and Version:
Linux xxxxxxx 6.11.0-25-generic #25-Ubuntu SMP PREEMPT_DYNAMIC Fri Apr 11 23:29:18 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
RTX 5090
Describe the bug
Kokoro TTS failed with error "failed to load model with internal loader: grpc service not ready"
logs show error: Kokoro failed with ImportError: cannot import name 'KPipeline' from 'kokoro' (/backends/cuda12-kokoro/kokoro.py)
To Reproduce
Install Kokoro-cuda12 backend + kokoro model.
Expected behavior
Should work
Logs
12:24PM DBG context local model name not found, setting to the first model first model name=voice-fr-gilles-low
12:24PM DBG config does not include model, using input input.ModelName=kokoro
12:24PM DBG LocalAI TTS Request received model=kokoro
12:24PM INF BackendLoader starting backend=kokoro modelID=kokoro o.model=kokoro
12:24PM DBG Loading model in memory from file: /models/kokoro
12:24PM DBG Loading Model kokoro with gRPC (file: /models/kokoro) (backend: kokoro): {backendString:kokoro model:kokoro modelID:kokoro context:{emptyCtx:{}} gRPCOptions:0xc000402dc8 externalBackends:map[] grpcAttempts:20 grpcAttemptsDelay:2 parallelRequests:false}
12:24PM DBG Loading external backend: /backends/cuda12-kokoro/run.sh
12:24PM DBG external backend is file: &{name:run.sh size:191 mode:448 modTime:{wall:0 ext:63890580792 loc:0x46812a0} sys:{Dev:66306 Ino:24660250 Nlink:1 Mode:33216 Uid:0 Gid:0 X__pad0:0 Rdev:0 Size:191 Blksize:4096 Blocks:8 Atim:{Sec:1755692478 Nsec:160889424} Mtim:{Sec:1754983992 Nsec:0} Ctim:{Sec:1755692478 Nsec:133889495} X__unused:[0 0 0]}}
12:24PM DBG Loading GRPC Process: /backends/cuda12-kokoro/run.sh
12:24PM DBG GRPC Service for kokoro will be running at: '127.0.0.1:41391'
12:24PM DBG GRPC Service state dir: /tmp/go-processmanager738905022
12:24PM DBG GRPC Service Started
12:24PM DBG Wait for the service to start up
12:24PM DBG Options: ContextSize:1024 Seed:511307814 NBatch:512 MMap:true NGPULayers:9999999 Threads:8 Options:"lang_code:a"
12:24PM DBG GRPC(kokoro-127.0.0.1:41391): stdout Initializing libbackend for cuda12-kokoro
12:24PM DBG GRPC(kokoro-127.0.0.1:41391): stdout virtualenv activated
12:24PM DBG GRPC(kokoro-127.0.0.1:41391): stdout activated virtualenv has been ensured
12:24PM DBG GRPC(kokoro-127.0.0.1:41391): stderr /backends/cuda12-kokoro/venv/lib/python3.10/site-packages/google/protobuf/runtime_version.py:98: UserWarning: Protobuf gencode version 5.29.0 is exactly one major version older than the runtime version 6.31.1 at backend.proto. Please update the gencode to avoid compatibility violations in the next runtime release.
12:24PM DBG GRPC(kokoro-127.0.0.1:41391): stderr warnings.warn(
12:24PM DBG GRPC(kokoro-127.0.0.1:41391): stderr Traceback (most recent call last):
12:24PM DBG GRPC(kokoro-127.0.0.1:41391): stderr File "/backends/cuda12-kokoro/backend.py", line 15, in
12:24PM DBG GRPC(kokoro-127.0.0.1:41391): stderr from kokoro import KPipeline
12:24PM DBG GRPC(kokoro-127.0.0.1:41391): stderr ImportError: cannot import name 'KPipeline' from 'kokoro' (/backends/cuda12-kokoro/kokoro.py)
12:24PM INF Success ip=127.0.0.1 latency="27.63µs" method=GET status=200 url=/readyz
12:24PM ERR failed starting/connecting to the gRPC service error="rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing: dial tcp 127.0.0.1:41391: connect: connection refused""
12:24PM DBG GRPC Service NOT ready
12:24PM ERR Server error error="failed to load model with internal loader: grpc service not ready" ip=192.168.2.33 latency=40.024446751s method=POST status=500 url=/tts
12:25PM INF Success ip=127.0.0.1 latency="27.21µs" method=GET status=200 url=/readyz
Additional context