Huggingface Text Generation Model example error: cannot import name 'Conversation' from 'transformers.pipelines'

[stevexiao2012]

Hi

When I tried the Huggingface Text Generation Model , the model can not be loaded properly.

I am using the seldon core 2.10.2 and the image of mlserver is seldonio/mlserver:1.7.1 。

Below is the error log printed by the predicator server's container:

ImportError: cannot import name 'Conversation' from 'transformers.pipelines' (/opt/conda/lib/python3.10/site-packages/transformers/pipelines/init.py)
2026-02-10 06:48:23,200 [mlserver] WARNING - Model name 'transformer' is different than model's folder name 'distilgpt2'.
2026-02-10 06:48:23,202 [mlserver.grpc] ERROR - cannot import name 'Conversation' from 'transformers.pipelines' (/opt/conda/lib/python3.10/site-packages/transformers/pipelines/init.py)
Traceback (most recent call last):
File "/opt/conda/lib/python3.10/site-packages/mlserver/grpc/utils.py", line 44, in _inner
return await f(self, request, context)
File "/opt/conda/lib/python3.10/site-packages/mlserver/grpc/servicers.py", line 145, in RepositoryModelLoad
await self._model_repository_handlers.load(request.model_name)
File "/opt/conda/lib/python3.10/site-packages/mlserver/handlers/model_repository.py", line 67, in load
model = await self._model_registry.load(model_settings)
File "/opt/conda/lib/python3.10/site-packages/mlserver/registry.py", line 299, in load
return await self._models[model_settings.name].load(model_settings)
File "/opt/conda/lib/python3.10/site-packages/mlserver/registry.py", line 144, in load
new_model = self._model_initialiser(model_settings)
File "/opt/conda/lib/python3.10/site-packages/mlserver/parallel/registry.py", line 235, in model_initialiser
return model_initialiser(model_settings)
File "/opt/conda/lib/python3.10/site-packages/mlserver/registry.py", line 52, in model_initialiser
model_class = model_settings.implementation
File "/opt/conda/lib/python3.10/site-packages/mlserver/settings.py", line 403, in implementation
reload_module(self.implementation)
File "/opt/conda/lib/python3.10/site-packages/mlserver/settings.py", line 65, in _reload_module
module = importlib.import_module(module_path)
File "/opt/conda/lib/python3.10/importlib/init.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 1050, in _gcd_import
File "", line 1027, in _find_and_load
File "", line 1006, in _find_and_load_unlocked
File "", line 688, in _load_unlocked
File "", line 883, in exec_module
File "", line 241, in _call_with_frames_removed
File "/opt/conda/lib/python3.10/site-packages/mlserver_huggingface/init.py", line 1, in
from .runtime import HuggingFaceRuntime
File "/opt/conda/lib/python3.10/site-packages/mlserver_huggingface/runtime.py", line 14, in
from .codecs import HuggingfaceRequestCodec
File "/opt/conda/lib/python3.10/site-packages/mlserver_huggingface/codecs/init.py", line 1, in
from .base import MultiInputRequestCodec, HuggingfaceRequestCodec
File "/opt/conda/lib/python3.10/site-packages/mlserver_huggingface/codecs/base.py", line 25, in
from .json import HuggingfaceSingleJSONCodec
File "/opt/conda/lib/python3.10/site-packages/mlserver_huggingface/codecs/json.py", line 4, in
from .utils import json_decode, json_encode
File "/opt/conda/lib/python3.10/site-packages/mlserver_huggingface/codecs/utils.py", line 7, in
from transformers.pipelines import Conversation