[XPU] Update doc and add scripts for downloading dependencies#2845
Merged
Conversation
|
Thanks for your contribution! |
| @@ -0,0 +1,54 @@ | |||
| #!/bin/bash | |||
Collaborator
There was a problem hiding this comment.
脚本名改成 download_dependencies.sh 吧
| fi | ||
|
|
||
| echo "Installation completed in: $THIRDPARTY_DIR" | ||
| echo "You can set environment variables to use XVLLM and XTDK in the following way:" |
Collaborator
There was a problem hiding this comment.
You can set environment variables as follows to use XVLLM and XTDK:
| wget https://klx-sdk-release-public.su.bcebos.com/xre/kl3-release/5.0.21.21/xre-Linux-x86_64-5.0.21.21.tar.gz && \ | ||
| tar -zxf xre-Linux-x86_64-5.0.21.21.tar.gz && mv xre-Linux-x86_64-5.0.21.21 xre | ||
| tar -zxf xre-Linux-x86_64-5.0.21.21.tar.gz && mv xre-Linux-x86_64-5.0.21.21 xre && \ | ||
| cd /workspace/FastDeploy && bash custom_ops/xpu_ops/src/download_dependency.sh stable |
Collaborator
There was a problem hiding this comment.
脚本名改成 download_dependencies.sh 吧
|
|
||
| For detailed OpenAI protocol specifications, see [OpenAI Chat Compeltion API](https://platform.openai.com/docs/api-reference/chat/create). Differences from the standard OpenAI protocol are documented in [OpenAI Protocol-Compatible API Server](../../online_serving/README.md). | ||
|
|
||
| ## Supported Models |
Collaborator
There was a problem hiding this comment.
这个能放在 ## Quick start 前面吗？
然后在Quick start 可以删掉这一大段『The P800 supports the deployment of the ERNIE-4.5-300B-A47B-Paddle model using the following configurations (Note: Different configurations may result in variations in performance).
- 32K WINT4 with 8 XPUs (Recommended)
- 128K WINT4 with 8 XPUs
- 32K WINT4 with 4 XPUs』
同时，『#### Start service 』只保留一个推荐的启动方法吧。
| ```bash | ||
| XTDK: https://klx-sdk-release-public.su.bcebos.com/xtdk_15fusion/dev/latest/xtdk-llvm15-ubuntu2004_x86_64.tar.gz | ||
| XVLLM: https://klx-sdk-release-public.su.bcebos.com/xinfer/daily/eb/latest/output.tar.gz | ||
| bash custom_ops/xpu_ops/src/download_dependency.sh develop |
|
|
||
| OpenAI 协议的更多说明可参考文档 [OpenAI Chat Compeltion API](https://platform.openai.com/docs/api-reference/chat/create)，以及与 OpenAI 协议的区别可以参考 [兼容 OpenAI 协议的服务化部署](../../online_serving/README.md)。 | ||
|
|
||
| ## 支持的模型 |
hong19860320
changed the title
xiaoguoguo626807
pushed a commit
to xiaoguoguo626807/FastDeploy
that referenced
this pull request
May 7, 2026
…Paddle#2845) * [XPU] update xvllm download * update supported models * fix xpu model runner in huge memory with small model * update doc
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.