Skip to content

feat(llm): support asynchronous streaming generation in rag block by using async_generator and asyncio.wait. - #190

Merged
imbajin merged 22 commits into
apache:mainfrom
vichayturen:streaming-output
Mar 6, 2025
Merged

imbajin merged 22 commits into
apache:mainfrom
vichayturen:streaming-output

Conversation

@vichayturen

@vichayturen vichayturen commented Mar 3, 2025 •

Copy link
Copy Markdown
Contributor

follow #172

In order to achieve asynchronization, we compromised by changing gremlin_generate_operator to a synchronous generation mode. This can be changed back to an asynchronous mode after achieving full asynchronization in the subsequent agentization process.

@dosubot dosubot Bot added the size:L This PR changes 100-499 lines, ignoring generated files. label Mar 3, 2025
@github-actions github-actions Bot added the llm label Mar 3, 2025
@dosubot dosubot Bot added the enhancement New feature or request label Mar 3, 2025
@dosubot dosubot Bot added size:XL This PR changes 500-999 lines, ignoring generated files. and removed size:L This PR changes 100-499 lines, ignoring generated files. labels Mar 6, 2025
@dosubot dosubot Bot added the lgtm This PR has been approved by a maintainer label Mar 6, 2025
@imbajin
imbajin merged commit 7ae5d6f into apache:main Mar 6, 2025
haohao0103 pushed a commit to haohao0103/incubator-hugegraph-ai that referenced this pull request Jun 30, 2026
…#190)

follow apache#172 

In order to achieve asynchronization, we compromised by changing `gremlin_generate_operator` to a synchronous generation mode. This can be changed back to an asynchronous mode after achieving full asynchronization in the subsequent agentization process.

---------

Co-authored-by: chenzihong <522023320011@smail.nju.edu.cn>
Co-authored-by: chenzihong <58508660+ChenZiHong-Gavin@users.noreply.github.com>
Co-authored-by: imbajin <jin@apache.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request lgtm This PR has been approved by a maintainer llm size:XL This PR changes 500-999 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants