Skip to content

Improve multi-turn capability for agent#1248

Merged
minmin-intel merged 14 commits intoopea-project:mainfrom
minmin-intel:multi-turn-v1.3
Feb 14, 2025
Merged

Improve multi-turn capability for agent#1248
minmin-intel merged 14 commits intoopea-project:mainfrom
minmin-intel:multi-turn-v1.3

Conversation

@minmin-intel
Copy link
Collaborator

Description

  1. Removed "with_store" and replaced with "memory_type". Two memory types: volatile, persistent. Volatile uses MemorySaver checkpointer, only knows thread_id, no assistant_id. Persistent uses RedisPersistence, knows assistant_id and thread_id. chat_completions APIs only supports volatile, assistants APIs support both types.
  2. Improved react_llama prompt templates and code for multi-turn conversations.
  3. Changed react_llama code to output thought process in AIMessage for UI to show to users.

Issues

#1184
#1183

Type of change

List the type of change like below. Please delete options that are not relevant.

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds new functionality)
  • Breaking change (fix or feature that would break existing design and interface)
  • Others (enhancement, documentation, validation, etc.)

Dependencies

No

Tests

Unit tests

Signed-off-by: minmin-intel <minmin.hou@intel.com>
Signed-off-by: minmin-intel <minmin.hou@intel.com>
Signed-off-by: minmin-intel <minmin.hou@intel.com>
Signed-off-by: minmin-intel <minmin.hou@intel.com>
Signed-off-by: minmin-intel <minmin.hou@intel.com>
Signed-off-by: minmin-intel <minmin.hou@intel.com>
Signed-off-by: minmin-intel <minmin.hou@intel.com>
@minmin-intel minmin-intel requested review from WenjiaoYue and removed request for chensuyue January 31, 2025 19:29
Signed-off-by: minmin-intel <minmin.hou@intel.com>
Signed-off-by: minmin-intel <minmin.hou@intel.com>
Signed-off-by: minmin-intel <minmin.hou@intel.com>
@lkk12014402 lkk12014402 added this to the v1.3 milestone Feb 14, 2025
@minmin-intel minmin-intel merged commit 0e3f8ab into opea-project:main Feb 14, 2025
16 checks passed
aMahanna added a commit to arangoml/GenAIComps1.2 that referenced this pull request Feb 26, 2025
commit ad8f517
Author: Dina Suehiro Jones <dina.s.jones@intel.com>
Date:   Wed Feb 26 11:35:04 2025 -0800

    Dataprep Multimodal Redis README fixes (opea-project#1330)

    Signed-off-by: Dina Suehiro Jones <dina.s.jones@intel.com>

commit c70f868
Author: Ervin Castelino <89144265+Ervin0307@users.noreply.github.com>
Date:   Tue Feb 25 07:42:42 2025 +0000

    Update README.md (opea-project#1253)

    Signed-off-by: Ervin <ervincas13@gmail.com>

commit 589587a
Author: ZePan110 <ze.pan@intel.com>
Date:   Mon Feb 24 17:54:51 2025 +0800

    Fix docker image security issues (opea-project#1321)

    Signed-off-by: ZePan110 <ze.pan@intel.com>

commit f5699e4
Author: Brijesh Thummar <brijeshthummar02@gmail.com>
Date:   Mon Feb 24 12:00:19 2025 +0530

    [Doc] vLLM - typo in README.md (opea-project#1302)

    Fix Typo in README

    Signed-off-by: brijeshthummar02@gmail.com <brijeshthummar02@gmail.com>

commit 364ccad
Author: Jonathan Minkin <minkinj@amazon.com>
Date:   Sun Feb 23 19:27:31 2025 -0800

    Add support for string message in Bedrock textgen (opea-project#1291)

    * Add support for string message in bedrock, update README
    * Add test for string message in test script

    Signed-off-by: Jonathan Minkin <minkinj@amazon.com>

commit 625aec9
Author: Daniel De León <111013930+daniel-de-leon-user293@users.noreply.github.com>
Date:   Fri Feb 21 13:20:58 2025 -0800

    Add native support for toxicity detection guardrail microservice (opea-project#1258)

    * add opea native support for toxic-prompt-roberta

    * add test script back

    * [pre-commit.ci] auto fixes from pre-commit.com hooks

    for more information, see https://pre-commit.ci

    * add comp name env variable

    * set default port to 9090

    Signed-off-by: Daniel Deleon <daniel.de.leon@intel.com>

    * add service to compose

    Signed-off-by: Daniel Deleon <daniel.de.leon@intel.com>

    * removed debug print

    Signed-off-by: Daniel Deleon <daniel.de.leon@intel.com>

    * remove triton version because habana updated

    Signed-off-by: Daniel Deleon <daniel.de.leon@intel.com>

    * add locust results

    Signed-off-by: Daniel Deleon <daniel.de.leon@intel.com>

    * [pre-commit.ci] auto fixes from pre-commit.com hooks

    for more information, see https://pre-commit.ci

    * skip warmup for halluc test

    Signed-off-by: Daniel Deleon <daniel.de.leon@intel.com>

    ---------

    Signed-off-by: Daniel Deleon <daniel.de.leon@intel.com>
    Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
    Co-authored-by: Liang Lv <liang1.lv@intel.com>
    Co-authored-by: Abolfazl Shahbazi <12436063+ashahba@users.noreply.github.com>

commit 4352636
Author: ZePan110 <ze.pan@intel.com>
Date:   Fri Feb 21 17:14:08 2025 +0800

    Fix trivy issue in Dockerfile (opea-project#1304)

    Signed-off-by: ZePan110 <ze.pan@intel.com>

commit 135ef91
Author: rbrugaro <rita.brugarolas.brufau@intel.com>
Date:   Thu Feb 20 15:29:39 2025 -0800

    Change neo4j Bolt default PORT from 7687 to $NEO4J_PORT2 (opea-project#1292)

    * Change neo4j Bolt default PORT from 7687 to

    -configured the port in neo4j compose.yaml to use variable value
    -made all corresponding changes in neo4j  dataprep and retriever components and test scripts to use env variable instead of default port value.

    Signed-off-by: rbrugaro <rita.brugarolas.brufau@intel.com>

    * [pre-commit.ci] auto fixes from pre-commit.com hooks

    for more information, see https://pre-commit.ci

    * missing positional arg in milvus dataprep

    Signed-off-by: rbrugaro <rita.brugarolas.brufau@intel.com>

    * remove redundance in stop_docker

    Signed-off-by: rbrugaro <rita.brugarolas.brufau@intel.com>

    * resolve retriever to neo4j connectivity issue bad URL

    Signed-off-by: rbrugaro <rita.brugarolas.brufau@intel.com>

    * [pre-commit.ci] auto fixes from pre-commit.com hooks

    for more information, see https://pre-commit.ci

    * set neo4j ports to neo4j defaults and fix environment variables in READMEs

    Signed-off-by: rbrugaro <rita.brugarolas.brufau@intel.com>

    ---------

    Signed-off-by: rbrugaro <rita.brugarolas.brufau@intel.com>
    Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
    Co-authored-by: Liang Lv <liang1.lv@intel.com>

commit a4f6af1
Author: Letong Han <106566639+letonghan@users.noreply.github.com>
Date:   Thu Feb 20 13:38:01 2025 +0800

    Refine dataprep test scripts (opea-project#1305)

    * Refine dataprep Milvus CI
    Signed-off-by: letonghan <letong.han@intel.com>

commit 2102a8e
Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Date:   Thu Feb 20 10:46:19 2025 +0800

    Bump transformers (opea-project#1278)

    Signed-off-by: dependabot[bot] <support@github.com>
    Co-authored-by: Liang Lv <liang1.lv@intel.com>

commit a033c05
Author: Liang Lv <liang1.lv@intel.com>
Date:   Wed Feb 19 14:19:02 2025 +0800

    Fix milvus dataprep ingest files failure (opea-project#1299)

    Signed-off-by: lvliang-intel <liang1.lv@intel.com>
    Co-authored-by: Letong Han <106566639+letonghan@users.noreply.github.com>

commit 022d052
Author: lkk <33276950+lkk12014402@users.noreply.github.com>
Date:   Wed Feb 19 09:50:59 2025 +0800

    fix agent message format. (opea-project#1297)

    1. set default session_id for react_langchain strategy, because the langchain version upgrade.
    2. fix request message format

commit 7727235
Author: Liang Lv <liang1.lv@intel.com>
Date:   Tue Feb 18 20:55:20 2025 +0800

    Refine CLIP embedding microservice by leveraging the third-party CLIP (opea-project#1298)

    * Refine CLI embedding microservice using dependency
    Signed-off-by: lvliang-intel <liang1.lv@intel.com>

commit a353f99
Author: Spycsh <39623753+Spycsh@users.noreply.github.com>
Date:   Mon Feb 17 11:35:38 2025 +0800

    Fix telemetry connection issue when disabling telemetry (opea-project#1290)

    * Fix telemetry connection issue when disabling telemetry

    - use ENABLE_OPEA_TELEMETRY to control whether to enable open telemetry, default false
    - fix the issue that logs always show telemetry connection error with each request when telemetry is disabled
    - ban the above error propagation to microservices when telemetry is disabled

    Signed-off-by: Spycsh <sihan.chen@intel.com>

    * [pre-commit.ci] auto fixes from pre-commit.com hooks

    for more information, see https://pre-commit.ci

    * Fix ut failure where required the flag to be on

    * [pre-commit.ci] auto fixes from pre-commit.com hooks

    for more information, see https://pre-commit.ci

    ---------

    Signed-off-by: Spycsh <sihan.chen@intel.com>
    Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

commit 7c2e7f6
Author: xiguiw <111278656+xiguiw@users.noreply.github.com>
Date:   Sat Feb 15 11:25:15 2025 +0800

    update vLLM CPU to latest tag (opea-project#1285)

    Get the latest vLLM stable version.
    Signed-off-by: Wang, Xigui <xigui.wang@intel.com>

commit c3c8497
Author: Letong Han <106566639+letonghan@users.noreply.github.com>
Date:   Fri Feb 14 22:29:38 2025 +0800

    Fix Qdrant retriever RAG issue. (opea-project#1289)

    * Fix Qdrant retriever no retrieved result issue.
    Signed-off-by: letonghan <letong.han@intel.com>

commit 47f68a4
Author: Letong Han <106566639+letonghan@users.noreply.github.com>
Date:   Fri Feb 14 20:29:27 2025 +0800

    Fix the retriever issue of Milvus (opea-project#1286)

    * Fix the retriever issue of Milvus DB that data can not be retrieved
    after ingested using dataprep.

    Signed-off-by: letonghan <letong.han@intel.com>

    ---------

    Signed-off-by: letonghan <letong.han@intel.com>
    Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

commit 0e3f8ab
Author: minmin-intel <minmin.hou@intel.com>
Date:   Thu Feb 13 20:24:02 2025 -0800

    Improve multi-turn capability for agent (opea-project#1248)

    * first code for multi-turn

    Signed-off-by: minmin-intel <minmin.hou@intel.com>

    * test redispersistence

    Signed-off-by: minmin-intel <minmin.hou@intel.com>

    * integrate persistent store in react llama

    Signed-off-by: minmin-intel <minmin.hou@intel.com>

    * test multi-turn

    Signed-off-by: minmin-intel <minmin.hou@intel.com>

    * multiturn for assistants api and chatcompletion api

    Signed-off-by: minmin-intel <minmin.hou@intel.com>

    * update readme and ut script

    Signed-off-by: minmin-intel <minmin.hou@intel.com>

    * update readme and ut scripts

    Signed-off-by: minmin-intel <minmin.hou@intel.com>

    * [pre-commit.ci] auto fixes from pre-commit.com hooks

    for more information, see https://pre-commit.ci

    * fix bug

    Signed-off-by: minmin-intel <minmin.hou@intel.com>

    * change memory type naming

    Signed-off-by: minmin-intel <minmin.hou@intel.com>

    * fix with_memory as str

    Signed-off-by: minmin-intel <minmin.hou@intel.com>

    ---------

    Signed-off-by: minmin-intel <minmin.hou@intel.com>
    Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

commit 4a90692
Author: rbrugaro <rita.brugarolas.brufau@intel.com>
Date:   Thu Feb 13 18:12:25 2025 -0800

    Bug Fix neo4j dataprep ingest error handling and skip_ingestion argument passing (opea-project#1288)

    * Fix dataprpe ingest error handling and skip_ingestion argument passing in dataprep neo4j integration
    Signed-off-by: rbrugaro <rita.brugarolas.brufau@intel.com>

commit d1dfd0e
Author: Spycsh <39623753+Spycsh@users.noreply.github.com>
Date:   Thu Feb 13 22:39:47 2025 +0800

    Align mongo related chathistory/feedbackmanagement/promptregistry image names with examples (opea-project#1284)

    Align mongo related chathistory/feedbackmanagement/promptregistry image names with examples

    Signed-off-by: Spycsh <sihan.chen@intel.com>
    Co-authored-by: Liang Lv <liang1.lv@intel.com>

commit bef501c
Author: Liang Lv <liang1.lv@intel.com>
Date:   Thu Feb 13 21:18:58 2025 +0800

    Fix VDMS retrieval issue (opea-project#1252)

    * Fix VDMS retrieval issue
    Signed-off-by: lvliang-intel <liang1.lv@intel.com>

commit 23b2be2
Author: ZePan110 <ze.pan@intel.com>
Date:   Thu Feb 13 16:07:14 2025 +0800

    Fix Build latest images on push event workflow (opea-project#1282)

    Signed-off-by: ZePan110 <ze.pan@intel.com>

commit f8e6216
Author: Spycsh <39623753+Spycsh@users.noreply.github.com>
Date:   Wed Feb 12 15:45:14 2025 +0800

    fix metric id issue when init multiple Orchestrator instance (opea-project#1280)

    Signed-off-by: Spycsh <sihan.chen@intel.com>

commit d3906ce
Author: chen, suyue <suyue.chen@intel.com>
Date:   Wed Feb 12 14:56:55 2025 +0800

    update default service list (opea-project#1276)

    Signed-off-by: chensuyue <suyue.chen@intel.com>

commit 17b9672
Author: XinyaoWa <xinyao.wang@intel.com>
Date:   Wed Feb 12 13:53:31 2025 +0800

    Fix langchain and huggingface version to avoid bug in FaqGen and DocSum, remove vllm hpu triton version fix (opea-project#1275)

    * Fix langchain and huggingface version to avoid bug

    Signed-off-by: Xinyao Wang <xinyao.wang@intel.com>

commit b777db7
Author: Letong Han <106566639+letonghan@users.noreply.github.com>
Date:   Mon Feb 10 16:00:55 2025 +0800

    Fix Dataprep Ingest Data Issue. (opea-project#1271)

    * Fix Dataprep Ingest Data Issue.

    Trace:
    1. The update of `langchain_huggingface.HuggingFaceEndpointEmbeddings` caused the wrong size of embedding vectors.
    2. Wrong size vectors are wrongly saved into Redis database, and the indices are not created correctly.
    3. The retriever can not retrieve data from Redis using index due to the
       reasons above.
    4. Then the RAG seems `not work`, for the file uploaded can not be
       retrieved from database.

    Solution:
    Replace all of the `langchain_huggingface.HuggingFaceEndpointEmbeddings`
    to `langchain_community.embeddings.HuggingFaceInferenceAPIEmbeddings`,
    and modify related READMEs and scirpts.

    Related issue:
    - opea-project/GenAIExamples#1473
    - opea-project/GenAIExamples#1482

    ---------

    Signed-off-by: letonghan <letong.han@intel.com>
    Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

commit 0df374b
Author: Daniel De León <111013930+daniel-de-leon-user293@users.noreply.github.com>
Date:   Sun Feb 9 22:01:58 2025 -0800

    Update docs for LLamaGuard & WildGuard Microservice (opea-project#1259)

    * working README for CLI and compose

    Signed-off-by: Daniel Deleon <daniel.de.leon@intel.com>

    * update for direct python execution

    Signed-off-by: Daniel Deleon <daniel.de.leon@intel.com>

    * fix formatting

    Signed-off-by: Daniel Deleon <daniel.de.leon@intel.com>

    * [pre-commit.ci] auto fixes from pre-commit.com hooks

    for more information, see https://pre-commit.ci

    * bring back depends_on condition

    Signed-off-by: Daniel Deleon <daniel.de.leon@intel.com>

    ---------

    Signed-off-by: Daniel Deleon <daniel.de.leon@intel.com>
    Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
    Co-authored-by: Abolfazl Shahbazi <12436063+ashahba@users.noreply.github.com>

commit fb86b5e
Author: Louie Tsai <louie.tsai@intel.com>
Date:   Sat Feb 8 00:58:33 2025 -0800

    Add Deepseek model into validated model table and add required Gaudi cards for LLM microservice  (opea-project#1267)

    * Update README.md for Deepseek support and numbers of required gaudi cards

    Signed-off-by: Tsai, Louie <louie.tsai@intel.com>

    * Update README.md

    Signed-off-by: Tsai, Louie <louie.tsai@intel.com>

    ---------

    Signed-off-by: Tsai, Louie <louie.tsai@intel.com>

commit ecb7f7b
Author: Spycsh <39623753+Spycsh@users.noreply.github.com>
Date:   Fri Feb 7 16:58:22 2025 +0800

    Fix web-retrievers hub client and tei endpoint issue (opea-project#1270)

    * fix web-retrievers hub client and tei endpoint issue

    Signed-off-by: Spycsh <sihan.chen@intel.com>

commit 5baada8
Author: ZePan110 <ze.pan@intel.com>
Date:   Thu Feb 6 15:03:00 2025 +0800

    Fix CD test issue. (opea-project#1263)

    1.Fix template name in README
    2.Fix invalid release name

    Signed-off-by: ZePan110 <ze.pan@intel.com>

commit fa01f46
Author: minmin-intel <minmin.hou@intel.com>
Date:   Wed Feb 5 13:57:57 2025 -0800

    fix tei embedding and tei reranking bug (opea-project#1256)

    Signed-off-by: minmin-intel <minmin.hou@intel.com>
    Co-authored-by: Abolfazl Shahbazi <12436063+ashahba@users.noreply.github.com>

commit 4ede405
Author: Eero Tamminen <eero.t.tamminen@intel.com>
Date:   Wed Feb 5 22:04:50 2025 +0200

    Create token metrics only when they are available (opea-project#1092)

    * Create token metrics only when they are available

    This avoids generation of useless token/request histogram metrics
    for services that use Orchestrator class, but never call its token
    processing functionality.

    (Helps in differentiating frontend megaservice metrics from backend
    megaservice ones, especially when multiple OPEA applications run in
    the same cluster.)

    Also change Orchestrator CI test workaround to use unique prefix for
    each metric instance, instead of metrics being (singleton) class
    variables.

    Signed-off-by: Eero Tamminen <eero.t.tamminen@intel.com>

    * Add locking for latency metric creation / method change

    As that that could be called from multiple request handling threads.

    Signed-off-by: Eero Tamminen <eero.t.tamminen@intel.com>

    ---------

    Signed-off-by: Eero Tamminen <eero.t.tamminen@intel.com>
    Co-authored-by: Malini Bhandaru <malini.bhandaru@intel.com>
dtrawins pushed a commit to dtrawins/GenAIComps that referenced this pull request Mar 8, 2025
* first code for multi-turn

Signed-off-by: minmin-intel <minmin.hou@intel.com>

* test redispersistence

Signed-off-by: minmin-intel <minmin.hou@intel.com>

* integrate persistent store in react llama

Signed-off-by: minmin-intel <minmin.hou@intel.com>

* test multi-turn

Signed-off-by: minmin-intel <minmin.hou@intel.com>

* multiturn for assistants api and chatcompletion api

Signed-off-by: minmin-intel <minmin.hou@intel.com>

* update readme and ut script

Signed-off-by: minmin-intel <minmin.hou@intel.com>

* update readme and ut scripts

Signed-off-by: minmin-intel <minmin.hou@intel.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix bug

Signed-off-by: minmin-intel <minmin.hou@intel.com>

* change memory type naming

Signed-off-by: minmin-intel <minmin.hou@intel.com>

* fix with_memory as str

Signed-off-by: minmin-intel <minmin.hou@intel.com>

---------

Signed-off-by: minmin-intel <minmin.hou@intel.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Signed-off-by: Dariusz Trawinski <Dariusz.Trawinski@intel.com>
dtrawins pushed a commit to dtrawins/GenAIComps that referenced this pull request Mar 8, 2025
* first code for multi-turn

Signed-off-by: minmin-intel <minmin.hou@intel.com>

* test redispersistence

Signed-off-by: minmin-intel <minmin.hou@intel.com>

* integrate persistent store in react llama

Signed-off-by: minmin-intel <minmin.hou@intel.com>

* test multi-turn

Signed-off-by: minmin-intel <minmin.hou@intel.com>

* multiturn for assistants api and chatcompletion api

Signed-off-by: minmin-intel <minmin.hou@intel.com>

* update readme and ut script

Signed-off-by: minmin-intel <minmin.hou@intel.com>

* update readme and ut scripts

Signed-off-by: minmin-intel <minmin.hou@intel.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix bug

Signed-off-by: minmin-intel <minmin.hou@intel.com>

* change memory type naming

Signed-off-by: minmin-intel <minmin.hou@intel.com>

* fix with_memory as str

Signed-off-by: minmin-intel <minmin.hou@intel.com>

---------

Signed-off-by: minmin-intel <minmin.hou@intel.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Signed-off-by: Dariusz Trawinski <Dariusz.Trawinski@intel.com>
madison-evans pushed a commit to SAPD-Intel/GenAIComps that referenced this pull request May 12, 2025
* first code for multi-turn

Signed-off-by: minmin-intel <minmin.hou@intel.com>

* test redispersistence

Signed-off-by: minmin-intel <minmin.hou@intel.com>

* integrate persistent store in react llama

Signed-off-by: minmin-intel <minmin.hou@intel.com>

* test multi-turn

Signed-off-by: minmin-intel <minmin.hou@intel.com>

* multiturn for assistants api and chatcompletion api

Signed-off-by: minmin-intel <minmin.hou@intel.com>

* update readme and ut script

Signed-off-by: minmin-intel <minmin.hou@intel.com>

* update readme and ut scripts

Signed-off-by: minmin-intel <minmin.hou@intel.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix bug

Signed-off-by: minmin-intel <minmin.hou@intel.com>

* change memory type naming

Signed-off-by: minmin-intel <minmin.hou@intel.com>

* fix with_memory as str

Signed-off-by: minmin-intel <minmin.hou@intel.com>

---------

Signed-off-by: minmin-intel <minmin.hou@intel.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

3 participants