Support Long context for DocSum

[XinyaoWa]

Description

Support Long context for DocSum with four modes:
• Stuff (Default mode): input actual tokens, need to increase the max_input_token if want to use large context
• Truncate: truncate the tokens exceed the limitation
• Map_reduce: split the inputs into multiple chunks, map each document to an individual summary, then consolidate those summaries into a single global summary
• Refine: split the inputs into multiple chunks, generate summary for the first one, then combine with the second, loops over every remaining chunks to get the final summary

Issues

List the issue or RFC link this PR is working on. If there is no such link, please mark it as n/a.

Type of change

List the type of change like below. Please delete options that are not relevant.

• New feature (non-breaking change which adds new functionality)

Dependencies

List the newly introduced 3rd party dependency if exists.

Tests

Describe the tests that you ran to verify your changes.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Files with missing lines	Coverage Δ
comps/cores/proto/docarray.py	99.44% <100.00%> (+0.01%)	⬆️

... and 1 file with indirect coverage changes

[eero-t]

docsum wrapper for TGI failed in CI with: ImportError: cannot import name 'DocSumLLMParams' from 'comps' (/home/user/comps/__init__.py)

[jjmaturino]

@eero-t How long does it usually take for these PR's to get reviewed? I opened a couple this week...

[eero-t]

@eero-t How long does it usually take for these PR's to get reviewed? I opened a couple this week...

I'm not developer in this project, but people assume PR to be incomplete / not ready for review when it fails CI tests.

(If you think CI failure is not due to your own changes, file ticket for it, and state that it blocks the PR.)

Once CI tests pass, I think you could ping here one of the people who've reviewed / merged other PRs, if nobody comments it within few days / weeks (depending on how urgent you think it is). As you're from Intel, you could also ping OPEA Intel devs internally.

[XinyaoWa]

@eero-t How long does it usually take for these PR's to get reviewed? I opened a couple this week...

Hi, sorry I was busy with other projects some time ago, now I will continue to fix this PR, it should be ready in 1~2 workdays!

[eero-t]

I do not see why example-test is failing, but the llms_summarization_tgi_langchain, intel_cpu test issue seems like some (hopefully temporary) issue with CI, as container it creates from image is not available:

+ docker run -d --name=test-comps-llm-sum-tgi-server -p 5076:9000 --ipc=host -e http_proxy= -e https_proxy= -e TGI_LLM_ENDPOINT=http://100.80.243.74:5075/ -e HUGGINGFACEHUB_API_TOKEN=*** opea/llm-sum-tgi:comps
6e2f7a14694b36b68359f0764a6b7bf12a0a754d244d82baa60e415f42a13a19
docker: Error response from daemon: container is marked for removal and cannot be started.
...
Error response from daemon: No such container: test-comps-llm-sum-tgi-server
+ exit 1
Error: Process completed with exit code 1.

[eero-t]

Current CI failure is due to bad SERVICE_NAME value:

+ local 'SERVICE_NAME=llm - summarization'
...
./llms/test_llms_summarization_tgi_langchain.sh: line 71: ${LOG_PATH}/${SERVICE_NAME}.log: ambiguous redirect

Please use shellcheck (from shellcheck package) to check that scripts handle things like arguments properly.