Skip to content

feat: Support prompt caching#3587

Merged
Wendong-Fan merged 31 commits intocamel-ai:masterfrom
Zephyroam:prompt-caching
Feb 3, 2026
Merged

feat: Support prompt caching#3587
Wendong-Fan merged 31 commits intocamel-ai:masterfrom
Zephyroam:prompt-caching

Conversation

@Zephyroam
Copy link
Copy Markdown
Collaborator

Description

Describe your changes in detail (optional if the linked issue already contains a detailed description of the changes).

Closes #3586

Checklist

Go over all the following points, and put an x in all the boxes that apply.

  • I have read the CONTRIBUTION guide (required)
  • I have linked this PR to an issue using the Development section on the right sidebar or by adding Fixes #issue-number in the PR description (required)
  • I have checked if any dependencies need to be added or updated in pyproject.toml and uv lock
  • I have updated the tests accordingly (required for a bug fix or a new feature)
  • I have updated the documentation if needed:
  • I have added examples if this is a new feature

If you are unsure about any of these, don't hesitate to ask. We are here to help!

@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented Dec 17, 2025

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

  • 🔍 Trigger a full review
✨ Finishing touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment

Important

Action Needed: IP Allowlist Update

If your organization protects your Git platform with IP whitelisting, please add the new CodeRabbit IP address to your allowlist:

  • 136.113.208.247/32 (new)
  • 34.170.211.100/32
  • 35.222.179.152/32

Reviews will stop working after February 8, 2026 if the new IP is not added to your allowlist.


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown
Member

@Wendong-Fan Wendong-Fan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks @Zephyroam , left some comments below, love the optimize for current workforce prompt

Comment thread camel/models/openai_model.py Outdated
Comment thread camel/models/anthropic_model.py
Comment thread camel/configs/anthropic_config.py Outdated
@review-notebook-app
Copy link
Copy Markdown

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.


Powered by ReviewNB

@Zephyroam
Copy link
Copy Markdown
Collaborator Author

Also removed retired Claude models according to: https://platform.claude.com/docs/en/about-claude/model-deprecations

@Zephyroam Zephyroam requested review from fengju0213 and removed request for fengju0213 January 19, 2026 19:06
@Zephyroam Zephyroam requested a review from bytecii January 20, 2026 19:03
Comment thread camel/configs/openai_config.py Outdated
Copy link
Copy Markdown
Collaborator

@JINO-ROHIT JINO-ROHIT left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @Zephyroam thanks for the PR! from a quick skim , ive added some feedback, i would like to test this feature a bit in the evening when i have some more time.

Comment thread camel/models/anthropic_model.py
Comment thread examples/models/anthropic_model_example.py
Copy link
Copy Markdown
Collaborator

@bytecii bytecii left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM if there are no issues for the https://github.com/camel-ai/camel/pull/3587/files#r2706162404 (prompt modifications)

Comment thread examples/models/anthropic_prompt_caching_example.py
@Zephyroam Zephyroam requested a review from JINO-ROHIT January 21, 2026 16:13
Copy link
Copy Markdown
Collaborator

@JINO-ROHIT JINO-ROHIT left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@Zephyroam
Copy link
Copy Markdown
Collaborator Author

@bytecraftii @Wendong-Fan Now the prompt changes only include changing orders.

Copy link
Copy Markdown
Member

@Wendong-Fan Wendong-Fan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

when i ran examples/agents/chatagent_stream.py with claude model I got error msg:

Error processing with model: <camel.models.anthropic_model.AnthropicModel object at 0x104eda320>
2026-02-03 23:55:14,725 - camel.camel.agents.chat_agent - ERROR - Error in streaming model response: Error code: 400 - {'type': 'error', 'error': {'type': 'invalid_request_error', 'message': 'messages.5: `tool_use` ids were found without `tool_result` blocks immediately after: toolu_016QXsnH8t84QqBCec2zGBez, toolu_01QudEKMtgDwFB56qZKTw6Kc. Each `tool_use` block must have a corresponding `tool_result` block in the next message.'}, 'request_id': 'req_011CXmQbGMdRWQXqjmX4pgdM'}
Traceback (most recent call last):
  File "/Users/enrei/Desktop/repos/camel0120/camel/camel/agents/chat_agent.py", line 4233, in _stream_response
    response = self.model_backend.run(
  File "/Users/enrei/Desktop/repos/camel0120/camel/camel/models/model_manager.py", line 239, in run
    raise exc
  File "/Users/enrei/Desktop/repos/camel0120/camel/camel/models/model_manager.py", line 229, in run
    response = self.current_model.run(messages, response_format, tools)
  File "/Users/enrei/Desktop/repos/camel0120/camel/camel/models/base_model.py", line 212, in wrapped_run
    return original_run(self, messages, *args, **kwargs)
  File "/Users/enrei/Desktop/repos/camel0120/camel/camel/models/base_model.py", line 633, in run
    result = self._run(messages, response_format, tools)
  File "/Users/enrei/Desktop/repos/camel0120/camel/camel/models/anthropic_model.py", line 751, in _run
    stream = create_func(**request_params, stream=True)
  File "/Users/enrei/Desktop/repos/camel0120/camel/.venv/lib/python3.10/site-packages/anthropic/_utils/_utils.py", line 275, in wrapper
    return func(*args, **kwargs)
  File "/Users/enrei/Desktop/repos/camel0120/camel/.venv/lib/python3.10/site-packages/anthropic/resources/messages/messages.py", line 953, in create
    return self._post(
  File "/Users/enrei/Desktop/repos/camel0120/camel/.venv/lib/python3.10/site-packages/anthropic/_base_client.py", line 1336, in post
    return cast(ResponseT, self.request(cast_to, opts, stream=stream, stream_cls=stream_cls))
  File "/Users/enrei/Desktop/repos/camel0120/camel/.venv/lib/python3.10/site-packages/anthropic/_base_client.py", line 1013, in request
    return self._request(
  File "/Users/enrei/Desktop/repos/camel0120/camel/.venv/lib/python3.10/site-packages/anthropic/_base_client.py", line 1117, in _request
    raise self._make_status_error_from_response(err.response) from None
anthropic.BadRequestError: Error code: 400 - {'type': 'error', 'error': {'type': 'invalid_request_error', 'message': 'messages.5: `tool_use` ids were found without `tool_result` blocks immediately after: toolu_016QXsnH8t84QqBCec2zGBez, toolu_01QudEKMtgDwFB56qZKTw6Kc. Each `tool_use` block must have a corresponding `tool_result` block in the next message.'}, 'request_id': 'req_011CXmQbGMdRWQXqjmX4pgdM'}
Error: Error code: 400 - {'type': 'error', 'error': {'type': 'invalid_request_error', 'message': 'messages.5: `tool_use` ids were found without `tool_result` blocks immediately after: toolu_016QXsnH8t84QqBCec2zGBez, toolu_01QudEKMtgDwFB56qZKTw6Kc. Each `tool_use` block must have a corresponding `tool_result` block in the next message.'}, 'request_id': 'req_011CXmQbGMdRWQXqjmX4pgdM'}

could help checking this?

Comment thread camel/models/anthropic_model.py Outdated
Comment thread docs/cookbooks/multi_agent_society/azure_openai_claude_society.ipynb Outdated
@Zephyroam
Copy link
Copy Markdown
Collaborator Author

Issue fixed

@Zephyroam Zephyroam requested a review from Wendong-Fan February 3, 2026 18:55
Copy link
Copy Markdown
Member

@Wendong-Fan Wendong-Fan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks @Zephyroam , LGTM now, we'd better also check the prompt cache feature for gemini model and aws bedrock/ azure, i created one follow up issue for this #3770

@Wendong-Fan Wendong-Fan merged commit 220679f into camel-ai:master Feb 3, 2026
7 of 13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature Request] Support prompt caching

6 participants