fix: apply context compression to prevent token overflow (Issue #721) (#722)

* fix: apply context compression to prevent token overflow (Issue #721)

- Add token_limit configuration to conf.yaml.example for BASIC_MODEL and REASONING_MODEL
- Implement context compression in _execute_agent_step() before agent invocation
- Preserve first 3 messages (system prompt + context) during compression
- Enhance ContextManager logging with better token count reporting
- Prevent 400 Input tokens exceeded errors by automatically compressing message history

* feat: add model-based token limit inference for Issue #721

- Add smart default token limits based on common LLM models
- Support model name inference when token_limit not explicitly configured
- Models include: OpenAI (GPT-4o, GPT-4, etc.), Claude, Gemini, Doubao, DeepSeek, etc.
- Conservative defaults prevent token overflow even without explicit configuration
- Priority: explicit config > model inference > safe default (100,000 tokens)
- Ensures Issue #721 protection for all users, not just those with token_limit set
This commit is contained in:
Willem Jiang
2025-11-28 18:52:42 +08:00
committed by GitHub
parent 223ec57fe4
commit b24f4d3f38
4 changed files with 110 additions and 8 deletions

View File

@@ -12,6 +12,7 @@ BASIC_MODEL:
api_key: xxxx
# max_retries: 3 # Maximum number of retries for LLM calls
# verify_ssl: false # Uncomment this line to disable SSL certificate verification for self-signed certificates
# token_limit: 200000 # Maximum input tokens for context compression (prevents token overflow errors)
# Local model configuration example:
@@ -39,6 +40,7 @@ BASIC_MODEL:
# model: "doubao-1-5-thinking-pro-m-250428"
# api_key: xxxx
# max_retries: 3 # Maximum number of retries for LLM calls
# token_limit: 150000 # Maximum input tokens for context compression
# OTHER SETTINGS: