- Implement index-based grouping of tool call chunks in _process_tool_call_chunks()
- Add _validate_tool_call_chunks() for debug logging and validation
- Enhance _process_message_chunk() with tool call ID validation and boundary detection
- Add comprehensive unit tests (17 tests) for tool call chunk processing
- Fix issue where tool names were incorrectly concatenated (e.g., 'web_searchweb_search')
- Ensure chunks from different tool calls (different indices) remain properly separated
- Add detailed logging for debugging tool call streaming issues
* update the code with suggestions of reviewing
* fix: resolve issue #650 - repair missing step_type fields in Plan validation
- Add step_type repair logic to validate_and_fix_plan() to auto-infer missing step_type
- Infer as 'research' when need_search=true, 'processing' when need_search=false
- Add explicit CRITICAL REQUIREMENT section to planner.md emphasizing step_type mandatory for every step
- Include validation checklist and examples showing both research and processing steps
- Add 23 comprehensive unit tests for validate_and_fix_plan() covering all scenarios
- Add 4 integration tests specifically for Issue #650 with actual Plan validation
- Prevents Pydantic ValidationError: 'Field required' for missing step_type
* Update tests/unit/graph/test_plan_validation.py
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Update tests/unit/graph/test_plan_validation.py
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* update the planner.zh_CN.md with recent changes of planner.md
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* feat: Add comprehensive Chinese localization support for issue #412
- Add locale parameter to ChatRequest model to capture user's language preference
- Implement language-aware template loading in template.py with fallback to English
- Update all apply_prompt_template calls to pass locale through the workflow
- Create Chinese translations for 14 core prompt files:
* Main agents: coordinator, planner, researcher, reporter, coder
* Subprocess agents: podcast_script_writer, ppt_composer, prompt_enhancer
* Writing assistant: all 6 prose prompts
- Update app.py to extract and propagate locale through workflow state
- Support both zh-CN and en-US locales with automatic fallback
- Ensure locale flows through all agent nodes and template rendering
* address the review suggestions
* fix: resolve issue #467 - message content validation and Tavily search error handling
This commit implements a comprehensive fix for issue #467 where the application
crashed with 'Field required: input.messages.3.content' error when generating reports.
## Root Cause Analysis
The issue had multiple interconnected causes:
1. Tavily tool returned mixed types (lists/error strings) instead of consistent JSON
2. background_investigation_node didn't handle error cases properly, returning None
3. Missing message content validation before LLM calls
4. Insufficient error diagnostics for content-related errors
## Changes Made
### Part 1: Fix Tavily Search Tool (tavily_search_results_with_images.py)
- Modified _run() and _arun() methods to return JSON strings instead of mixed types
- Error responses now return JSON: {"error": repr(e)}
- Successful responses return JSON string: json.dumps(cleaned_results)
- Ensures tool results always have valid string content for ToolMessages
### Part 2: Fix background_investigation_node Error Handling (graph/nodes.py)
- Initialize background_investigation_results to empty list instead of None
- Added proper JSON parsing for string responses from Tavily tool
- Handle error responses with explicit error logging
- Always return valid JSON (empty list if error) instead of None
### Part 3: Add Message Content Validation (utils/context_manager.py)
- New validate_message_content() function validates all messages before LLM calls
- Ensures all messages have content attribute and valid string content
- Converts complex types (lists, dicts) to JSON strings
- Provides graceful fallback for messages with issues
### Part 4: Enhanced Error Diagnostics (_execute_agent_step in graph/nodes.py)
- Call message validation before agent invocation
- Add detailed logging for content-related errors
- Log message types, content types, and lengths when validation fails
- Helps with future debugging of similar issues
## Testing
- All unit tests pass (395 tests)
- Python syntax verified for all modified files
- No breaking changes to existing functionality
* test: update tests for issue #467 fixes
Update test expectations to match the new implementation:
- Tavily search tool now returns JSON strings instead of mixed types
- background_investigation_node returns empty list [] for errors instead of None
- All tests updated to verify the new behavior
- All 391 tests pass successfully
* Update src/graph/nodes.py
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* fix: support additional Tavily search parameters via configuration to fix#548
- Add include_answer, search_depth, include_raw_content, include_images, include_image_descriptions to SEARCH_ENGINE config
- Update get_web_search_tool() to load these parameters from configuration with sensible defaults
- Parameters are now properly passed to TavilySearchWithImages during initialization
- This fixes 'got an unexpected keyword argument' errors when using web_search tool
- Update tests to verify new parameters are correctly set
* test: add comprehensive unit tests for web search configuration loading
- Add test for custom configuration values (include_answer, search_depth, etc.)
- Add test for empty configuration (all defaults)
- Add test for image_descriptions logic when include_images is false
- Add test for partial configuration
- Add test for missing config file
- Add test for multiple domains in include/exclude lists
All 7 new tests pass and provide comprehensive coverage of configuration loading
and parameter handling for Tavily search tool initialization.
* test: verify all Tavily configuration parameters are optional
Add 8 comprehensive tests to verify that all Tavily engine configuration
parameters are truly optional:
- test_tavily_with_no_search_engine_section: SEARCH_ENGINE section missing
- test_tavily_with_completely_empty_config: Entire config missing
- test_tavily_with_only_include_answer_param: Single param, rest default
- test_tavily_with_only_search_depth_param: Single param, rest default
- test_tavily_with_only_include_domains_param: Domain param, rest default
- test_tavily_with_explicit_false_boolean_values: False values work correctly
- test_tavily_with_empty_domain_lists: Empty lists handled correctly
- test_tavily_all_parameters_optional_mix: Multiple missing params work
These tests verify:
- Tool creation never fails regardless of missing configuration
- All parameters have sensible defaults
- Boolean parameters can be explicitly set to False
- Any combination of optional parameters works
- Domain lists can be empty or omitted
All 15 Tavily configuration tests pass successfully.
* fix: support local models by making thought field optional in Plan model
- Make thought field optional in Plan model to fix Pydantic validation errors with local models
- Add Ollama configuration example to conf.yaml.example
- Update documentation to include local model support
- Improve planner prompt with better JSON format requirements
Fixes local model integration issues where models like qwen3:14b would fail
due to missing thought field in JSON output.
* feat: Add intelligent clarification feature for research queries
- Add multi-turn clarification process to refine vague research questions
- Implement three-dimension clarification standard (Tech/App, Focus, Scope)
- Add clarification state management in coordinator node
- Update coordinator prompt with detailed clarification guidelines
- Add UI settings to enable/disable clarification feature (disabled by default)
- Update workflow to handle clarification rounds recursively
- Add comprehensive test coverage for clarification functionality
- Update documentation with clarification feature usage guide
Key components:
- src/graph/nodes.py: Core clarification logic and state management
- src/prompts/coordinator.md: Detailed clarification guidelines
- src/workflow.py: Recursive clarification handling
- web/: UI settings integration
- tests/: Comprehensive test coverage
- docs/: Updated configuration guide
* fix: Improve clarification conversation continuity
- Add comprehensive conversation history to clarification context
- Include previous exchanges summary in system messages
- Add explicit guidelines for continuing rounds in coordinator prompt
- Prevent LLM from starting new topics during clarification
- Ensure topic continuity across clarification rounds
Fixes issue where LLM would restart clarification instead of building upon previous exchanges.
* fix: Add conversation history to clarification context
* fix: resolve clarification feature message to planer, prompt, test issues
- Optimize coordinator.md prompt template for better clarification flow
- Simplify final message sent to planner after clarification
- Fix API key assertion issues in test_search.py
* fix: Add configurable max_clarification_rounds and comprehensive tests
- Add max_clarification_rounds parameter for external configuration
- Add comprehensive test cases for clarification feature in test_app.py
- Fixes issues found during interactive mode testing where:
- Recursive call failed due to missing initial_state parameter
- Clarification exited prematurely at max rounds
- Incorrect logging of max rounds reached
* Move clarification tests to test_nodes.py and add max_clarification_rounds to zh.json
* fix: add max_clarification_rounds parameter passing from frontend to backend
- Add max_clarification_rounds parameter in store.ts sendMessage function
- Add max_clarification_rounds type definition in chat.ts
- Ensure frontend settings page clarification rounds are correctly passed to backend
* fix: refine clarification workflow state handling and coverage
- Add clarification history reconstruction
- Fix clarified topic accumulation
- Add clarified_research_topic state field
- Preserve clarification state in recursive calls
- Add comprehensive test coverage
* refactor: optimize coordinator logic and type annotations
- Simplify handoff topic logic in coordinator_node
- Update type annotations from Tuple to tuple
- Improve code readability and maintainability
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix: ensure web search is performed for research plans to fix#535
When using certain models (DeepSeek-V3, Qwen3, or local deployments), the
agent framework failed to trigger web search tools, resulting in hallucinated
data. This fix implements multiple safeguards:
1. Add enforce_web_search configuration flag:
- New config option to mandate web search in research plans
- Defaults to False for backward compatibility
2. Add plan validation function validate_and_fix_plan():
- Validates that plans include at least one research step with web search
- Enforces web search requirement when enabled
- Adds default research step if plan has no steps
3. Enhance coordinator_node fallback logic:
- When model fails to call tools, fallback to planner instead of __end__
- Ensures workflow continues even when tool calling fails
- Logs detailed diagnostic info for debugging
4. Update prompts for stricter requirements:
- planner.md: Add MANDATORY web search requirement and clear warnings
- coordinator.md: Add CRITICAL tool calling requirement
- Emphasize consequences of missing web search (hallucinated data)
5. Update tests to reflect new behavior:
- test_coordinator_node_no_tool_calls: Expect planner instead of __end__
- test_coordinator_empty_llm_response_corner_case: Same expectation
Fixes#535 by ensuring:
- Web search is always performed for research tasks
- Workflow doesn't terminate on tool calling failures
- Models with poor tool calling support can still proceed
- No hallucinated data without real information gathering
* Update src/graph/nodes.py
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Update src/graph/nodes.py
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* accept the review suggestion of getting configuration
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
When editing reports, tiptap-markdown escapes special characters (*, _, [, ])
which corrupts LaTeX formulas. This fix:
1. Adds unescapeLatexInMath() function to reverse markdown escaping within
math delimiters ($...$ and 94410...94410)
2. Applies the unescape function in the editor's onChange callback to clean
the markdown before storing it
3. Adds comprehensive tests covering edge cases and round-trip scenarios
The fix ensures formulas like $(f * g)[n]$ remain unescaped when editing,
preventing display errors after save/reload.
Keep fixing #631
This pull request updates the crawl_tool function to return its results as a JSON string instead of a dictionary, and adjusts the unit tests accordingly to handle the new return type. The changes ensure consistent serialization of output and proper validation in tests.
- Change image result type from 'image' to 'image_url' to match OpenAI API expectations
- Wrap image URL in dict structure: {"url": "..."} instead of plain string
- Update SearchResultPostProcessor to handle dict-based image_url during duplicate removal
- Update tests to validate new image format
This fixes the 400 error: Invalid value: 'image'. Supported values are: 'text', 'image_url'...
Co-authored-by: Willem Jiang <143703838+willem-bd@users.noreply.github.com>
- Backend: Convert non-string content (lists, dicts) to JSON strings in _create_event_stream_message to ensure frontend always receives string content
- Frontend: Add type guard before calling startsWith() on toolCall.result for defensive programming
This fixes the TypeError: toolCall.result.startsWith is not a function when tools return complex objects.
Fixes#570 where browser freezes when research plan has 8+ steps.
Performance optimizations:
- Add animation throttling: only animate first 10 activity items
- Reduce animation durations (0.4s → 0.3s for activities, 0.2s → 0.15s for results)
- Remove scale animations (GPU-intensive) from search results
- Limit displayed results (20 pages, 10 images max)
- Add conditional animations based on item index
- Cap animation delays to prevent excessive staggering
- Add React.memo to ActivityMessage and ActivityListItem components
These changes significantly improve performance when rendering multiple
research steps while maintaining visual appeal for smaller lists.
* fix: add missing RunnableConfig parameter to human_feedback_node
This fixes issue #569 where interrupt() was being called outside of a runnable context.
The human_feedback_node was missing the config: RunnableConfig parameter that all other
node functions have, which caused RuntimeError when interrupt() tried to access the config.
- Add config: RunnableConfig parameter to function signature
- Add State type annotation to state parameter for consistency
- Maintains LangGraph execution context required by interrupt()
* test: update human_feedback_node tests to pass RunnableConfig parameter
Update all test functions that call human_feedback_node to include the new
required config parameter. These tests were failing because they were not
providing the RunnableConfig argument after the fix to add proper LangGraph
execution context.
Tests updated:
- test_human_feedback_node_auto_accepted
- test_human_feedback_node_edit_plan
- test_human_feedback_node_accepted
- test_human_feedback_node_invalid_interrupt
- test_human_feedback_node_json_decode_error_first_iteration
- test_human_feedback_node_json_decode_error_second_iteration
- test_human_feedback_node_not_enough_context
All tests now pass the mock_config fixture to human_feedback_node.
- Wrap agent.ainvoke() calls in try-except blocks
- Log full exception tracebacks for better debugging
- Return detailed error messages to users instead of generic 'internal error'
- Include step title and agent name in error context
- Allow workflow to continue gracefully when agent execution fails
- Store error details in observations for audit trail
* fix: prevent repeated content animation during thinking streaming (#614)
- Implement chunked rendering using reasoningContentChunks
- Static content (previous chunks) renders without animation
- Only current streaming chunk animates
- Disable animation on plan content (title, thought, steps) during streaming
- Animation applies after content finishes streaming (when complete)
- Prevents visual duplication of repeated sentences in thinking process
- Changed key from question text to combination of index and question text
- Ensures unique keys even if translation has duplicate questions
- Resolves React warning: 'Each child in a list should have a unique key prop'
- Set asyncio.WindowsSelectorEventLoopPolicy() on Windows at app module level
- Ensures psycopg can run in async mode on Windows regardless of entry point
- Fixes 'ProactorEventLoop' error when using PostgreSQL checkpointer
- Works with all entry points: server.py, uvicorn, langgraph dev, etc.
Added 'node --test tests/*.test.ts' to the lint-frontend target to ensure
frontend unit tests are run as part of the CI/quality checks workflow.
This ensures:
- Math formula normalization tests run before build
- Tests are validated alongside linting and type checking
- All 19 frontend tests pass before deployment
Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
Test files use .ts extensions in imports for Node's native test runner
compatibility, which conflicts with TypeScript's default behavior.
Excluding test files from tsconfig allows tests to run with Node while
maintaining strict type checking for the main codebase.
Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
This fix addresses the issue where math formulas become corrupted or
incorrectly displayed after editing the generated report in the editor.
**Root Cause:**
The issue occurred due to incompatibility between markdown processing
in the display component and the Tiptap editor:
1. Display component used \[\] and \(\) LaTeX delimiters
2. Tiptap Mathematics extension expects $ and 70868 delimiters
3. tiptap-markdown didn't have built-in math node serialization
4. Math syntax was lost/corrupted during editor save operations
**Solution Implemented:**
1. Created MathematicsWithMarkdown extension that adds markdown
serialization support to Tiptap's Mathematics nodes
2. Added math delimiter normalization functions:
- normalizeMathForEditor(): Converts LaTeX delimiters to $/70868
- normalizeMathForDisplay(): Standardizes all delimiters to 70868
3. Updated Markdown component to use new normalization
4. Updated ReportEditor to normalize content before loading
**Changes:**
- web/src/components/editor/math-serializer.ts (new)
- web/src/components/editor/extensions.tsx
- web/src/components/editor/index.tsx
- web/src/components/deer-flow/markdown.tsx
- web/src/core/utils/markdown.ts
- web/tests/markdown-math-editor.test.ts (new)
- web/tests/markdown-katex.test.ts
**Testing:**
- Added 15 comprehensive tests for math normalization round-trip
- All tests passing (math editor + existing katex tests)
- Verified TypeScript compilation and linting
Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
Bug Fix
This PR fixes the issue where max_clarification_rounds parameter was not being passed from the frontend to the backend, causing a TypeError: '<' not supported between instances of 'int' and 'NoneType' error.
Technical Details
The issue was that the frontend was not passing the max_clarification_rounds parameter to the backend API, causing the backend to receive None values and fail during comparison operations. This fix ensures the parameter is properly typed and passed through the entire request chain.
* fix: support local models by making thought field optional in Plan model
- Make thought field optional in Plan model to fix Pydantic validation errors with local models
- Add Ollama configuration example to conf.yaml.example
- Update documentation to include local model support
- Improve planner prompt with better JSON format requirements
Fixes local model integration issues where models like qwen3:14b would fail
due to missing thought field in JSON output.
* feat: Add intelligent clarification feature for research queries
- Add multi-turn clarification process to refine vague research questions
- Implement three-dimension clarification standard (Tech/App, Focus, Scope)
- Add clarification state management in coordinator node
- Update coordinator prompt with detailed clarification guidelines
- Add UI settings to enable/disable clarification feature (disabled by default)
- Update workflow to handle clarification rounds recursively
- Add comprehensive test coverage for clarification functionality
- Update documentation with clarification feature usage guide
Key components:
- src/graph/nodes.py: Core clarification logic and state management
- src/prompts/coordinator.md: Detailed clarification guidelines
- src/workflow.py: Recursive clarification handling
- web/: UI settings integration
- tests/: Comprehensive test coverage
- docs/: Updated configuration guide
* fix: Improve clarification conversation continuity
- Add comprehensive conversation history to clarification context
- Include previous exchanges summary in system messages
- Add explicit guidelines for continuing rounds in coordinator prompt
- Prevent LLM from starting new topics during clarification
- Ensure topic continuity across clarification rounds
Fixes issue where LLM would restart clarification instead of building upon previous exchanges.
* fix: Add conversation history to clarification context
* fix: resolve clarification feature message to planer, prompt, test issues
- Optimize coordinator.md prompt template for better clarification flow
- Simplify final message sent to planner after clarification
- Fix API key assertion issues in test_search.py
* fix: Add configurable max_clarification_rounds and comprehensive tests
- Add max_clarification_rounds parameter for external configuration
- Add comprehensive test cases for clarification feature in test_app.py
- Fixes issues found during interactive mode testing where:
- Recursive call failed due to missing initial_state parameter
- Clarification exited prematurely at max rounds
- Incorrect logging of max rounds reached
* Move clarification tests to test_nodes.py and add max_clarification_rounds to zh.json
- Make thought field optional in Plan model to fix Pydantic validation errors with local models
- Add Ollama configuration example to conf.yaml.example
- Update documentation to include local model support
- Improve planner prompt with better JSON format requirements
Fixes local model integration issues where models like qwen3:14b would fail
due to missing thought field in JSON output.
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* feat:Add context compress
* feat: Add unit test
* feat: add unit test for context manager
* feat: add postprocessor param && code format
* feat: add configuration guide
* fix: fix the configuration_guide
* fix: fix the unit test
* fix: fix the default value
* feat: add test and log for context_manager
* add searx/searxng support
* nit
* Fix indentation in search.py for readability
* Clean up imports in search.py
Removed unused imports from search.py
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* feat:support config tavily search results
* feat: support config tavily search results
* feat: update the default value of include_images
* fix: fix the test
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* Debug deerflow server, web with vscode
Signed-off-by: shjy <asdf_0403@qq.com>
* removed the duplicated setting of .vscode
---------
Signed-off-by: shjy <asdf_0403@qq.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* feat: add Google AI Studio API support with platform-based detection
* chore: update configuration_guide.md
* fix the uv.lock formate error
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* feat: Implement MilvusRetriever with embedding model and resource management
* chore: Update configuration and loader files for consistency
* chore: Clean up test_milvus.py for improved readability and organization
* feat: Add tests for DashscopeEmbeddings query and document embedding methods
* feat: Add tests for embedding model initialization and example file loading in MilvusProvider
* chore: Remove unused imports and clean up test_milvus.py for better readability
* chore: Clean up test_milvus.py for improved readability and organization
* chore: Clean up test_milvus.py for improved readability and organization
* fix: replace print statements with logging in recursion limit function
* Implement feature X to enhance user experience and optimize performance
* refactor: clean up unused imports and comments in AboutTab component
* Implement feature X to enhance user experience and fix bug Y in module Z
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
