fix: ensure web search is performed for research plans to fix #535 (#640)

* fix: ensure web search is performed for research plans to fix #535

          When using certain models (DeepSeek-V3, Qwen3, or local deployments), the
          agent framework failed to trigger web search tools, resulting in hallucinated
          data. This fix implements multiple safeguards:

          1. Add enforce_web_search configuration flag:
             - New config option to mandate web search in research plans
             - Defaults to False for backward compatibility

          2. Add plan validation function validate_and_fix_plan():
             - Validates that plans include at least one research step with web search
             - Enforces web search requirement when enabled
             - Adds default research step if plan has no steps

          3. Enhance coordinator_node fallback logic:
             - When model fails to call tools, fallback to planner instead of __end__
             - Ensures workflow continues even when tool calling fails
             - Logs detailed diagnostic info for debugging

          4. Update prompts for stricter requirements:
             - planner.md: Add MANDATORY web search requirement and clear warnings
             - coordinator.md: Add CRITICAL tool calling requirement
             - Emphasize consequences of missing web search (hallucinated data)

          5. Update tests to reflect new behavior:
             - test_coordinator_node_no_tool_calls: Expect planner instead of __end__
             - test_coordinator_empty_llm_response_corner_case: Same expectation

          Fixes #535 by ensuring:
          - Web search is always performed for research tasks
          - Workflow doesn't terminate on tool calling failures
          - Models with poor tool calling support can still proceed
          - No hallucinated data without real information gathering

* Update src/graph/nodes.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update src/graph/nodes.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* accept the review suggestion of getting configuration

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

src/prompts/coordinator.md

@@ -51,6 +51,15 @@ Your primary responsibilities are:
 - For all other inputs (category 3 - which includes most questions):
   - call `handoff_to_planner()` tool to handoff to planner for research without ANY thoughts.
 
+# Tool Calling Requirements
+
+**CRITICAL**: You MUST call one of the available tools for research requests. This is mandatory:
+- Do NOT respond to research questions without calling a tool
+- For research questions, ALWAYS use either `handoff_to_planner()` or `handoff_after_clarification()`
+- Tool calling is required to ensure the workflow proceeds correctly
+- Never skip tool calling even if you think you can answer the question directly
+- Responding with text alone for research requests will cause the workflow to fail
+
 # Clarification Process (When Enabled)
 
 Goal: Get 2+ dimensions before handing off to planner.

src/prompts/planner.md

@@ -64,6 +64,8 @@ Different types of steps have different web search requirements:
    - Collecting competitor analysis
    - Researching current events or news
    - Finding statistical data or reports
+   - **CRITICAL**: Research plans MUST include at least one step with `need_search: true` to gather real information
+   - Without web search, the report will contain hallucinated/fabricated data
 
 2. **Data Processing Steps** (`need_search: false`):
    - API calls and data extraction
@@ -71,6 +73,15 @@ Different types of steps have different web search requirements:
    - Raw data collection from existing sources
    - Mathematical calculations and analysis
    - Statistical computations and data processing
+   - **NOTE**: Processing steps alone are insufficient - you must include research steps with web search
+
+## Web Search Requirement
+
+**MANDATORY**: Every research plan MUST include at least one step with `need_search: true`. This is critical because:
+- Without web search, models generate hallucinated data
+- Research steps must gather real information from external sources
+- Pure processing steps cannot generate credible information for the final report
+- At least one research step must search the web for factual data
 
 ## Exclusions
 
@@ -143,6 +154,7 @@ When planning information gathering, consider these key aspects and ensure COMPR
   - Create NO MORE THAN {{ max_step_num }} focused and comprehensive steps that cover the most essential aspects
   - Ensure each step is substantial and covers related information categories
   - Prioritize breadth and depth within the {{ max_step_num }}-step constraint
+  - **MANDATORY**: Include at least ONE research step with `need_search: true` to avoid hallucinated data
   - For each step, carefully assess if web search is needed:
     - Research and external data gathering: Set `need_search: true`
     - Internal data processing: Set `need_search: false`
@@ -150,6 +162,7 @@ When planning information gathering, consider these key aspects and ensure COMPR
 - Prioritize depth and volume of relevant information - limited information is not acceptable.
 - Use the same language as the user to generate the plan.
 - Do not include steps for summarizing or consolidating the gathered information.
+- **CRITICAL**: Verify that your plan includes at least one step with `need_search: true` before finalizing
 
 # Output Format