Files
deer-flow/tests/unit/agents/test_tool_interceptor.py
Willem Jiang bcc403ecd3 feat: implement tool-specific interrupts for create_react_agent (#572) (#659)
* feat: implement tool-specific interrupts for create_react_agent (#572)

          Add selective tool interrupt capability allowing interrupts before specific tools
          rather than all tools. Users can now configure which tools trigger interrupts via
          the interrupt_before_tools parameter.

          Changes:
          - Create ToolInterceptor class to handle tool-specific interrupt logic
          - Add interrupt_before_tools parameter to create_agent() function
          - Extend Configuration with interrupt_before_tools field
          - Add interrupt_before_tools to ChatRequest API
          - Update nodes.py to pass interrupt configuration to agents
          - Update app.py workflow to support tool interrupt configuration
          - Add comprehensive unit tests for tool interceptor

          Features:
          - Selective tool interrupts: interrupt only specific tools by name
          - Approval keywords: recognize user approval (approved, proceed, accept, etc.)
          - Backward compatible: optional parameter, existing code unaffected
          - Flexible: works with default tools and MCP-powered tools
          - Works with existing resume mechanism for seamless workflow

          Example usage:
            request = ChatRequest(
              messages=[...],
              interrupt_before_tools=['db_tool', 'sensitive_api']
            )

* test: add comprehensive integration tests for tool-specific interrupts (#572)

Add 24 integration tests covering all aspects of the tool interceptor feature:

Test Coverage:
- Agent creation with tool interrupts
- Configuration support (with/without interrupts)
- ChatRequest API integration
- Multiple tools with selective interrupts
- User approval/rejection flows
- Tool wrapping and functionality preservation
- Error handling and edge cases
- Approval keyword recognition
- Complex tool inputs
- Logging and monitoring

All tests pass with 100% coverage of tool interceptor functionality.

Tests verify:
✓ Selective tool interrupts work correctly
✓ Only specified tools trigger interrupts
✓ Non-matching tools execute normally
✓ User feedback is properly parsed
✓ Tool functionality is preserved after wrapping
✓ Error handling works as expected
✓ Configuration options are properly respected
✓ Logging provides useful debugging info

* fix: mock get_llm_by_type in agent creation test

Fix test_agent_creation_with_tool_interrupts which was failing because
get_llm_by_type() was being called before create_react_agent was mocked.

Changes:
- Add mock for get_llm_by_type in test
- Use context manager composition for multiple patches
- Test now passes and validates tool wrapping correctly

All 24 integration tests now pass successfully.

* refactor: use mock assertion methods for consistent and clearer error messages

Update integration tests to use mock assertion methods instead of direct
attribute checking for consistency and clearer error messages:

Changes:
- Replace 'assert mock_interrupt.called' with 'mock_interrupt.assert_called()'
- Replace 'assert not mock_interrupt.called' with 'mock_interrupt.assert_not_called()'

Benefits:
- Consistent with pytest-mock and unittest.mock best practices
- Clearer error messages when assertions fail
- Better IDE autocompletion support
- More professional test code

All 42 tests pass with improved assertion patterns.

* refactor: use default_factory for interrupt_before_tools consistency

Improve consistency between ChatRequest and Configuration implementations:

Changes:
- ChatRequest.interrupt_before_tools: Use Field(default_factory=list) instead of Optional[None]
- Remove unnecessary 'or []' conversion in app.py line 505
- Aligns with Configuration.interrupt_before_tools implementation pattern
- No functional changes - all tests still pass

Benefits:
- Consistent field definition across codebase
- Simpler and cleaner code
- Reduced chance of None/empty list bugs
- Better alignment with Pydantic best practices

All 42 tests passing.

* refactor: improve tool input formatting in interrupt messages

Enhance tool input representation for better readability in interrupt messages:

Changes:
- Add json import for better formatting
- Create _format_tool_input() static method with JSON serialization
- Use JSON formatting for dicts, lists, tuples with indent=2
- Fall back to str() for non-serializable types
- Handle None input specially (returns 'No input')
- Improve interrupt message formatting with better spacing

Benefits:
- Complex tool inputs now display as readable JSON
- Nested structures are properly indented and visible
- Better user experience when reviewing tool inputs before approval
- Handles edge cases gracefully with fallbacks
- Improved logging output for debugging

Example improvements:
Before: {'query': 'SELECT...', 'limit': 10, 'nested': {'key': 'value'}}
After:
{
  "query": "SELECT...",
  "limit": 10,
  "nested": {
    "key": "value"
  }
}

All 42 tests still passing.

* test: add comprehensive unit tests for tool input formatting
2025-10-26 09:47:03 +08:00

434 lines
15 KiB
Python

# Copyright (c) 2025 Bytedance Ltd. and/or its affiliates
# SPDX-License-Identifier: MIT
import pytest
from unittest.mock import Mock, patch, MagicMock, AsyncMock
from langchain_core.tools import tool, BaseTool
from src.agents.tool_interceptor import (
ToolInterceptor,
wrap_tools_with_interceptor,
)
class TestToolInterceptor:
"""Tests for ToolInterceptor class."""
def test_init_with_tools(self):
"""Test initializing interceptor with tool list."""
tools = ["db_tool", "api_tool"]
interceptor = ToolInterceptor(tools)
assert interceptor.interrupt_before_tools == tools
def test_init_without_tools(self):
"""Test initializing interceptor without tools."""
interceptor = ToolInterceptor()
assert interceptor.interrupt_before_tools == []
def test_should_interrupt_with_matching_tool(self):
"""Test should_interrupt returns True for matching tools."""
tools = ["db_tool", "api_tool"]
interceptor = ToolInterceptor(tools)
assert interceptor.should_interrupt("db_tool") is True
assert interceptor.should_interrupt("api_tool") is True
def test_should_interrupt_with_non_matching_tool(self):
"""Test should_interrupt returns False for non-matching tools."""
tools = ["db_tool", "api_tool"]
interceptor = ToolInterceptor(tools)
assert interceptor.should_interrupt("search_tool") is False
assert interceptor.should_interrupt("crawl_tool") is False
def test_should_interrupt_empty_list(self):
"""Test should_interrupt with empty interrupt list."""
interceptor = ToolInterceptor([])
assert interceptor.should_interrupt("db_tool") is False
def test_parse_approval_with_approval_keywords(self):
"""Test parsing user feedback with approval keywords."""
assert ToolInterceptor._parse_approval("approved") is True
assert ToolInterceptor._parse_approval("approve") is True
assert ToolInterceptor._parse_approval("yes") is True
assert ToolInterceptor._parse_approval("proceed") is True
assert ToolInterceptor._parse_approval("continue") is True
assert ToolInterceptor._parse_approval("ok") is True
assert ToolInterceptor._parse_approval("okay") is True
assert ToolInterceptor._parse_approval("accepted") is True
assert ToolInterceptor._parse_approval("accept") is True
assert ToolInterceptor._parse_approval("[approved]") is True
def test_parse_approval_case_insensitive(self):
"""Test parsing is case-insensitive."""
assert ToolInterceptor._parse_approval("APPROVED") is True
assert ToolInterceptor._parse_approval("Approved") is True
assert ToolInterceptor._parse_approval("PROCEED") is True
def test_parse_approval_with_surrounding_text(self):
"""Test parsing with surrounding text."""
assert ToolInterceptor._parse_approval("Sure, proceed with the tool") is True
assert ToolInterceptor._parse_approval("[ACCEPTED] I approve this") is True
def test_parse_approval_rejection(self):
"""Test parsing rejects non-approval feedback."""
assert ToolInterceptor._parse_approval("no") is False
assert ToolInterceptor._parse_approval("reject") is False
assert ToolInterceptor._parse_approval("cancel") is False
assert ToolInterceptor._parse_approval("random feedback") is False
def test_parse_approval_empty_string(self):
"""Test parsing empty string."""
assert ToolInterceptor._parse_approval("") is False
def test_parse_approval_none(self):
"""Test parsing None."""
assert ToolInterceptor._parse_approval(None) is False
@patch("src.agents.tool_interceptor.interrupt")
def test_wrap_tool_with_interrupt(self, mock_interrupt):
"""Test wrapping a tool with interrupt."""
mock_interrupt.return_value = "approved"
# Create a simple test tool
@tool
def test_tool(input_text: str) -> str:
"""Test tool."""
return f"Result: {input_text}"
interceptor = ToolInterceptor(["test_tool"])
# Wrap the tool
wrapped_tool = ToolInterceptor.wrap_tool(test_tool, interceptor)
# Invoke the wrapped tool
result = wrapped_tool.invoke("hello")
# Verify interrupt was called
mock_interrupt.assert_called_once()
assert "test_tool" in mock_interrupt.call_args[0][0]
@patch("src.agents.tool_interceptor.interrupt")
def test_wrap_tool_without_interrupt(self, mock_interrupt):
"""Test wrapping a tool that doesn't trigger interrupt."""
# Create a simple test tool
@tool
def test_tool(input_text: str) -> str:
"""Test tool."""
return f"Result: {input_text}"
interceptor = ToolInterceptor(["other_tool"])
# Wrap the tool
wrapped_tool = ToolInterceptor.wrap_tool(test_tool, interceptor)
# Invoke the wrapped tool
result = wrapped_tool.invoke("hello")
# Verify interrupt was NOT called
mock_interrupt.assert_not_called()
assert "Result: hello" in str(result)
@patch("src.agents.tool_interceptor.interrupt")
def test_wrap_tool_user_rejects(self, mock_interrupt):
"""Test user rejecting tool execution."""
mock_interrupt.return_value = "no"
@tool
def test_tool(input_text: str) -> str:
"""Test tool."""
return f"Result: {input_text}"
interceptor = ToolInterceptor(["test_tool"])
wrapped_tool = ToolInterceptor.wrap_tool(test_tool, interceptor)
# Invoke the wrapped tool
result = wrapped_tool.invoke("hello")
# Verify tool was not executed
assert isinstance(result, dict)
assert "error" in result
assert result["status"] == "rejected"
def test_wrap_tools_with_interceptor_empty_list(self):
"""Test wrapping tools with empty interrupt list."""
@tool
def test_tool(input_text: str) -> str:
"""Test tool."""
return f"Result: {input_text}"
tools = [test_tool]
wrapped_tools = wrap_tools_with_interceptor(tools, [])
# Should return tools as-is
assert len(wrapped_tools) == 1
assert wrapped_tools[0].name == "test_tool"
def test_wrap_tools_with_interceptor_none(self):
"""Test wrapping tools with None interrupt list."""
@tool
def test_tool(input_text: str) -> str:
"""Test tool."""
return f"Result: {input_text}"
tools = [test_tool]
wrapped_tools = wrap_tools_with_interceptor(tools, None)
# Should return tools as-is
assert len(wrapped_tools) == 1
@patch("src.agents.tool_interceptor.interrupt")
def test_wrap_tools_with_interceptor_multiple(self, mock_interrupt):
"""Test wrapping multiple tools."""
mock_interrupt.return_value = "approved"
@tool
def db_tool(query: str) -> str:
"""DB tool."""
return f"Query result: {query}"
@tool
def search_tool(query: str) -> str:
"""Search tool."""
return f"Search result: {query}"
tools = [db_tool, search_tool]
wrapped_tools = wrap_tools_with_interceptor(tools, ["db_tool"])
# Only db_tool should trigger interrupt
db_result = wrapped_tools[0].invoke("test query")
assert mock_interrupt.call_count == 1
search_result = wrapped_tools[1].invoke("test query")
# No additional interrupt calls for search_tool
assert mock_interrupt.call_count == 1
def test_wrap_tool_preserves_tool_properties(self):
"""Test that wrapping preserves tool properties."""
@tool
def my_tool(input_text: str) -> str:
"""My tool description."""
return f"Result: {input_text}"
interceptor = ToolInterceptor([])
wrapped_tool = ToolInterceptor.wrap_tool(my_tool, interceptor)
assert wrapped_tool.name == "my_tool"
assert wrapped_tool.description == "My tool description."
class TestFormatToolInput:
"""Tests for tool input formatting functionality."""
def test_format_tool_input_none(self):
"""Test formatting None input."""
result = ToolInterceptor._format_tool_input(None)
assert result == "No input"
def test_format_tool_input_string(self):
"""Test formatting string input."""
input_str = "SELECT * FROM users"
result = ToolInterceptor._format_tool_input(input_str)
assert result == input_str
def test_format_tool_input_simple_dict(self):
"""Test formatting simple dictionary."""
input_dict = {"query": "test", "limit": 10}
result = ToolInterceptor._format_tool_input(input_dict)
# Should be valid JSON
import json
parsed = json.loads(result)
assert parsed == input_dict
# Should be indented
assert "\n" in result
def test_format_tool_input_nested_dict(self):
"""Test formatting nested dictionary."""
input_dict = {
"query": "SELECT * FROM users",
"config": {
"timeout": 30,
"retry": True
}
}
result = ToolInterceptor._format_tool_input(input_dict)
import json
parsed = json.loads(result)
assert parsed == input_dict
assert "timeout" in result
assert "retry" in result
def test_format_tool_input_list(self):
"""Test formatting list input."""
input_list = ["item1", "item2", 123]
result = ToolInterceptor._format_tool_input(input_list)
import json
parsed = json.loads(result)
assert parsed == input_list
def test_format_tool_input_complex_list(self):
"""Test formatting list with mixed types."""
input_list = ["text", 42, 3.14, True, {"key": "value"}]
result = ToolInterceptor._format_tool_input(input_list)
import json
parsed = json.loads(result)
assert parsed == input_list
def test_format_tool_input_tuple(self):
"""Test formatting tuple input."""
input_tuple = ("item1", "item2", 123)
result = ToolInterceptor._format_tool_input(input_tuple)
import json
parsed = json.loads(result)
# JSON converts tuples to lists
assert parsed == list(input_tuple)
def test_format_tool_input_integer(self):
"""Test formatting integer input."""
result = ToolInterceptor._format_tool_input(42)
assert result == "42"
def test_format_tool_input_float(self):
"""Test formatting float input."""
result = ToolInterceptor._format_tool_input(3.14)
assert result == "3.14"
def test_format_tool_input_boolean(self):
"""Test formatting boolean input."""
result_true = ToolInterceptor._format_tool_input(True)
result_false = ToolInterceptor._format_tool_input(False)
assert result_true == "True"
assert result_false == "False"
def test_format_tool_input_deeply_nested(self):
"""Test formatting deeply nested structure."""
input_dict = {
"level1": {
"level2": {
"level3": {
"level4": ["a", "b", "c"],
"data": {"key": "value"}
}
}
}
}
result = ToolInterceptor._format_tool_input(input_dict)
import json
parsed = json.loads(result)
assert parsed == input_dict
def test_format_tool_input_empty_dict(self):
"""Test formatting empty dictionary."""
result = ToolInterceptor._format_tool_input({})
assert result == "{}"
def test_format_tool_input_empty_list(self):
"""Test formatting empty list."""
result = ToolInterceptor._format_tool_input([])
assert result == "[]"
def test_format_tool_input_special_characters(self):
"""Test formatting dict with special characters."""
input_dict = {
"query": 'SELECT * FROM users WHERE name = "John"',
"path": "/usr/local/bin",
"unicode": "你好世界"
}
result = ToolInterceptor._format_tool_input(input_dict)
import json
parsed = json.loads(result)
assert parsed == input_dict
def test_format_tool_input_numbers_as_strings(self):
"""Test formatting with various number types."""
input_dict = {
"int": 42,
"float": 3.14159,
"negative": -100,
"zero": 0,
"scientific": 1e-5
}
result = ToolInterceptor._format_tool_input(input_dict)
import json
parsed = json.loads(result)
assert parsed["int"] == 42
assert abs(parsed["float"] - 3.14159) < 0.00001
assert parsed["negative"] == -100
assert parsed["zero"] == 0
def test_format_tool_input_with_none_values(self):
"""Test formatting dict with None values."""
input_dict = {
"key1": "value1",
"key2": None,
"key3": {"nested": None}
}
result = ToolInterceptor._format_tool_input(input_dict)
import json
parsed = json.loads(result)
assert parsed == input_dict
def test_format_tool_input_indentation(self):
"""Test that output uses proper indentation (2 spaces)."""
input_dict = {"outer": {"inner": "value"}}
result = ToolInterceptor._format_tool_input(input_dict)
# Should have indented lines
assert " " in result # 2-space indentation
lines = result.split("\n")
# Check that indentation increases with nesting
assert any(line.startswith(" ") for line in lines)
def test_format_tool_input_preserves_order_insertion(self):
"""Test that dict order is preserved in output."""
input_dict = {
"first": 1,
"second": 2,
"third": 3
}
result = ToolInterceptor._format_tool_input(input_dict)
import json
parsed = json.loads(result)
# Verify all keys are present
assert set(parsed.keys()) == {"first", "second", "third"}
def test_format_tool_input_long_strings(self):
"""Test formatting with long string values."""
long_string = "x" * 1000
input_dict = {"long": long_string}
result = ToolInterceptor._format_tool_input(input_dict)
import json
parsed = json.loads(result)
assert parsed["long"] == long_string
def test_format_tool_input_mixed_types_in_list(self):
"""Test formatting list with mixed complex types."""
input_list = [
"string",
42,
{"dict": "value"},
[1, 2, 3],
True,
None
]
result = ToolInterceptor._format_tool_input(input_list)
import json
parsed = json.loads(result)
assert len(parsed) == 6
assert parsed[0] == "string"
assert parsed[1] == 42
assert parsed[2] == {"dict": "value"}
assert parsed[3] == [1, 2, 3]
assert parsed[4] is True
assert parsed[5] is None