Commit Graph

52 Commits

Author SHA1 Message Date
greatmengqi
accf5b5f8e fix: add sync after_model to TitleMiddleware (#1190) 2026-03-19 15:46:31 +08:00
Ryanba
f67c3d2c9e fix(harness): skip duplicate memory facts (#1193)
* fix(harness): skip duplicate memory facts

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-opencode)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>

* docs: note memory fact deduplication

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-opencode)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>

* Apply suggestions from code review

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
2026-03-18 22:41:13 +08:00
Ryanba
4c78188896 fix(gateway): remove generated markdown on upload delete (#1170)
* fix(gateway): remove generated markdown on upload delete

Keep thread upload storage consistent by deleting the generated markdown companion when the original convertible upload is removed.

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-opencode)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
2026-03-18 16:31:26 +08:00
DanielWalnut
feac03ecbc fix(harness): allow agent read access to /mnt/skills in local sandbox (#1178)
* fix(harness): allow agent read access to /mnt/skills in local sandbox

Skill files under /mnt/skills/ were blocked by the path validator,
preventing agents from reading skill definitions. This change:

- Refactors `resolve_local_tool_path` into `validate_local_tool_path`,
  a pure security gate that no longer resolves paths (left to the sandbox)
- Permits read-only access to the skills container path (/mnt/skills by
  default, configurable via config.skills.container_path)
- Blocks write access to skills paths (PermissionError)
- Allows /mnt/skills in bash command path validation
- Adds `LocalSandbox.update_path_mappings` and injects per-thread
  user-data mappings into the sandbox so all virtual-path resolution
  is handled uniformly by the sandbox layer
- Covers all new behaviour with tests

Fixes #1177

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* refactor(sandbox): unify all virtual path resolution in tools.py

Move skills path resolution from LocalSandbox into tools.py so that all
virtual-to-host path translation (user-data and skills) lives in one
layer.  LocalSandbox becomes a pure execution layer that receives only
real host paths — no more path_mappings, _resolve_path, or reverse
resolve logic.

This addresses architecture feedback that path resolution was split
across two layers (tools.py for user-data, LocalSandbox for skills),
making the flow hard to follow.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(sandbox): address Copilot review — cache-on-success and error path masking

- Replace @lru_cache with manual cache-on-success for _get_skills_container_path
  and _get_skills_host_path so transient failures at startup don't permanently
  disable skills access.
- Add _sanitize_error() helper that masks host filesystem paths in error
  messages via mask_local_paths_in_output before returning them to the agent.
- Apply _sanitize_error() to all catch-all (Exception/OSError) handlers in
  sandbox tool functions to prevent host path leakage in error output.
- Remove unused lru_cache import.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-17 21:44:36 +08:00
lhd
0091d9f071 feat(tools): add tool_search for deferred MCP tool loading (#1176)
* feat(tools): add tool_search for deferred MCP tool loading

When multiple MCP servers are enabled, total tool count can exceed 30-50,
causing context bloat and degraded tool selection accuracy. This adds a
deferred tool loading mechanism controlled by `tool_search.enabled` config.

- Add ToolSearchConfig with single `enabled` field
- Add DeferredToolRegistry with regex search (select:, +keyword, keyword)
- Add tool_search tool returning OpenAI-compatible function JSON
- Add DeferredToolFilterMiddleware to hide deferred schemas from bind_tools
- Add <available-deferred-tools> section to system prompt
- Enable MCP tool_name_prefix to prevent cross-server name collisions
- Add 34 unit tests covering registry, tool, prompt, and middleware

* fix: reset stale deferred registry and bump config_version

- Reset deferred registry upfront in get_available_tools() to prevent
  stale tool entries when MCP servers are disabled between calls
- Bump config_version to 2 for new tool_search config field

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(tests): mock get_app_config in prompt section tests for CI

CI has no config.yaml, causing TestDeferredToolsPromptSection to fail
with FileNotFoundError. Add autouse fixture to mock get_app_config.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-17 20:43:55 +08:00
Ryanba
b1913a1902 fix(harness): normalize structured content for titles (#1155)
* fix(harness): normalize structured content for titles

Flatten structured LangChain message content before prompting the title model so list/block payloads don't leak Python reprs into generated thread titles.

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-opencode)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>

* Apply suggestions from code review

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
2026-03-17 09:10:09 +08:00
-Astraia-
191b60a326 fix: issue 1138 windows encoding (#1139)
* fix(windows): use utf-8 for text file operations

* fix(windows): normalize sandbox path masking

* fix(windows): preserve utf-8 handling after backend split
2026-03-16 16:53:12 +08:00
DanielWalnut
76803b826f refactor: split backend into harness (deerflow.*) and app (app.*) (#1131)
* refactor: extract shared utils to break harness→app cross-layer imports

Move _validate_skill_frontmatter to src/skills/validation.py and
CONVERTIBLE_EXTENSIONS + convert_file_to_markdown to src/utils/file_conversion.py.
This eliminates the two reverse dependencies from client.py (harness layer)
into gateway/routers/ (app layer), preparing for the harness/app package split.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor: split backend/src into harness (deerflow.*) and app (app.*)

Physically split the monolithic backend/src/ package into two layers:

- **Harness** (`packages/harness/deerflow/`): publishable agent framework
  package with import prefix `deerflow.*`. Contains agents, sandbox, tools,
  models, MCP, skills, config, and all core infrastructure.

- **App** (`app/`): unpublished application code with import prefix `app.*`.
  Contains gateway (FastAPI REST API) and channels (IM integrations).

Key changes:
- Move 13 harness modules to packages/harness/deerflow/ via git mv
- Move gateway + channels to app/ via git mv
- Rename all imports: src.* → deerflow.* (harness) / app.* (app layer)
- Set up uv workspace with deerflow-harness as workspace member
- Update langgraph.json, config.example.yaml, all scripts, Docker files
- Add build-system (hatchling) to harness pyproject.toml
- Add PYTHONPATH=. to gateway startup commands for app.* resolution
- Update ruff.toml with known-first-party for import sorting
- Update all documentation to reflect new directory structure

Boundary rule enforced: harness code never imports from app.
All 429 tests pass. Lint clean.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: add harness→app boundary check test and update docs

Add test_harness_boundary.py that scans all Python files in
packages/harness/deerflow/ and fails if any `from app.*` or
`import app.*` statement is found. This enforces the architectural
rule that the harness layer never depends on the app layer.

Update CLAUDE.md to document the harness/app split architecture,
import conventions, and the boundary enforcement test.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: add config versioning with auto-upgrade on startup

When config.example.yaml schema changes, developers' local config.yaml
files can silently become outdated. This adds a config_version field and
auto-upgrade mechanism so breaking changes (like src.* → deerflow.*
renames) are applied automatically before services start.

- Add config_version: 1 to config.example.yaml
- Add startup version check warning in AppConfig.from_file()
- Add scripts/config-upgrade.sh with migration registry for value replacements
- Add `make config-upgrade` target
- Auto-run config-upgrade in serve.sh and start-daemon.sh before starting services
- Add config error hints in service failure messages

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix comments

* fix: update src.* import in test_sandbox_tools_security to deerflow.*

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: handle empty config and search parent dirs for config.example.yaml

Address Copilot review comments on PR #1131:
- Guard against yaml.safe_load() returning None for empty config files
- Search parent directories for config.example.yaml instead of only
  looking next to config.yaml, fixing detection in common setups

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: correct skills root path depth and config_version type coercion

- loader.py: fix get_skills_root_path() to use 5 parent levels (was 3)
  after harness split, file lives at packages/harness/deerflow/skills/
  so parent×3 resolved to backend/packages/harness/ instead of backend/
- app_config.py: coerce config_version to int() before comparison in
  _check_config_version() to prevent TypeError when YAML stores value
  as string (e.g. config_version: "1")
- tests: add regression tests for both fixes

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: update test imports from src.* to deerflow.*/app.* after harness refactor

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-14 22:55:52 +08:00
YolenSong
9b49a80dda feat(feishu): stream updates on a single card (#1031)
* feat(feishu): stream updates on a single card

* fix(feishu): ensure final message on stream error and warn on missing card ID

- Wrap streaming loop in try/except/finally so a is_final=True outbound
  message is always published, even when the LangGraph stream breaks
  mid-way. This prevents _running_card_ids memory leaks and ensures the
  Feishu card shows a DONE reaction instead of hanging on "Working on it".
- Log a warning when _ensure_running_card gets no message_id back from
  the Feishu reply API, making silent fallback to new-card behavior
  visible in logs.
- Add test_handle_feishu_stream_error_still_sends_final to cover the
  error path.
- Reformat service.py dict comprehension (ruff format, no logic change).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Avoid blocking inbound on Feishu card creation

---------

Co-authored-by: songyaolun <songyaolun@bytedance.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-03-14 22:24:35 +08:00
virtaava
d18a9ae5aa feat: add LoopDetectionMiddleware to break repetitive tool call loops (#1056)
* feat: add LoopDetectionMiddleware to break repetitive tool call loops

Adds a new AgentMiddleware that detects when the agent is stuck calling
the same tools with the same arguments repeatedly, which currently runs
until the recursion limit kills the run.

Detection: per-thread sliding window of tool call hashes (name + args).
- Warn threshold (default 3): injects a "wrap up" system message
- Hard limit (default 5): strips tool_calls, forcing final text output

Includes 13 unit tests covering hashing, thresholds, window sliding,
reset, and edge cases.

Closes #1055

* fix: address PR #1056 review feedback for LoopDetectionMiddleware

- Remove unused imports (Awaitable, Callable, ModelCallResult,
  ModelRequest, ModelResponse, AIMessage) from loop_detection_middleware
- Remove unused pytest import from test file
- Fix _hash_tool_calls sort key: sort by (name, serialized args) for
  deterministic hashing when multiple calls share the same tool name
- Revert subagent_enabled default to False in agent.py to match
  DeerFlowClient and channel defaults
- Remove unrelated SearxNG tools and Next.js rewrite changes from PR

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: address 2nd round review feedback on PR #1056

- Inject loop warning only once per thread (prevents context bloat)
- Add threading.Lock for thread-safe history mutations
- Use runtime.context thread_id instead of workspace_path
- Add LRU eviction for per-thread history (max 100 threads)
- Add 5 new tests covering warn-once, LRU eviction, thread isolation,
  fallback thread_id, and lock presence

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: resolve lint errors in loop detection middleware tests

Sort imports (I001) and remove unused _WARNING_MSG import (F401)
to fix ruff lint failures in CI.

* Apply suggestions from code review

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-03-14 22:17:54 +08:00
Octopus
bbd87df6eb Add MiniMax as an OpenAI-compatible model provider (#1120)
* Add MiniMax as an OpenAI-compatible model provider

MiniMax offers high-performance LLMs (M2.5, M2.5-highspeed) with
204K context windows. This commit adds MiniMax as a selectable
provider in the configuration system.

Changes:
- Add MiniMax to SUPPORTED_MODELS with model definitions
- Add MiniMax provider configuration in conf/config.yaml
- Update documentation with MiniMax setup instructions

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Update README to remove MiniMax API details

Removed mention of MiniMax API usage and configuration examples.

---------

Co-authored-by: octo-patch <octo-patch@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-03-14 22:05:18 +08:00
Matt(허철진)
d197d50146 fix: preserve conversation context in Telegram private chats (#1105)
* fix: preserve conversation context in Telegram private chats

In private (1-on-1) chats, set topic_id=None so all messages map to a
single DeerFlow thread per chat instead of creating a new thread for
every message. Also fix _cmd_generic to use topic_id=None in private
chats so /new correctly targets the default thread.

Group chat behavior is unchanged (reply_to or msg_id as topic_id).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: preserve conversation context in Telegram private chats

Fixes #1101

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: mirror _on_text reply logic in _cmd_generic for group chats

_cmd_generic now prefers reply_to_message.message_id over msg_id in
group/supergroup chats, consistent with _on_text. This ensures commands
like /new and /status target the correct conversation thread when sent
as a reply in group chats.

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: JeffJiang <for-eleven@hotmail.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-03-14 09:47:24 +08:00
Willem Jiang
253fe4d87f feat(sandbox): harden local file access and mask host paths (#983)
* feat(sandbox): harden local file access and mask host paths

- enforce local sandbox file tools to only accept /mnt/user-data paths
- add path traversal checks against thread workspace/uploads/outputs roots
- preserve requested virtual paths in tool error messages (no host path leaks)
- mask local absolute paths in bash output back to virtual sandbox paths
- update bash tool guidance to prefer thread-local venv + python -m pip
- add regression tests for path mapping, masking, and access restrictions

Fixes #968

* feat(sandbox): restrict risky absolute paths in local bash commands

- validate absolute path usage in local-mode bash commands
- allow only /mnt/user-data virtual paths for user data access
- keep a small allowlist for system executable/device paths
- return clear permission errors for unsafe command paths
- add regression tests for bash path validation rules

* test(sandbox): add success path test for resolve_local_tool_path (#992)

* Initial plan

* test(sandbox): add success path test for resolve_local_tool_path

Co-authored-by: WillemJiang <219644+WillemJiang@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: WillemJiang <219644+WillemJiang@users.noreply.github.com>

* fix(sandbox): reject bare virtual root early with clear error in resolve_local_tool_path (#991)

* Initial plan

* fix(sandbox): reject bare virtual root early with clear error in resolve_local_tool_path

Co-authored-by: WillemJiang <219644+WillemJiang@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: WillemJiang <219644+WillemJiang@users.noreply.github.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>

---------

Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
2026-03-13 22:38:32 +08:00
Ryanba
b155923ab0 fix(gateway): ignore archive metadata wrappers (#1108)
* fix(gateway): ignore archive metadata wrappers

Treat top-level __MACOSX and dotfile entries as packaging metadata so valid .skill archives still resolve to their real skill directory.

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-opencode)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>

* Apply suggestions from code review

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
2026-03-13 21:27:54 +08:00
Ryanba
cda9fb7bca fix(gateway): allow standard skill frontmatter metadata (#1103)
* fix(gateway): allow standard skill frontmatter metadata

Accept standard optional frontmatter fields during .skill installs so external skills with version, author, or compatibility metadata do not fail validation.

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-opencode)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>

* docs: sync skill installer metadata behavior

Document the skill install allowlist so user-facing and backend contributor docs match the gateway validation contract.

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-opencode)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>

* Apply suggestions from code review

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
2026-03-13 21:23:35 +08:00
Ryanba
03cafea715 fix(gateway): normalize suggestion response content (#1098)
* fix(gateway): normalize suggestion response content

Handle list-style model content before JSON parsing so provider wrappers do not silently drop follow-up suggestions.

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-opencode)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>

* docs: sync suggestions endpoint behavior

Document the rich-content normalization path so the README and backend gateway notes stay aligned with the current router contract.

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-opencode)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>

* Apply suggestions from code review

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-03-13 21:20:15 +08:00
Willem Jiang
b5fcb1334a fix(memory): inject stored facts into system prompt memory context (#1083)
* fix(memory): inject stored facts into system prompt memory context

- add Facts section rendering in format_memory_for_injection
- rank facts by confidence and coerce confidence values safely
- enforce max token budget while appending fact lines
- add regression tests for fact inclusion, ordering, and budget behavior

Fixes #1059

* Update the document with the latest status

* fix(memory): harden fact injection — NaN/inf confidence, None content, incremental token budget (#1090)

* Initial plan

* fix(memory): address review feedback on confidence coercion, None content, and token budget

Co-authored-by: WillemJiang <219644+WillemJiang@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: WillemJiang <219644+WillemJiang@users.noreply.github.com>

---------

Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
2026-03-13 14:37:40 +08:00
Liu Jice
3521cc2668 fix(middleware): degrade tool-call exceptions to error tool messages (#1110)
* fix(middleware): degrade tool-call exceptions to error tool messages

* update script

* fix(middleware): preserve LangGraph control-flow exceptions in tool error handling
2026-03-13 09:41:59 +08:00
Ryanba
4bae3c724c fix(client): Harden upload validation and conversion flow (#989)
* fix(client): Harden upload validation and conversion flow

* test(client): cover event-loop upload conversion reuse

* test(client): remove unused import in upload regression coverage

* fix(client): load optional shared checkpointer fallback

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-opencode)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>

* docs(backend): document config preflight and IM channels

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-opencode)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
2026-03-11 15:17:31 +08:00
Willem Jiang
96dbee00e3 fix(tracing): support LANGCHAIN_* env fallback for LangSmith config (#1065)
* fix(tracing): support LANGCHAIN_* env fallback for LangSmith config

- add backward-compatible env parsing in tracing_config.py
- support fallback keys:
   LANGCHAIN_TRACING_V2 / LANGCHAIN_TRACING
   LANGCHAIN_API_KEY
   LANGCHAIN_PROJECT
   LANGCHAIN_ENDPOINT
- keep LANGSMITH_* as preferred source when both are present
- add regression tests in test_tracing_config.py

* fix(tracing): correct LANGSMITH_* precedence over LANGCHAIN_* for enabled flag (#1067)

* Initial plan

* fix(tracing): use first-present-wins logic for enabled flag, add precedence docs and test

Co-authored-by: WillemJiang <219644+WillemJiang@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: WillemJiang <219644+WillemJiang@users.noreply.github.com>

---------

Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
2026-03-11 10:26:56 +08:00
DanielWalnut
33f086b612 feat(channels): upload file attachments via IM channels (Slack, Telegram, Feishu) (#1040) 2026-03-10 09:11:57 +08:00
momorebi
0409f8cefd fix(subagents): cleanup background tasks after completion to prevent memory leak (#1030)
* fix(subagents): cleanup background tasks after completion to prevent memory leak

Added cleanup_background_task() function to remove completed subagent results
from the global _background_tasks dict. Found a small issue: completed tasks
were never removed, causing memory to grow indefinitely with each subagent
execution.

Alternative approaches considered:
- Future + SubagentHandle pattern: Not chosen due to requiring refactoring

Chose the simple cleanup approach for minimal code changes while effectively
resolving the memory leak.

Changes:
- Add cleanup_background_task() in executor.py
- Call cleanup in all task_tool return paths (completed, failed, timed out)

* fix(subagents): prevent race condition in background task cleanup

Address Copilot review feedback on memory leak fix:

- Add terminal state check in cleanup_background_task() to only remove
  tasks that are COMPLETED/FAILED/TIMED_OUT or have completed_at set
- Remove cleanup call from polling safety-timeout branch in task_tool
  since the task may still be running
- Add comprehensive tests for cleanup behavior including:
  - Verification that cleanup is called on terminal states
  - Verification that cleanup is NOT called on polling timeout
  - Tests for terminal state check logic in executor

This prevents KeyError when the background executor tries to update
a task that was prematurely removed from _background_tasks.

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-03-10 07:41:48 +08:00
Willem Jiang
46918f0786 Revert "feat(threads): paginate full history via summaries endpoint (#1022)" (#1037)
This reverts commit 2f47f1ced2.
2026-03-09 16:25:08 +08:00
aworki
2f47f1ced2 feat(threads): paginate full history via summaries endpoint (#1022)
* feat(threads): add paginated summaries API and load full history

* fix(threads): address summaries review feedback

- validate summaries sort params and log gateway failures
- page frontend thread summaries without stale query keys or silent truncation
- export router modules and tighten thread list typing

Refs: 2901804166, 2901804176, 2901804179, 2901804180, 2901804183, 2901804187, 2901804191

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-03-09 16:08:02 +08:00
lailoo
959b4f2b09 fix(checkpointer): return InMemorySaver instead of None when not configured (#1016) (#1019)
* fix(checkpointer): return InMemorySaver instead of None when not configured (#1016)

* fix(checkpointer): also fix get_checkpointer() to return InMemorySaver

Make all three checkpointer functions consistent:
- make_checkpointer() (async) → InMemorySaver
- checkpointer_context() (sync) → InMemorySaver
- get_checkpointer() (sync singleton) → InMemorySaver

This ensures DeerFlowClient always has a valid checkpointer.

* fix: address CI failure and Copilot review feedback

- Fix import order in test_checkpointer_none_fix.py (I001 ruff error)
- Fix type annotation: _checkpointer should be Checkpointer | None
- Update docstring: change "None if not configured" to "InMemorySaver if not configured"
- Ensure app config is loaded before checking checkpointer config to prevent incorrect InMemorySaver fallback

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-03-09 15:48:27 +08:00
aworki
ac1e1915ef feat(channels): make mobile session settings configurable by channel and user (#1021) 2026-03-08 22:19:40 +08:00
JeffJiang
cf9af1fe75 Enhance chat UI and compatible anthropic thinking messages (#1018) 2026-03-08 20:19:31 +08:00
JeffJiang
3512279ce3 feat: add thinking settings to compatible anthropic api (#1017) 2026-03-08 20:18:21 +08:00
DanielWalnut
75b7302000 feat: add IM channels for Feishu, Slack, and Telegram (#1010)
* feat: add IM channels system for Feishu, Slack, and Telegram integration

Bridge external messaging platforms to DeerFlow via LangGraph Server with
async message bus, thread management, and per-channel configuration.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: address review comments on IM channels system

Fix topic_id handling in store remove/list_entries and manager commands,
correct Telegram reply threading, remove unused imports/variables, update
docstrings and docs to match implementation, and prevent config mutation.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* update skill creator

* fix im reply text

* fix comments

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-08 15:21:18 +08:00
JeffJiang
d664ae5a4b Support langgraph checkpointer (#1005)
* Add checkpointer configuration to config.example.yaml

- Introduced a new section for checkpointer configuration to enable state persistence for the embedded DeerFlowClient.
- Documented supported types: memory, sqlite, and postgres, along with examples for each.
- Clarified that the LangGraph Server manages its own state persistence separately.

* refactor(checkpointer): streamline checkpointer initialization and logging

* fix(uv.lock): update revision and add new wheel URLs for brotlicffi package

* feat: add langchain-anthropic dependency and update related configurations

* Fix checkpointer lifecycle, docstring, and path resolution bugs from PR #1005 review (#4)

* Initial plan

* Address all review suggestions from PR #1005

Co-authored-by: foreleven <4785594+foreleven@users.noreply.github.com>

* Fix resolve_path to always return real Path; move SQLite special-string handling to callers

Co-authored-by: foreleven <4785594+foreleven@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: foreleven <4785594+foreleven@users.noreply.github.com>

---------

Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
Co-authored-by: foreleven <4785594+foreleven@users.noreply.github.com>
2026-03-07 21:07:21 +08:00
Xinmin Zeng
09325ca28f fix: normalize presented artifact paths (#998)
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-03-06 22:51:27 +08:00
null4536251
9d2144d431 feat: may_ask (#981)
* feat: u may ask

* chore: adjust code according to CR

* chore: adjust code according to CR

* ut: test for suggestions.py

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-03-06 22:39:58 +08:00
infoquest-byteplus
28e1257e1e support infoquest (#960)
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-03-06 15:32:13 +08:00
FangHao
3e4a24f48b fix(subagent): support async MCP tools in subagent executor (#917)
* fix(subagent): support async MCP tools in subagent executor

SubagentExecutor.execute() was synchronous and could not handle async-only                                                                                                                                  tools like MCP tools. This caused failures when trying to use MCP tools within subagents.

Changes:
- Add _aexecute() async method using agent.astream() for async execution
- Refactor execute() to use asyncio.run() wrapping _aexecute()
- This allows subagents to use async tools (MCP) within ThreadPoolExecutor

* test(subagent): add unit tests for executor async/sync paths

Add comprehensive tests covering:
- Async _aexecute() with success/error cases
- Sync execute() wrapper using asyncio.run()
- Async tool (MCP) support verification
- Thread pool execution safety

* fix(subagent): subagent-test-circular-depend

- Use session-scoped fixture with delayed import to handle circular dependencies
    without affecting other test modules

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-03-06 14:40:56 +08:00
Willem Jiang
3a5e0b935d fix(backend): upgrade langgraph-api to 0.7 and stabilize memory path tests (#984)
- replace  with explicit runtime deps:
- regenerate  after dependency changes
- make  deterministic by patching
  to avoid leaked global  affecting expected paths
2026-03-06 09:44:40 +08:00
JeffJiang
1b3939cb78 fix(chat): handle empty uploaded files case and improve artifact selection logic (#979)
* fix(chat): handle empty uploaded files case and improve artifact selection logic

* Update frontend/src/components/workspace/chats/chat-box.tsx

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* fix: address code review suggestions from PR #979 (#3)

* Initial plan

* fix: address PR #979 review suggestions

- utils.ts: scope (empty) check inside <uploaded_files> tag content
- chat-box.tsx: remove stale `artifacts` from useEffect deps
- context.tsx: wrap select/deselect with useCallback for stable refs
- test: add test_empty_new_files_produces_empty_marker

Co-authored-by: foreleven <4785594+foreleven@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: foreleven <4785594+foreleven@users.noreply.github.com>

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
Co-authored-by: foreleven <4785594+foreleven@users.noreply.github.com>
2026-03-05 17:45:25 +08:00
JeffJiang
b17c087174 Implement optimistic UI for file uploads and enhance message handling (#967)
* feat(upload): implement optimistic UI for file uploads and enhance message handling

* feat(middleware): enhance file handling by collecting historical uploads from directory

* feat(thread-title): update page title handling for new threads and improve loading state

* feat(uploads-middleware): enhance file extraction by verifying file existence in uploads directory

* feat(thread-stream): update file path reference to use virtual_path for uploads

* feat(tests): add core behaviour tests for UploadsMiddleware

* feat(tests): remove unused pytest import from test_uploads_middleware_core_logic.py

* feat: enhance file upload handling and localization support

- Update UploadsMiddleware to validate filenames more robustly.
- Modify MessageListItem to parse uploaded files from raw content for backward compatibility.
- Add localization for uploading messages in English and Chinese.
- Introduce parseUploadedFiles utility to extract uploaded files from message content.
2026-03-05 11:16:34 +08:00
DanielWalnut
3ada4f98b1 fix(memory): prevent file upload events from persisting in long-term memory (#971)
* fix(memory): prevent file upload events from persisting in long-term memory

Uploaded files are session-scoped and unavailable in future sessions.
Previously, upload interactions were recorded in memory, causing the
agent to search for non-existent files in subsequent conversations.

Changes:
- memory_middleware: skip human messages containing <uploaded_files>
  and their paired AI responses from the memory queue
- updater: post-process generated memory to strip upload mentions
  before saving to file
- prompt: instruct the memory LLM to ignore file upload events

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(memory): address Copilot review feedback on upload filtering

- memory_middleware: strip <uploaded_files> block from human messages
  instead of dropping the entire turn; only skip the turn (and paired
  AI response) when nothing remains after stripping
- updater: narrow the upload-scrubbing regex to explicit upload events
  (avoids false-positive removal of "User works with CSV files" etc.);
  also filter upload-event facts from the facts array
- prompt: move `import re` to module scope; skip upload-only human
  messages (empty after stripping) rather than appending "User: "

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(memory): allow optional words between 'upload' and 'file' in scrub regex

The previous pattern required 'uploading file' with no intervening words,
so 'uploading a test file' was not matched and leaked into long-term memory.
Allow up to 3 modifier words between the verb and noun (e.g. 'uploading a
test file', 'uploaded the attachment').

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* test(memory): add unit tests for upload filtering in memory pipeline

Covers _filter_messages_for_memory and _strip_upload_mentions_from_memory
per Copilot review suggestion.  15 test cases verify:

- Upload-only turns (and paired AI responses) are excluded from memory queue
- User's real question is preserved when combined with an upload block
- Upload file paths are never present in filtered message content
- Intermediate tool messages are always excluded
- Multi-turn conversations: only the upload turn is dropped
- Multimodal (list-content) human messages are handled
- Upload-event sentences are removed from summaries and facts
- Legitimate file-related facts (CSV preferences, PDF exports) are preserved
- "uploading a test file" (words between verb and noun) is caught by regex

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-03-05 11:14:34 +08:00
JeffJiang
7de94394d4 feat(agent):Supports custom agent and chat experience with refactoring (#957)
* feat: add agent management functionality with creation, editing, and deletion

* feat: enhance agent creation and chat experience

- Added AgentWelcome component to display agent description on new thread creation.
- Improved agent name validation with availability check during agent creation.
- Updated NewAgentPage to handle agent creation flow more effectively, including enhanced error handling and user feedback.
- Refactored chat components to streamline message handling and improve user experience.
- Introduced new bootstrap skill for personalized onboarding conversations, including detailed conversation phases and a structured SOUL.md template.
- Updated localization files to reflect new features and error messages.
- General code cleanup and optimizations across various components and hooks.

* Refactor workspace layout and agent management components

- Updated WorkspaceLayout to use useLayoutEffect for sidebar state initialization.
- Removed unused AgentFormDialog and related edit functionality from AgentCard.
- Introduced ArtifactTrigger component to manage artifact visibility.
- Enhanced ChatBox to handle artifact selection and display.
- Improved message list rendering logic to avoid loading states.
- Updated localization files to remove deprecated keys and add new translations.
- Refined hooks for local settings and thread management to improve performance and clarity.
- Added temporal awareness guidelines to deep research skill documentation.

* feat: refactor chat components and introduce thread management hooks

* feat: improve artifact file detail preview logic and clean up console logs

* feat: refactor lead agent creation logic and improve logging details

* feat: validate agent name format and enhance error handling in agent setup

* feat: simplify thread search query by removing unnecessary metadata

* feat: update query key in useDeleteThread and useRenameThread for consistency

* feat: add isMock parameter to thread and artifact handling for improved testing

* fix: reorder import of setup_agent for consistency in builtins module

* feat: append mock parameter to thread links in CaseStudySection for testing purposes

* fix: update load_agent_soul calls to use cfg.name for improved clarity

* fix: update date format in apply_prompt_template for consistency

* feat: integrate isMock parameter into artifact content loading for enhanced testing

* docs: add license section to SKILL.md for clarity and attribution

* feat(agent): enhance model resolution and agent configuration handling

* chore: remove unused import of _resolve_model_name from agents

* feat(agent): remove unused field

* fix(agent): set default value for requested_model_name in _resolve_model_name function

* feat(agent): update get_available_tools call to handle optional agent_config and improve middleware function signature

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-03-03 21:32:01 +08:00
Xinmin Zeng
8342e88534 fix(models): handle google provider import errors and add dependency (#952)
* fix(models): improve provider import guidance and add google provider dep

* Apply suggestions from code review

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* fix(reflection): prefer provider install hint on transitive import errors

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-03-03 14:56:54 +08:00
Xinmin Zeng
7754c49217 feat(skills): support recursive nested skill loading (#950)
* feat(skills): support recursive nested skill loading

* Apply suggestions from code review

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-03-02 21:02:03 +08:00
Zhiyunyao
a138d5388a feat: add reasoning_effort configuration support for Doubao/GPT-5 models (#947)
* feat: Add reasoning effort configuration support

* Add `reasoning_effort` parameter to model config and agent initialization
* Support reasoning effort levels (minimal/low/medium/high) for Doubao/GPT-5 models
* Add UI controls in input box for reasoning effort selection
* Update doubao-seed-1.8 example config with reasoning effort support

Fixes & Cleanup:
* Ensure UTF-8 encoding for file operations
* Remove unused imports

* fix: set reasoning_effort to None for unsupported models

* fix: unit test error

* Update frontend/src/components/workspace/input-box.tsx

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-03-02 20:49:41 +08:00
Willem Jiang
a2f91c7594 feat(mcp): add OAuth support for HTTP/SSE MCP servers (#908)
add oauth schema to MCP server config (extensions_config.json)
support client_credentials and refresh_token grants
implement token manager with caching and pre-expiry refresh
inject OAuth Authorization header for MCP tool discovery and tool calls
extend MCP gateway config models to read/write OAuth settings
update docs and examples for OAuth configuration
add unit tests for token fetch/cache and header injection
2026-03-01 22:38:58 +08:00
エイカク
80316c131e fix(backend): Fix readability extraction crash when Node parser fails (#937)
* Fix readability fallback when Node extraction fails

* Narrow readability fallback errors and enrich logs

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-03-01 22:24:02 +08:00
Willem Jiang
8c6dd9e264 fix(uploads): persist thread uploads canonically and fail fast on upload errors (#943)
* fix(uploads): persist thread uploads canonically and fail fast on upload errors

 - write uploads to thread-scoped storage first to guarantee agent visibility
 - sync files to sandbox virtual path only for non-local sandboxes
 - fix markdown conversion flow to operate on canonical saved files and sync converted files when needed
 - prevent silent attachment upload failures in frontend submit flow (show error + abort submit)
 - add regression tests for local vs non-local upload behavior
 - update upload docs with thread-first persistence and troubleshooting notes

* Update frontend/src/core/threads/hooks.ts

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* fix(uploads): reject "." and ".." filenames in upload sanitization (#944)

* Initial plan

* fix(uploads): reject '.' and '..' filenames in upload sanitization

Co-authored-by: WillemJiang <219644+WillemJiang@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: WillemJiang <219644+WillemJiang@users.noreply.github.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
2026-03-01 15:35:30 +08:00
YolenSong
3d3ea84a57 test(backend): add core logic unit tests for task/title/mcp (#936)
* test(backend): add core logic unit tests for task/title/mcp

* test(backend): fix lint issues in client test modules

---------

Co-authored-by: songyaolun <songyaolun@bytedance.com>
2026-03-01 12:36:09 +08:00
greatmengqi
30d948711f test: add Gateway conformance tests for DeerFlowClient (#931)
Validate that all dict-returning client methods conform to Gateway
Pydantic response models (ModelsListResponse, ModelResponse,
SkillsListResponse, SkillResponse, SkillInstallResponse,
McpConfigResponse, UploadResponse, MemoryConfigResponse,
MemoryStatusResponse). Pydantic ValidationError in CI catches
schema drift between client and Gateway with zero production coupling.

Also includes prior review fixes: enhanced client methods, expanded
unit tests (67→77), live integration test improvements, and updated
documentation.

Co-authored-by: greatmengqi <chenmengqi.0376@bytedance.com>
2026-02-28 16:08:04 +08:00
greatmengqi
9d48c42a20 feat: add DeerFlowClient for embedded programmatic access (#926)
Add `DeerFlowClient` class that provides direct in-process access to
DeerFlow's agent and Gateway capabilities without requiring LangGraph
Server or Gateway API processes. This enables users to import and use
DeerFlow as a Python library.

Co-authored-by: greatmengqi <chenmengqi.0376@bytedance.com>
2026-02-28 14:38:15 +08:00
Xinmin Zeng
6a55860a15 fix: recover from stale model context when configured models change (#898)
* fix: recover from stale model context after config model changes

* fix: fail fast on missing model config and expand model resolution tests

* fix: remove duplicate get_app_config imports

* fix: align model resolution tests with runtime imports

* Apply suggestions from code review

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* fix: remove duplicate model resolution test case

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-02-26 13:54:29 +08:00
DanielWalnut
faa422072c feat(subagents): make subagent timeout configurable via config.yaml (#897)
* feat(subagents): make subagent timeout configurable via config.yaml

- Add SubagentsAppConfig supporting global and per-agent timeout_seconds
- Load subagents config section in AppConfig.from_file()
- Registry now applies config.yaml overrides without mutating builtin defaults
- Polling safety-net in task_tool is now dynamic (execution timeout + 60s buffer)
- Document subagents section in config.example.yaml
- Add make test command and enforce TDD policy in CLAUDE.md
- Add 38 unit tests covering config validation, timeout resolution, registry
  override behavior, and polling timeout formula

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* feat(subagents): add logging for subagent timeout config and execution

- Log loaded timeout config (global default + per-agent overrides) on startup
- Log debug message in registry when config.yaml overrides a builtin timeout
- Include timeout in executor's async execution start log
- Log effective timeout and polling limit when a task is dispatched
- Fix UnboundLocalError: move max_poll_count assignment before logger.info

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* ci(backend): add lint step and run all unit tests via Makefile

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix lint

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-25 08:39:29 +08:00