Commit Graph

50 Commits

Author SHA1 Message Date
JeffJiang
adfe5c4b44 Enforces config env var checks and improves startup handling (#892)
* Enforces config env var checks and improves startup handling

Ensures critical environment variables are validated during config resolution,
raising clear errors if missing. Improves server startup reliability by
verifying that backend services are listening and by terminating on
misconfiguration at launch. Adds more robust feedback to developers when
API startup fails, reducing silent misconfigurations and speeding up
troubleshooting.

* Initial plan

* Implement suggestions from PR #892: fix env var checks and improve error logging

Co-authored-by: foreleven <4785594+foreleven@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: foreleven <4785594+foreleven@users.noreply.github.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-02-25 16:12:59 +08:00
DanielWalnut
faa422072c feat(subagents): make subagent timeout configurable via config.yaml (#897)
* feat(subagents): make subagent timeout configurable via config.yaml

- Add SubagentsAppConfig supporting global and per-agent timeout_seconds
- Load subagents config section in AppConfig.from_file()
- Registry now applies config.yaml overrides without mutating builtin defaults
- Polling safety-net in task_tool is now dynamic (execution timeout + 60s buffer)
- Document subagents section in config.example.yaml
- Add make test command and enforce TDD policy in CLAUDE.md
- Add 38 unit tests covering config validation, timeout resolution, registry
  override behavior, and polling timeout formula

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* feat(subagents): add logging for subagent timeout config and execution

- Log loaded timeout config (global default + per-agent overrides) on startup
- Log debug message in registry when config.yaml overrides a builtin timeout
- Include timeout in executor's async execution start log
- Log effective timeout and polling limit when a task is dispatched
- Fix UnboundLocalError: move max_poll_count assignment before logger.info

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* ci(backend): add lint step and run all unit tests via Makefile

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix lint

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-25 08:39:29 +08:00
CHANGXUBO
85af540076 feat: add LangSmith tracing integration (#878)
* feat: add LangSmith tracing integration

Add optional LangSmith tracing support that can be enabled via environment
variables (LANGSMITH_TRACING, LANGSMITH_API_KEY, LANGSMITH_PROJECT,
LANGSMITH_ENDPOINT). When enabled, a LangChainTracer callback is attached
to chat models and run metadata is injected for trace tagging.

Co-Authored-By: Claude <noreply@anthropic.com>

* Update backend/src/config/tracing_config.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update backend/src/agents/lead_agent/agent.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update backend/src/agents/lead_agent/agent.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update backend/src/models/factory.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Add threading lock to ensure thread-safe access to tracing configuration

---------

Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-02-21 16:41:34 +08:00
JeffJiang
300e5a519a Adds Kubernetes sandbox provisioner support (#35)
* Adds Kubernetes sandbox provisioner support

* Improves Docker dev setup by standardizing host paths

Replaces hardcoded host paths with a configurable root directory,
making the development environment more portable and easier to use
across different machines. Automatically sets the root path if not
already defined, reducing manual setup steps.
2026-02-12 11:02:09 +08:00
hetao
770d92fe36 feat: make max concurrent subagents configurable via runtime config
Support configuring max_concurrent_subagents (2-4, default 3) through
config.configurable, with automatic clamping and dynamic prompt generation.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-11 11:51:11 +08:00
hetao
4a85c5de7b feat: enable skills support for subagents
Extract get_skills_prompt_section() from apply_prompt_template() so
subagents can also receive the available skills list in their system
prompt. This allows subagents to discover and load skills via read_file,
just like the lead agent.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-11 11:04:50 +08:00
LofiSu
6109216d54 fix: citations prompt 2026-02-10 00:18:33 +08:00
LofiSu
1af14bf7e4 Merge branch 'experimental' of github.com:hetaoBackend/deer-flow into feat/citations 2026-02-09 22:05:44 +08:00
JeffJiang
b6da3a219e Add Kubernetes-based sandbox provider for multi-instance support (#19)
* feat: adds docker-based dev environment

* docs: updates Docker command help

* fix local dev

* feat(sandbox): add Kubernetes-based sandbox provider for multi-instance support

* fix: skills path in k8s

* feat: add example config for k8s sandbox

* fix: docker config

* fix: load skills on docker dev

* feat: support sandbox execution to Kubernetes Deployment model

* chore: rename web service name
2026-02-09 21:59:13 +08:00
LofiSu
2f50e5d969 feat(citations): inline citation links with [citation:Title](URL)
- Backend: add citation format to lead_agent and general_purpose prompts
- Add CitationLink component (Badge + HoverCard) for citation cards
- MarkdownContent: detect citation: prefix in link text, render CitationLink
- Message/artifact/subtask: use MarkdownContent or Streamdown with CitationLink
- message-list-item: pass img via components prop (remove isHuman/img)
- message-group, subtask-card: drop unused imports; fix import order (lint)

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-02-09 21:40:20 +08:00
LofiSu
6a540d8408 Merge upstream/experimental: resolve conflict in lead_agent/prompt.py
- Keep upstream subagent HARD LIMIT (max 3 task calls, batching) in subagent_reminder
- Keep our removal of Citations: do not add back 'Citations when synthesizing'

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-02-09 16:27:36 +08:00
LofiSu
46048c76ce chore: 移除所有 Citations 相关逻辑,为后续重构做准备
- Backend: 删除 lead_agent / general_purpose 中的 citations_format 与引用相关 reminder;artifacts 下载不再对 markdown 做 citation 清洗,统一走 FileResponse,保留 Response 用于二进制 inline
- Frontend: 删除 core/citations 模块、inline-citation、safe-citation-content;新增 MarkdownContent 仅做 Markdown 渲染;消息/artifact 预览与复制均使用原始 content
- i18n: 移除 citations 命名空间(loadingCitations、loadingCitationsWithCount)
- 技能与 demo: 措辞改为 references,demo 数据去掉 <citations> 块
- 文档: 更新 CLAUDE/AGENTS/README 描述,新增按文件 diff 的代码变更总结

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-02-09 16:24:01 +08:00
hetao
f68b3c26c3 feat: enforce subagent concurrency limit of 3 per turn with batch execution
Strengthen the SUBAGENT_SECTION prompt to prevent the model from launching
more than 3 subagents in a single response. When >3 sub-tasks are needed,
the model is now explicitly instructed to plan and execute in sequential
batches of ≤3. Reinforced at three prompt injection points: thinking style,
main subagent instructions, and critical reminders.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-09 13:50:54 +08:00
hetao
caf12da0f2 feat: add DanglingToolCallMiddleware and SubagentLimitMiddleware
Add two new middlewares to improve robustness of the agent pipeline:
- DanglingToolCallMiddleware injects placeholder ToolMessages for
  interrupted tool calls, preventing LLM errors from malformed history
- SubagentLimitMiddleware truncates excess parallel task tool calls at
  the model response level, replacing the runtime check in task_tool

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-09 13:22:49 +08:00
LofiSu
2a39947830 feat: citations prompts, path_utils, and citation code cleanup
- Prompt: add citation reminders for web_search and subagent synthesis (lead_agent, general_purpose)
- Gateway: add path_utils for shared thread virtual path resolution; refactor artifacts and skills to use it
- Citations: simplify removeAllCitations (single parse); backend _extract_citation_urls and remove_citations_block cleanup

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-02-09 12:55:12 +08:00
hetaoBackend
808e028338 feat: limit concurrent subagents to 3 per turn
Prevent resource exhaustion by capping the number of parallel subagents.
Adds runtime enforcement in task_tool and updates prompts/examples accordingly.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-08 22:12:21 +08:00
hetaoBackend
f41d9b3be5 refactor: optimize task tool parameter order and improve task tracking
- Reorder task tool parameters to prioritize description first for better usability
- Add tool_call_id injection for better task traceability
- Use tool_call_id as task_id in executor for consistent tracking
- Simplify event messages by removing redundant task_type field
- Update task examples in prompt to reflect new parameter order

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-07 16:56:13 +08:00
LofiSu
f0075e0d64 Merge upstream/experimental into feat/citations
Resolved conflicts:
- backend/src/gateway/routers/artifacts.py: Keep citations block removal for markdown downloads
- frontend/src/components/workspace/messages/message-list-item.tsx: Keep improved citation handling with rehypePlugins, humanMessagePlugins, and CitationsLoadingIndicator

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-02-07 00:53:16 +08:00
hetao
9e4f2512f3 fix: fix subagent prompt 2026-02-06 20:32:15 +08:00
hetao
9f367b5563 feat: fix task polling issue 2026-02-06 17:48:15 +08:00
hetao
449ffbad75 feat: add ultra mode 2026-02-06 17:48:14 +08:00
LofiSu
a91302ac72 fix(prompt): clarify citation link format must include URL
AI was outputting bare brackets like [arXiv:xxx] without URLs,
which do not render as links. Updated prompt to explicitly show
correct vs wrong formats and require complete markdown links.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-02-06 14:38:31 +08:00
hetao
85128f5f14 feat: add configuration to enable/disable subagents
Add subagents.enabled flag in config.yaml to control subagent feature:
- When disabled, task/task_status tools are not loaded
- When disabled, system prompt excludes subagent documentation
- Default is enabled for backward compatibility

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-05 20:49:02 +08:00
hetao
ef379a3100 feat: support sub agent mechanism 2026-02-05 19:59:25 +08:00
LofiSu
2debcf421c fix(citations): improve citation link rendering and copy behavior
- Use citation.title for display text in CitationLink to ensure correct
  titles show during streaming (instead of generic "Source" text)
- Render all external links as CitationLink badges for consistent styling
  during streaming output
- Add removeAllCitations when copying message content to clipboard
- Simplify citations_format prompt for cleaner AI output

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-02-04 16:34:12 +08:00
hetaoBackend
0ea666e0cf feat: add global memory mechanism for personalized conversations
Implement a memory system that stores user context and conversation history
in memory.json, uses LLM to summarize conversations, and injects relevant
context into system prompts for personalized responses.

Key components:
- MemoryConfig for configuration management
- MemoryUpdateQueue with debounce for batch processing
- MemoryUpdater for LLM-based memory extraction
- MemoryMiddleware to queue conversations after agent execution
- Memory injection into lead agent system prompt

Note: Add memory section to config.yaml to enable (see config.example.yaml)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-03 13:31:05 +08:00
hetaoBackend
1899333b95 refactor: update skills XML format in prompt template
Change skills rendering from attribute-based format to nested element format
with <available_skills>, <skill>, <name>, <description>, and <location> tags
for better readability and structure.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-31 21:54:29 +08:00
hetao
2c7a56dd33 feat: optimize vision tools and image handling
- Add model-aware vision tool loading based on supports_vision flag
- Move view_image_tool from config to builtin tools for dynamic inclusion
- Add timeout to image search to prevent hanging requests
- Optimize image search results format using thumbnails
- Add image validation for reference images in generation
- Improve error handling with detailed messages

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-01-29 14:57:26 +08:00
hetao
09d9c18a28 feat: add view_image tool and optimize web fetch tools
Add image viewing capability for vision-enabled models with ViewImageMiddleware and view_image_tool. Limit web_fetch tool output to 4096 characters to prevent excessive content. Update model config to support vision capability flag.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-01-29 13:44:39 +08:00
LofiSu
588673d043 merge: upstream/experimental with citations feature
- Merge upstream changes including image search, tooltips, and UI improvements
- Keep citations feature with inline hover cards
- Resolve conflict in message-list-item.tsx: use upstream img max-width (90%) while preserving citations logic
- Maintain file upload improvements with citations support

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-01-29 12:55:43 +08:00
Henry Li
eff241f9f2 feat: enable images in content 2026-01-29 12:51:04 +08:00
LofiSu
c14378a312 feat: refine citations format and improve content presentation
Backend:
- Simplify citations prompt format and rules
- Add clear distinction between chat responses and file content
- Enforce full URL usage in markdown links, prohibit [cite-1] format
- Require content-first approach: write full content, then add citations at end

Frontend:
- Hide <citations> block in both chat messages and markdown preview
- Remove top-level Citations/Sources list for cleaner UI
- Auto-remove <citations> block in code editor view for markdown files
- Keep inline citation hover cards for reference details

This ensures citations are presented like Claude: clean content with inline reference badges.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-01-29 12:29:13 +08:00
LofiSu
ad85b72064 feat: add inline citations and thread management features
Citations:
- Add citations parsing utilities for extracting source references from AI responses
- Render inline citations as hover card badges in message content
- Display citation cards with title, URL, and description on hover
- Add citation badge rendering in artifact markdown preview
- Update prompt to guide AI to output citations in correct format

Thread Management:
- Add rename functionality for chat threads with dialog UI
- Add share functionality to copy thread link to clipboard
- Share links use Vercel URL for production accessibility
- Add useRenameThread hook for thread title updates

i18n:
- Add translations for rename, share, cancel, save, and linkCopied

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-01-28 19:15:11 +08:00
hetao
f6a20a69e3 feat: implement file upload feature 2026-01-23 18:47:39 +08:00
hetaoBackend
3191a3845f feat: integrate todo middleware 2026-01-20 22:38:04 +08:00
hetaoBackend
5888a5ba16 fix: fix skill md path 2026-01-20 21:10:05 +08:00
hetaoBackend
50810c8212 feat: add skills api 2026-01-20 13:57:36 +08:00
DanielWalnut
9a3eaea54e feat: implement summarization (#14) 2026-01-19 16:17:31 +08:00
DanielWalnut
645923c3bc feat: add clarification feature (#13) 2026-01-18 19:55:36 +08:00
DanielWalnut
c50540e3fc fix: Long thinking but with empty content (#12) 2026-01-18 14:21:40 +08:00
DanielWalnut
9f755ecc30 feat: add skills system for specialized agent workflows (#6)
Implement a skills framework that enables specialized workflows for
specific tasks (e.g., PDF processing, web page generation). Skills are
discovered from the skills/ directory and automatically mounted in
sandboxes with path mapping support.

- Add SkillsConfig for configuring skills path and container mount point
- Implement dynamic skill loading from SKILL.md files with YAML frontmatter
- Add path mapping in LocalSandbox to translate container paths to local paths
- Mount skills directory in AIO Docker sandbox containers
- Update lead agent prompt to dynamically inject available skills
- Add setup documentation and expand config.example.yaml

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-16 14:44:51 +08:00
DanielWalnut
7284eb15f1 feat: add gateway module with FastAPI server (#5)
* chore: add .claude/ to .gitignore

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat: add gateway module with FastAPI server

- Add new gateway module with FastAPI app for API routing
- Add gateway and serve commands to Makefile
- Add fastapi, httpx, uvicorn, sse-starlette dependencies
- Fix model config retrieval in lead_agent (support both model_name and model)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-16 13:22:26 +08:00
DanielWalnut
b44144dd2c feat: support function factory (#4) 2026-01-15 22:05:54 +08:00
DanielWalnut
a39f799a7e fix: fix local path for local sandbox (#3) 2026-01-15 14:37:00 +08:00
DanielWalnut
c92eedc572 feat: add thread data middleware (#2) 2026-01-15 13:22:30 +08:00
DanielWalnut
ab427731dc feat: add AIO sandbox provider and auto title generation (#1)
- Add AioSandboxProvider for Docker-based sandbox execution with
  configurable container lifecycle, volume mounts, and port management
- Add TitleMiddleware to auto-generate thread titles after first
  user-assistant exchange using LLM
- Add Claude Code documentation (CLAUDE.md, AGENTS.md)
- Extend SandboxConfig with Docker-specific options (image, port, mounts)
- Fix hardcoded mount path to use expanduser
- Add agent-sandbox and dotenv dependencies

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-14 23:29:18 +08:00
Henry Li
de2d18561a feat: integrated with sandbox 2026-01-14 12:32:34 +08:00
Henry Li
e5c69cb7ee docs: update tool docs 2026-01-14 09:12:03 +08:00
Henry Li
cb611f9270 chore: use ruff to lint and auto-format 2026-01-14 09:08:20 +08:00
Henry Li
7dc063ba25 feat: add agents 2026-01-14 07:20:00 +08:00