feat: Implement Milvus retriver for RAG (#516)

* feat: Implement MilvusRetriever with embedding model and resource management

* chore: Update configuration and loader files for consistency

* chore: Clean up test_milvus.py for improved readability and organization

* feat: Add tests for DashscopeEmbeddings query and document embedding methods

* feat: Add tests for embedding model initialization and example file loading in MilvusProvider

* chore: Remove unused imports and clean up test_milvus.py for better readability

* chore: Clean up test_milvus.py for improved readability and organization

* chore: Clean up test_milvus.py for improved readability and organization

* fix: replace print statements with logging in recursion limit function

* Implement feature X to enhance user experience and optimize performance

* refactor: clean up unused imports and comments in AboutTab component

* Implement feature X to enhance user experience and fix bug Y in module Z

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
This commit is contained in:
CHANGXUBO
2025-09-12 22:20:55 +08:00
committed by GitHub
parent eec8e4dd60
commit dd9af1eb50
15 changed files with 1875 additions and 43 deletions

View File

@@ -179,4 +179,40 @@ SEARCH_ENGINE:
exclude_domains:
- unreliable-site.com
- spam-domain.net
```
## RAG (Retrieval-Augmented Generation) Configuration
DeerFlow supports multiple RAG providers for document retrieval. Configure the RAG provider by setting environment variables.
### Supported RAG Providers
- **RAGFlow**: Document retrieval using RAGFlow API
- **VikingDB Knowledge Base**: ByteDance's VikingDB knowledge base service
- **Milvus**: Open-source vector database for similarity search
### Milvus Configuration
To use Milvus as your RAG provider, set the following environment variables:
```bash
# RAG_PROVIDER: milvus (using free milvus instance on zilliz cloud: https://docs.zilliz.com/docs/quick-start )
RAG_PROVIDER=milvus
MILVUS_URI=<endpoint_of_self_hosted_milvus_or_zilliz_cloud>
MILVUS_USER=<username_of_self_hosted_milvus_or_zilliz_cloud>
MILVUS_PASSWORD=<password_of_self_hosted_milvus_or_zilliz_cloud>
MILVUS_COLLECTION=documents
MILVUS_EMBEDDING_PROVIDER=openai
MILVUS_EMBEDDING_BASE_URL=
MILVUS_EMBEDDING_MODEL=
MILVUS_EMBEDDING_API_KEY=
# RAG_PROVIDER: milvus (using milvus lite on Mac or Linux)
RAG_PROVIDER=milvus
MILVUS_URI=./milvus_demo.db
MILVUS_COLLECTION=documents
MILVUS_EMBEDDING_PROVIDER=openai
MILVUS_EMBEDDING_BASE_URL=
MILVUS_EMBEDDING_MODEL=
MILVUS_EMBEDDING_API_KEY=
```