mirror of
https://gitee.com/wanwujie/deer-flow
synced 2026-04-27 07:44:48 +08:00
* fix: resolve issue #651 - crawl error with None content handling Fixed issue #651 by adding comprehensive null-safety checks and error handling to the crawl system. The fix prevents the ‘TypeError: Incoming markup is of an invalid type: None’ crash by: 1. Validating HTTP responses from Jina API 2. Handling None/empty content at extraction stage 3. Adding fallback handling in Article markdown/message conversion 4. Improving error diagnostics with detailed logging 5. Adding 16 new tests with 100% coverage for critical paths * Update src/crawler/readability_extractor.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update src/crawler/article.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
This commit is contained in:
@@ -23,4 +23,11 @@ class JinaClient:
|
||||
)
|
||||
data = {"url": url}
|
||||
response = requests.post("https://r.jina.ai/", headers=headers, json=data)
|
||||
|
||||
if response.status_code != 200:
|
||||
raise ValueError(f"Jina API returned status {response.status_code}: {response.text}")
|
||||
|
||||
if not response.text or not response.text.strip():
|
||||
raise ValueError("Jina API returned empty response")
|
||||
|
||||
return response.text
|
||||
|
||||
Reference in New Issue
Block a user