Willem Jiang 975b344ca7 fix: resolve issue #651 - crawl error with None content handling (#652) ...

* fix: resolve issue #651 - crawl error with None content handling
Fixed issue #651 by adding comprehensive null-safety checks and error handling to the crawl system.
The fix prevents the ‘TypeError: Incoming markup is of an invalid type: None’ crash by:
1. Validating HTTP responses from Jina API
2. Handling None/empty content at extraction stage
3. Adding fallback handling in Article markdown/message conversion
4. Improving error diagnostics with detailed logging
5. Adding 16 new tests with 100% coverage for critical paths

* Update src/crawler/readability_extractor.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update src/crawler/article.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

2025-10-24 17:06:54 +08:00

..

test_crawl.py

fix: resolve issue #651 - crawl error with None content handling (#652)

2025-10-24 17:06:54 +08:00

test_decorators.py

feat: 1. replace black with ruff for fomatting and sort import (#489)

2025-08-17 22:57:23 +08:00

test_python_repl.py

feat: 1. replace black with ruff for fomatting and sort import (#489)

2025-08-17 22:57:23 +08:00

test_search_postprocessor.py

feat: Add intelligent clarification feature in coordinate step for research queries (#613)

2025-10-14 13:35:57 +08:00

test_search.py

Fix: clarification bugs - max rounds, locale passing, and over-clarification (#647)

2025-10-24 16:43:39 +08:00

test_tavily_search_api_wrapper.py

fix: correct image result format for OpenAI compatibility to fix #632 (#634)

2025-10-20 23:14:09 +08:00

test_tavily_search_results_with_images.py

fix: resolve issue #467 - message content validation and Tavily search error handling (#645)

2025-10-23 22:08:14 +08:00

test_tools_retriever.py

feat: 1. replace black with ruff for fomatting and sort import (#489)

2025-08-17 22:57:23 +08:00