fix(memory): prevent file upload events from persisting in long-term memory (#971)

* fix(memory): prevent file upload events from persisting in long-term memory

Uploaded files are session-scoped and unavailable in future sessions.
Previously, upload interactions were recorded in memory, causing the
agent to search for non-existent files in subsequent conversations.

Changes:
- memory_middleware: skip human messages containing <uploaded_files>
  and their paired AI responses from the memory queue
- updater: post-process generated memory to strip upload mentions
  before saving to file
- prompt: instruct the memory LLM to ignore file upload events

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(memory): address Copilot review feedback on upload filtering

- memory_middleware: strip <uploaded_files> block from human messages
  instead of dropping the entire turn; only skip the turn (and paired
  AI response) when nothing remains after stripping
- updater: narrow the upload-scrubbing regex to explicit upload events
  (avoids false-positive removal of "User works with CSV files" etc.);
  also filter upload-event facts from the facts array
- prompt: move `import re` to module scope; skip upload-only human
  messages (empty after stripping) rather than appending "User: "

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(memory): allow optional words between 'upload' and 'file' in scrub regex

The previous pattern required 'uploading file' with no intervening words,
so 'uploading a test file' was not matched and leaked into long-term memory.
Allow up to 3 modifier words between the verb and noun (e.g. 'uploading a
test file', 'uploaded the attachment').

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* test(memory): add unit tests for upload filtering in memory pipeline

Covers _filter_messages_for_memory and _strip_upload_mentions_from_memory
per Copilot review suggestion.  15 test cases verify:

- Upload-only turns (and paired AI responses) are excluded from memory queue
- User's real question is preserved when combined with an upload block
- Upload file paths are never present in filtered message content
- Intermediate tool messages are always excluded
- Multi-turn conversations: only the upload turn is dropped
- Multimodal (list-content) human messages are handled
- Upload-event sentences are removed from summaries and facts
- Legitimate file-related facts (CSV preferences, PDF exports) are preserved
- "uploading a test file" (words between verb and noun) is caught by regex

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>

backend/src/agents/middlewares/memory_middleware.py

@@ -1,5 +1,6 @@
 """Middleware for memory mechanism."""
 
+import re
 from typing import Any, override
 
 from langchain.agents import AgentState
@@ -22,10 +23,16 @@ def _filter_messages_for_memory(messages: list[Any]) -> list[Any]:
     This filters out:
     - Tool messages (intermediate tool call results)
     - AI messages with tool_calls (intermediate steps, not final responses)
+    - The <uploaded_files> block injected by UploadsMiddleware into human messages
+      (file paths are session-scoped and must not persist in long-term memory).
+      The user's actual question is preserved; only turns whose content is entirely
+      the upload block (nothing remains after stripping) are dropped along with
+      their paired assistant response.
 
     Only keeps:
-    - Human messages (user input)
-    - AI messages without tool_calls (final assistant responses)
+    - Human messages (with the ephemeral upload block removed)
+    - AI messages without tool_calls (final assistant responses), unless the
+      paired human turn was upload-only and had no real user text.
 
     Args:
         messages: List of all conversation messages.
@@ -33,17 +40,47 @@ def _filter_messages_for_memory(messages: list[Any]) -> list[Any]:
     Returns:
         Filtered list containing only user inputs and final assistant responses.
     """
+    _UPLOAD_BLOCK_RE = re.compile(
+        r"<uploaded_files>[\s\S]*?</uploaded_files>\n*", re.IGNORECASE
+    )
+
     filtered = []
+    skip_next_ai = False
     for msg in messages:
         msg_type = getattr(msg, "type", None)
 
         if msg_type == "human":
-            # Always keep user messages
-            filtered.append(msg)
+            content = getattr(msg, "content", "")
+            if isinstance(content, list):
+                content = " ".join(
+                    p.get("text", "") for p in content if isinstance(p, dict)
+                )
+            content_str = str(content)
+            if "<uploaded_files>" in content_str:
+                # Strip the ephemeral upload block; keep the user's real question.
+                stripped = _UPLOAD_BLOCK_RE.sub("", content_str).strip()
+                if not stripped:
+                    # Nothing left — the entire turn was upload bookkeeping;
+                    # skip it and the paired assistant response.
+                    skip_next_ai = True
+                    continue
+                # Rebuild the message with cleaned content so the user's question
+                # is still available for memory summarisation.
+                from copy import copy
+
+                clean_msg = copy(msg)
+                clean_msg.content = stripped
+                filtered.append(clean_msg)
+                skip_next_ai = False
+            else:
+                filtered.append(msg)
+                skip_next_ai = False
         elif msg_type == "ai":
-            # Only keep AI messages that are final responses (no tool_calls)
             tool_calls = getattr(msg, "tool_calls", None)
             if not tool_calls:
+                if skip_next_ai:
+                    skip_next_ai = False
+                    continue
                 filtered.append(msg)
         # Skip tool messages and AI messages with tool_calls