fix(memory): prevent file upload events from persisting in long-term memory (#971)

* fix(memory): prevent file upload events from persisting in long-term memory

Uploaded files are session-scoped and unavailable in future sessions.
Previously, upload interactions were recorded in memory, causing the
agent to search for non-existent files in subsequent conversations.

Changes:
- memory_middleware: skip human messages containing <uploaded_files>
  and their paired AI responses from the memory queue
- updater: post-process generated memory to strip upload mentions
  before saving to file
- prompt: instruct the memory LLM to ignore file upload events

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(memory): address Copilot review feedback on upload filtering

- memory_middleware: strip <uploaded_files> block from human messages
  instead of dropping the entire turn; only skip the turn (and paired
  AI response) when nothing remains after stripping
- updater: narrow the upload-scrubbing regex to explicit upload events
  (avoids false-positive removal of "User works with CSV files" etc.);
  also filter upload-event facts from the facts array
- prompt: move `import re` to module scope; skip upload-only human
  messages (empty after stripping) rather than appending "User: "

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(memory): allow optional words between 'upload' and 'file' in scrub regex

The previous pattern required 'uploading file' with no intervening words,
so 'uploading a test file' was not matched and leaked into long-term memory.
Allow up to 3 modifier words between the verb and noun (e.g. 'uploading a
test file', 'uploaded the attachment').

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* test(memory): add unit tests for upload filtering in memory pipeline

Covers _filter_messages_for_memory and _strip_upload_mentions_from_memory
per Copilot review suggestion.  15 test cases verify:

- Upload-only turns (and paired AI responses) are excluded from memory queue
- User's real question is preserved when combined with an upload block
- Upload file paths are never present in filtered message content
- Intermediate tool messages are always excluded
- Multi-turn conversations: only the upload turn is dropped
- Multimodal (list-content) human messages are handled
- Upload-event sentences are removed from summaries and facts
- Legitimate file-related facts (CSV preferences, PDF exports) are preserved
- "uploading a test file" (words between verb and noun) is caught by regex

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>

backend/src/agents/memory/prompt.py

@@ -1,5 +1,6 @@
 """Prompt templates for memory update and injection."""
 
+import re
 from typing import Any
 
 try:
@@ -108,6 +109,9 @@ Important Rules:
 - For history sections, integrate new information chronologically into appropriate time period
 - Preserve technical accuracy - keep exact names of technologies, companies, projects
 - Focus on information useful for future interactions and personalization
+- IMPORTANT: Do NOT record file upload events in memory. Uploaded files are
+  session-specific and ephemeral — they will not be accessible in future sessions.
+  Recording upload events causes confusion in subsequent conversations.
 
 Return ONLY valid JSON, no explanation or markdown."""
 
@@ -249,6 +253,16 @@ def format_conversation_for_update(messages: list[Any]) -> str:
             text_parts = [p.get("text", "") for p in content if isinstance(p, dict) and "text" in p]
             content = " ".join(text_parts) if text_parts else str(content)
 
+        # Strip uploaded_files tags from human messages to avoid persisting
+        # ephemeral file path info into long-term memory.  Skip the turn entirely
+        # when nothing remains after stripping (upload-only message).
+        if role == "human":
+            content = re.sub(
+                r"<uploaded_files>[\s\S]*?</uploaded_files>\n*", "", str(content)
+            ).strip()
+            if not content:
+                continue
+
         # Truncate very long messages
         if len(str(content)) > 1000:
             content = str(content)[:1000] + "..."

backend/src/agents/memory/updater.py

@@ -1,6 +1,7 @@
 """Memory updater for reading, writing, and updating memory data."""
 
 import json
+import re
 import uuid
 from datetime import datetime
 from pathlib import Path
@@ -135,6 +136,47 @@ def _load_memory_from_file(agent_name: str | None = None) -> dict[str, Any]:
         return _create_empty_memory()
 
 
+# Matches sentences that describe a file-upload *event* rather than general
+# file-related work.  Deliberately narrow to avoid removing legitimate facts
+# such as "User works with CSV files" or "prefers PDF export".
+_UPLOAD_SENTENCE_RE = re.compile(
+    r"[^.!?]*\b(?:"
+    r"upload(?:ed|ing)?(?:\s+\w+){0,3}\s+(?:file|files?|document|documents?|attachment|attachments?)"
+    r"|file\s+upload"
+    r"|/mnt/user-data/uploads/"
+    r"|<uploaded_files>"
+    r")[^.!?]*[.!?]?\s*",
+    re.IGNORECASE,
+)
+
+
+def _strip_upload_mentions_from_memory(memory_data: dict[str, Any]) -> dict[str, Any]:
+    """Remove sentences about file uploads from all memory summaries and facts.
+
+    Uploaded files are session-scoped; persisting upload events in long-term
+    memory causes the agent to search for non-existent files in future sessions.
+    """
+    # Scrub summaries in user/history sections
+    for section in ("user", "history"):
+        section_data = memory_data.get(section, {})
+        for _key, val in section_data.items():
+            if isinstance(val, dict) and "summary" in val:
+                cleaned = _UPLOAD_SENTENCE_RE.sub("", val["summary"]).strip()
+                cleaned = re.sub(r"  +", " ", cleaned)
+                val["summary"] = cleaned
+
+    # Also remove any facts that describe upload events
+    facts = memory_data.get("facts", [])
+    if facts:
+        memory_data["facts"] = [
+            f
+            for f in facts
+            if not _UPLOAD_SENTENCE_RE.search(f.get("content", ""))
+        ]
+
+    return memory_data
+
+
 def _save_memory_to_file(memory_data: dict[str, Any], agent_name: str | None = None) -> bool:
     """Save memory data to file and update cache.
 
@@ -244,6 +286,12 @@ class MemoryUpdater:
             # Apply updates
             updated_memory = self._apply_updates(current_memory, update_data, thread_id)
 
+            # Strip file-upload mentions from all summaries before saving.
+            # Uploaded files are session-scoped and won't exist in future sessions,
+            # so recording upload events in long-term memory causes the agent to
+            # try (and fail) to locate those files in subsequent conversations.
+            updated_memory = _strip_upload_mentions_from_memory(updated_memory)
+
             # Save
             return _save_memory_to_file(updated_memory, agent_name)
 

backend/src/agents/middlewares/memory_middleware.py

@@ -1,5 +1,6 @@
 """Middleware for memory mechanism."""
 
+import re
 from typing import Any, override
 
 from langchain.agents import AgentState
@@ -22,10 +23,16 @@ def _filter_messages_for_memory(messages: list[Any]) -> list[Any]:
     This filters out:
     - Tool messages (intermediate tool call results)
     - AI messages with tool_calls (intermediate steps, not final responses)
+    - The <uploaded_files> block injected by UploadsMiddleware into human messages
+      (file paths are session-scoped and must not persist in long-term memory).
+      The user's actual question is preserved; only turns whose content is entirely
+      the upload block (nothing remains after stripping) are dropped along with
+      their paired assistant response.
 
     Only keeps:
-    - Human messages (user input)
-    - AI messages without tool_calls (final assistant responses)
+    - Human messages (with the ephemeral upload block removed)
+    - AI messages without tool_calls (final assistant responses), unless the
+      paired human turn was upload-only and had no real user text.
 
     Args:
         messages: List of all conversation messages.
@@ -33,17 +40,47 @@ def _filter_messages_for_memory(messages: list[Any]) -> list[Any]:
     Returns:
         Filtered list containing only user inputs and final assistant responses.
     """
+    _UPLOAD_BLOCK_RE = re.compile(
+        r"<uploaded_files>[\s\S]*?</uploaded_files>\n*", re.IGNORECASE
+    )
+
     filtered = []
+    skip_next_ai = False
     for msg in messages:
         msg_type = getattr(msg, "type", None)
 
         if msg_type == "human":
-            # Always keep user messages
-            filtered.append(msg)
+            content = getattr(msg, "content", "")
+            if isinstance(content, list):
+                content = " ".join(
+                    p.get("text", "") for p in content if isinstance(p, dict)
+                )
+            content_str = str(content)
+            if "<uploaded_files>" in content_str:
+                # Strip the ephemeral upload block; keep the user's real question.
+                stripped = _UPLOAD_BLOCK_RE.sub("", content_str).strip()
+                if not stripped:
+                    # Nothing left — the entire turn was upload bookkeeping;
+                    # skip it and the paired assistant response.
+                    skip_next_ai = True
+                    continue
+                # Rebuild the message with cleaned content so the user's question
+                # is still available for memory summarisation.
+                from copy import copy
+
+                clean_msg = copy(msg)
+                clean_msg.content = stripped
+                filtered.append(clean_msg)
+                skip_next_ai = False
+            else:
+                filtered.append(msg)
+                skip_next_ai = False
         elif msg_type == "ai":
-            # Only keep AI messages that are final responses (no tool_calls)
             tool_calls = getattr(msg, "tool_calls", None)
             if not tool_calls:
+                if skip_next_ai:
+                    skip_next_ai = False
+                    continue
                 filtered.append(msg)
         # Skip tool messages and AI messages with tool_calls