Context Engineering for Agents: Strategic Forgetting

·

컨텍스트 엔지니어링의 현주소

Reduced to a single sentence, the factor that determines the performance of an AI agent is closer to context engineering than the model itself: selecting and delivering the information needed for the current goal. Dividing responsibilities between deterministic code and probabilistic models, and periodically discarding outdated state, are emerging as alternatives that compensate for the limitations of long context.

Four Failures Created by Long Context

Long context windows hold not only more information, but also hallucinations, incorrect facts, and conflicting examples. The resulting failures include context contamination, in which the model fixates on incorrect information; attention dilution caused by excessive focus on the context; confusion, in which facts unrelated to the current problem influence the answer; and context collisions, where tools and skills unexpectedly conflict.

In Gemini 2.5’s Pokémon playing experiment, older game rules and newer tasks were presented together, causing the model to pursue outdated goals for several hours. According to an appendix in a technical report, once the context considerably exceeded 100,000 tokens, the model began repeating past actions instead of creating a new plan.

In Carolina Camassa and Derek Shiller’s “Do as I Say, Not as I Do” experiment, adherence to original instructions declined across every tested model as the number of conversational examples conflicting with new instructions increased. Gemma 3 12B dropped to less than half its compliance rate with only a few conflicting examples, and Opus 4.6 showed the same problem as examples accumulated.

Six Techniques for Context Management, from Augmentation to Summarization

Context management includes enriching information with RAG and skills, as well as selecting tools to expose only those suited to a request, preventing overload. One company loaded 600 skills for each engineer whenever a software project began, causing an abrupt decline in performance and making tool filtering necessary. Context isolation starts subagents and peer agents with an empty conversation history, separating information and goals from other tasks.

Other techniques include pruning outdated history, external storage that keeps information in Markdown files or databases, and summarization, which condenses older conversations to begin a new context. Claude Code and Codex also removed tool results that had become unnecessary during the initial stages of context compression. In practice, evaluating relevance first and then reducing history before summarizing can produce more useful compression results.

Moving Context Construction into the Harness

Context engineering evolved from prompts that dynamically assemble system context into harness engineering, which moves processing into code outside the context. A DSPy demonstration combined a ReAct module and Python function tools in a while loop to accumulate conversation history.

Tool selection used the large model Claude Sonnet 4.5 together with the smaller judgment model Jev. Jev evaluates requests and tools, selects the necessary tool set, isolates subagents, and reviews each conversation turn to remove unnecessary history. The change that incorporated these four improvements into a single loop was introduced in roughly 16 lines.

Failures in Long-Running Tasks and Strategic Forgetting

Long-running task models have introduced new problems: a “fight with the weights,” where learned behavior conflicts with current instructions, and “context inertia,” in which the model continues pursuing hypotheses and subgoals that should be abandoned. Claude Code’s system prompt and tool instructions repeatedly requested parallel execution of independent tool calls, revealing a behavioral bias in the Learning environment.

The Memento harness used in Anthropic’s self-replication research mitigated fixation by writing brief notes and clearing the context every 90 minutes. It preserved the environment and task results while discarding the ongoing conversation, allowing the agent to restart from a new perspective. Microsoft Research’s Fork-and-Flush addressed the problem of exploration stalling after major improvements, while UC Berkeley’s Scaling Discovery Requires Forgetting, led by Shangyin Tan, reported that performance rose again when the context was periodically cleared while retaining outputs.

For personal agents, memories and skills accumulate over time, and conflicting instructions are likely to become a new operational challenge. During six months of operating Hermes, unusual modifications accumulated in skills and could worsen problem-solving behavior; a general-purpose agent also faced conflicts between memories and instructions as it solved the same problem in different ways. Existing techniques such as isolation, external storage, and deleting accumulated history remain the available responses. The quality of an agent harness depends less on the size of the context window than on how precisely it automates goal-oriented information selection and strategic forgetting.

This shift can be observed in material on the current state of context engineering. Teams designing long-running agents must consider not only prompt length, but also what to retain in the context and when to clear it.

Original Source

This article was prepared after reviewing the following original source: geeknews — The Current State of Context Engineering

Leave a Reply

Your email address will not be published. Required fields are marked *