27 lines
1.1 KiB
Markdown
27 lines
1.1 KiB
Markdown
# Context Drift
|
|
|
|
## Definition
|
|
A failure mode in LLM interactions where the model gradually loses focus on the original task or context, veering off-topic as the conversation progresses. The LLM "forgets" the original goal and generates responses that may be locally coherent but globally irrelevant.
|
|
|
|
## Causes
|
|
- Long conversations that exceed the model's effective context window
|
|
- Cumulative token budget leading to attention dilution
|
|
- Poor initial prompt definition
|
|
- Model's tendency to follow the most recent instructions over original ones
|
|
|
|
## Impact on Multi-Agent Systems
|
|
- Can propagate errors through agent chains
|
|
- Workers in a Hierarchy may drift from Planner's intended tasks
|
|
- Debates may veer off the original proposition being evaluated
|
|
- Knock-out agents may lose sight of the evaluation criteria
|
|
|
|
## Mitigation
|
|
- Break long tasks into atomic steps (Hierarchy pattern)
|
|
- Use explicit task validation at each step
|
|
- Keep agent contexts focused and limited
|
|
- Reset context periodically rather than accumulating
|
|
|
|
## Related Concepts
|
|
- [[Hallucination]]
|
|
- [[Multi-Agent Hierarchy]]
|
|
- [[Validator]] |