Weaker LLMs Accidentally Delete Content, Shrinking...

Weaker models tend to incur deletion: accidentally dropping content, which makes the issue noticeable after several interactions due to an obvious shrinking in the overall document content. In frontier LLMs, however, the root issue is not deletion but corruption: they keep the documents' overall "look and feel", even maintaining a nearly intact word count, but they silently mistype, modify, or replace factual information with fabrications that still sound plausible. Here's the irony: the smarter the model, the more difficult it becomes to detect its corruptive behavior, as the final output still looks legitimate at first glance.

Context Overload and Distractor Attachments In a messy condition -- with a lot of context information or excessive attached documents -- models struggle to keep information structurally intact. As the document size increases or more "distractor files" are included as part of the prompt context, the severity and impact of degradation skyrockets, losing the grip on accurate details and filling gaps based on predictive logic. The model no longer adheres to the source text, as it finds it easier to just guess.

The Importance of Domain Familiarity One last reason why models tend to degrade documents in complex interactions involving delegation relates to the nature of the use case and how familiar the model is with it.