Users of public AI chatbots regularly report that assistants misread simple prompts, skip instructions or fail at tasks they handled well months earlier. Two research papers, one from Stanford and UC Berkeley and one published in Nature, offer measured explanations: the "same" model can change quietly between updates, and future models may degrade if they learn too heavily from AI-generated text.
The topic matters now because consumer chatbots are increasingly built into email, documents and search. Small shifts in how a model follows instructions can break workflows that people assumed were stable.
Technical Architecture and Specifications
The first study, by Lingjiao Chen, Matei Zaharia and James Zou, compared the March 2023 and June 2023 versions of GPT-3.5 and GPT-4. It tested them on math problems, sensitive questions, opinion surveys, multi-hop knowledge questions, code generation, US medical licensing questions and visual reasoning. The authors note that providers update these services in ways that are opaque to users.
The second study, led by Ilia Shumailov and published in Nature in July 2024, looks at a different mechanism called "model collapse." It describes what happens when models are trained repeatedly on content produced by earlier models. The researchers found that rarer content in the original data tends to disappear over successive generations.
Empirical Data and Real-World Impact
- GPT-4 identified prime versus composite numbers with 84% accuracy in March 2023 and 51% in June 2023. The authors link this partly to a drop in the model's willingness to follow chain-of-thought prompting.
- Both GPT-4 and GPT-3.5 produced more formatting mistakes in code generation in June than in March.
- GPT-4 became less willing to answer sensitive questions and opinion survey questions.
- Results were mixed rather than uniformly negative: GPT-4 improved on multi-hop questions, and GPT-3.5 was much better at the prime-number task in June.
- The authors provide evidence that GPT-4's ability to follow user instructions decreased over time, which they describe as one common factor behind the many behavior drifts.
In the Nature study, recursively trained language models tended to display repeating phrases. In one test that started with text about medieval architecture, the output by the ninth generation was a list of jackrabbits. The authors suggest that training on AI-generated data is not impossible, but that filtering such data must be taken seriously.
Industry Context and Future Outlook
Both papers come with limits. The Stanford and Berkeley authors state that their benchmarks do not comprehensively cover ChatGPT's behavior. Their aim was to show that drift exists even on simple tasks. The Nature results come from controlled recursive-training experiments and do not by themselves show that any specific commercial product has been affected.
The practical takeaway from the drift study is a call for continuous monitoring. The authors say the behavior of the "same" LLM service can change substantially in a short time, which can break downstream pipelines and make results hard to reproduce. The model-collapse paper adds that data from genuine human interactions will become more valuable as AI-generated text spreads across the web. Providers have not published detailed, comparable update histories for consumer chatbots, so independent testing remains the main way to catch regressions.
Sources: How is ChatGPT's behavior changing over time? (arXiv) and Tech Xplore summary of the Nature study on model collapse.
```
