Large language models use feedback loops that can lead to dull, repetitive outputs if their own generated text is used as training data.

“When you start doing that, the game is very soon over. You come, in this case, to us, a very dull, stable state, it just says the same thing over and over and over again forever. The language models are vulnerable to this process.”