“Generative inbreeding” and its risks for human culture

Visit our on-demand library to view VB Transform 2023 sessions. Sign up here

Inbreeding refers to genomic corruption when members of a population breed with other members who are too genetically similar. This often leads to offspring with significant health issues and other deformities, as it amplifies the expression of recessive genes. When inbreeding is widespread – as it may be in modern breeding – the entire gene pool can degrade over time, amplifying the deformations as the population becomes less and less diverse.

In the world of generative AI, a similar problem exists, potentially threatening the long-term effectiveness of AI systems and the diversity of human culture. From an evolutionary perspective, first-generation large language models (LLMs) and other generation AI systems were trained on a relatively clean "gene pool" of human artifacts, using massive amounts of textual, visual and audio content to represent the essence of our cultural sensibilities. .

But as the Internet is flooded with AI-generated artifacts, there is a significant risk that new AI systems will train on datasets that include large amounts of AI-created content. AI. This content is not direct human culture, but imitated human culture with varying levels of distortion, thus corrupting the "gene pool" through inbreeding. And as the use of generation AI systems increases, this problem will only accelerate. After all, new AI systems trained on copies of human culture will fill the world with increasingly distorted artifacts, forcing the next generation of AI systems to train on copies of copies of culture. human, and so on.

Degrade Generation AI Systems, Distort Human Culture

I call this emerging problem “generative inbreeding” and worry about two troubling consequences. First, there's the potential degradation of generational AI systems, as inbreeding reduces their ability to accurately represent human language, culture, and artifacts. Second, there is the distortion of human culture by inbred AI systems that introduce more and more "distortions" into our cultural gene pool that do not truly represent our collective sensibilities.

Event

VB Transform 2023 on demand

Did you miss a session of VB Transform 2023? Sign up to access the on-demand library for all of our featured sessions.

Register now

On the first issue, recent studies suggest that generative inbreeding could break AI systems, causing them to produce worse and worse artifacts over time, like making a photocopy of a photocopy of a photocopy. This is sometimes referred to as "pattern collapse" due to "data poisoning", and recent research suggests that foundation patterns are far more susceptible to this recursive hazard than previously thought. Another recent study found that as AI-generated data increases in a training set, generative models become increasingly "doomed" to gradually decrease in quality.

On the...