Dark forest theory of the web (2m31s)

Maggie Appleton opens with Yancey Strickler's 2019 framing — the web feels like a dark forest because anything you say sincerely will either drown in clickbait or attract a Twitter mob, illustrated with the Garden Lady who got pile-on'd for posting about morning coffee with her husband.

"I don't care if something good happened to you, it should have happened to me instead"

"we will take every opportunity to misinterpret things and be ungenerous to each other"

"this makes the web a very sincerely dangerous place to sincerely publish your thoughts"

The cosy web: where we've retreated (8m07s)

To escape, we move into newsletters, personal sites, and then into private Slacks, WhatsApps and Discords. Real human relationships happen there — but the knowledge isn't indexable, the spaces are hard to join from outside, and Discord's search has never worked.

"Venkatesh Rao pointed out that we've all started going underground"

"none of this is indexable or searchable. It's very hard to include people who aren't already in the group"

"and also, like, good luck finding anything on Discord"

Where generative AI is right now (9m41s)

A quick state-of-2023: GPT-4 and Claude for text, Midjourney and Stable Diffusion for images, RunwayML for video, all becoming multimodal. Tools like Blaze let any marketer generate 700-word SEO-optimised articles on demand and offer to spin them out into LinkedIn posts, tweets and YouTube scripts automatically.

"the cost of creating and publishing content just dropped to almost zero"

"this essentially murders Google Search... this just does away with SEO-optimised content because anyone can publish this immediately"

"humans are quite expensive and slow at making content... we demand people pay us like extraordinary rates"

The agent architecture and the Assistants API (13m44s)

The newer model: give the LM access to tools (web search, calculator, code, long-term memory), use chain-of-thought self-critique prompting, and you get an agent that plans, observes and acts in loops. OpenAI's brand-new (this Monday) Assistants API makes this trivially accessible to every developer.

"you tell the language model to act like an agent... it composes which tools it wants to use to achieve your goals"

"they used to require quite a lot of Python code and kind of insider knowledge — and they're just making it super easy for everyone to now do this architecture"

"we're about to enter this phase where this very capable agent architecture is becoming pervasive"

How content is about to drown (17m20s)

The Nigel example: any author, lobbying group or corporation can spin up an agent that drafts tweets, schedules a newsletter, generates podcast episodes in their voice, makes TikTok videos and replies to other people on LinkedIn — none of which violates spam rules and all of which scales infinitely. The As an AI language model phrase already gives away thousands of LinkedIn posts and Amazon reviews.

"I did a quick search for ['as an AI language model'] on LinkedIn, it got 16,000 hits"

"I think we're just gonna be absolutely swamped in masses of mediocre content"

"without an agent, 99% of Nigel-type people wouldn't have gone to all this effort — but with the agent, we have people like Nigel, but times 99"

What's different about generated content (23m27s)

Three distinguishing features: generated content has no real connection to physical reality (hence hallucination), it has no fixed social context (it can pretend to be a professor or a child as instructed), and it has no possibility of an ongoing human relationship. The film Her was supposed to be a warning; Replika took it as a business plan.

"we politely call this hallucination... like some kind of very smart person on some mild drugs"

"a language model is not a person, and it does not have a fixed reality"

"any kind of language model agent on the internet has no capacity for that back-and-forth relationship"

The reverse Turing test (30m08s)

The original Turing test had the computer prove it was human. On the post-2023 web, you have to prove you're human. Short-term defences: use evolving teenage jargon, write in a non-dominant language (Catalan, Welsh), reference specific real-world events and people only an embodied person could know.

"we are now the ones under scrutiny — we have to prove we're real"

"we will assume everyone is an agent until proven otherwise"

"if you speak something like Catalan or Welsh, you're probably in a pretty good position"

Habsburg AI: a circular flow of information (31m41s)

The most worrying feedback loop: AI researchers will run out of real-world training data within five years, so they'll feed model outputs back into new models. Google's AI summary already once said yes, you can melt an egg because it pulled the answer from a Quora post written by ChatGPT. Scientific papers with "Regenerate response" still embedded have been published in legitimate journals.

"all kinds of hallucinated answers are gonna become part of this loop if all these major websites are using ChatGPT to generate answers, and then they just feed it to each other"

"this paper... included the phrase 'Regenerate response' at the end of a paragraph, which is the button above ChatGPT's input box"

"it just takes the whole replication crisis to a whole new level"

More privatised, gate-kept spaces (34m47s)

Substack and Medium login walls, Reddit charging $42,000/month for API access, Twitter doing roughly the same — these aren't accidents. They're authors and platforms reacting to having their content scraped for training data, and the unintended consequence is that the web stops being open by default.

"you understand why authors do this — actually you're having your content scraped and then fed into generative models, [which] puts you at a disadvantage"

"Reddit did this recently... they just were like, 'we're gonna put the price so high that no company in their right mind would really pay it'"

"everything is locked down and gate-kept and kind of cordoned off"

Possible futures: meatspace premium, personal LMs, three principles (36m18s)

Several non-mutually-exclusive futures. Meatspace premium: you only trust people you've met in person, which disadvantages disabled and caregiving users. Worldcoin-style biometric humanity proofs: weird but persistent. Personal LMs as defenders: filtering, debunking, flagging misinformation in your browser. And three principles for building responsibly — protect human agency, treat models as reasoning engines (not sources of truth), and augment rather than replace cognition.

"we will start to doubt all people online, and the only way to confirm someone's humanity is to meet them in person"

"the ideal form of this is that the locus of agency stays within the human, and it has a collaborative agent on hand"

"we should think about robots as animals... in this very mutually-beneficial relationship... and not this big scary alien who might come kill us all"