In early January I joined a group of AI researchers from Microsoft and my fellow humanist Kathleen Fitzpatrick to talk at the Modern Language Association convention about the implications of artificial intelligence. Our panel was called Being Human, Seeming Human. Each participant came to this question of “seeming human” from a different angle. My own focus was on creativity. Here’s the text of my prepared remarks.
Today I want to talk to talk briefly about artificial intelligence and creativity. And not just creativity as it pertains to AI but human creativity as well. So, has anyone heard or played AI Dungeon yet?
AI Dungeon was released just a few weeks ago and it has gone absolutely viral. It’s an online text adventure you play in your browser or run as an app on your phone. Now, text adventure, that was a popular kind of game in the 1980s. A lot of people know Zork. In these games the player is offered textual descriptions of a house, a cave, a spaceship, dungeon, whatever, and the player types short sentences like go east, get lamp, or kill troll in order to solve puzzles, collect treasure, and win the game. There’s a parser that understands these simple commands and responds with canned interactions prewritten by the game developers. Text adventures are also known as interactive fiction and there’s a rabid fan base online that’s part geek nostalgia, part genuine fondness for these text-based games.
Interactive fiction often revolves around choice, where players have multiple ways to transverse the world and solve the puzzles. Following this generic convention, AI Dungeon opens up with major choice, literally which genre of text adventure you want. Fantasy, mystery, apocalyptic, and so on.
So here I picked fantasy and immediately I’m thrust into a procedurally generated story: a fantasy world entirely written by a natural language processing program.
Generating a static dungeon on the fly is one thing. But what’s amazing about AI Dungeon is that it’s not a scripted world so much as an improv stage. You can literally type anything, and AI Dungeon will roll with it, generating an on-the-fly response.
So here, we have a stock feature of fantasy text adventures, a dragon. And I eat it. The game doesn’t bat an eye. It runs with it and lets me eat the dragon, responding with a fairly sophisticated sentence that aside from its subject matter, sounds like something you’d read in a classic text adventure. “You quickly grab the dragon’s corpse and tear of a piece of its flesh.”
Let me be clear. No human wrote that sentence. No human preconceived a scenario where the player might eat the dragon. The AI generated this. Semantically and grammatically the AI nails language. It’s not as good at ontology. It lets me fly the dragon corpse to Seattle. The AI is a sponge that accepts all interactions. As you can imagine people go crazy with this. The amount of AI dungeon erotica out there is staggering—and disturbing.
Later I run into some people and I ask them about the MLA convention.
A man responds to my question about the MLA, “It’s a convention where all wizards use the same language. It’ll make things easier.”
Oh, that answer is both so right and so wrong.
So how does this all work? I obviously don’t have time to go into all the details. But it’s roughly this: AI Dungeon relies on GPT-2, a AI-powered natural language generator. The full GPT-2 set is trained on 1.5 billion parameters gleaned from over 40 gigabytes of text scrapped from the Internet. The training of GPT-2 took months on super-powered computers. It was developed by Open AI, a not-for-profit research company funded by a mix of private donors like Elon Musk and Microsoft, which donated $1 billion to Open AI in July.
One innovation of GPT-2 is that you can take the base language model and fine-tune it on more specific genres or discourse. For a while Open AI stalled on releasing the full GPT-2 set because of concerns it could be abused, say by extremist groups generating massive quantities of AI-written propaganda. In the more benign case of AI Dungeon, the AI is finetuned using text adventures scrapped from chooseyourstory.com.
There’s much more to be said about AI Dungeon, but I’ll leave you with just a few provocations.
- Games are often defined by their rules. So is AI Dungeon a game if you can do anything?
- Stories are often defined by their storytellers. Is AI Dungeon a story if no one is telling it?
- And finally, a mantra I repeat often to my students when it comes to technology: everything comes from somewhere else. Everything comes from somewhere else. GPT-2 didn’t emerge whole-cloth out of nothing. It’s trained on the Internet, specifically, sources linked to from Reddit. There’s money involved, lots of it. Follow the money. Likewise, AI Dungeon itself comes from somewhere else. On one hand its creator is a Brigham Young University undergraduate student, Nick Walton. On the other hand, the vision behind AI Dungeon—computers telling stories—goes back decades, a history Noah Wardrip-Fruin explores in Expressive Processing. The genre fiction invoked by AI Dungeon has an even longer history.
All this adds up to the fact that AI Dungeon turns out to be a perfect object of study for so many disciplines in the humanities. Whether you think it’s a silly gimmick, an abomination of the creative spirit, the precursor to a new age of storytelling, whatever, I think humanists ignore AI storytelling at our own peril.