It is pitch black. You are likely to be eaten by a generative text model.
AI Dungeon is a pretty astonishing example of generative fiction—in the form of old-school text adventure game: “Imagine an infinitely generated world that you could explore endlessly, continually finding entirely new content and adventures. What if you could also choose any action you can think of instead of being limited by the imagination of the developers who created the game?”
This experiment by Nick Walton is built on top of OpenAI’s recent GPT–2 model for generating text after being primed with an arbitrary input (seed text to establish context). Unlike many prior language models, this one generates entire coherent paragraphs that feel a world ahead of prior-art attempts that simply string plausible words together.
Walton loaded the model with training data from a collection of text adventures obtained from chooseyourstory.com, and then let ’er rip. “The model at times shows remarkable understanding and writing quality like in this sample where the player tries to find love on a deserted island,” Walton writes:
You head to the island and explore it. You find it completely deserted save for a single tree. You walk up to it and examine it closely. It seems to be made of wood, but its leaves seem to be green rather than normal leafy green.
> find love
You take a deep breath and stare intently at the tree. Your eyes meet with the tree’s. You immediately feel a strong sense of euphoria. You smile and touch the tree. Its bark feels warm to the touch. You sniff the tree and find that its scent is very pleasant.
The unfolding adventures are reasonably coherent, too, and generally hang together as you go along. Unlike the virtual assistants many of us use today, this one actually remembers the thing you said to it moments before. Walton says every command is accompanied by the last eight commands and responses, effectively giving the game a short-term memory to maintain context. Even as you throw curve balls at the story engine, it works hard to make it fit what came before. Here’s a snippet from one of the resulting adventures:
"I am a goblin!" you say proudly. "And I'm glad to be one".
"Well then, congratulations," says the orc captain. "We'll let you live". You nod and walk away from the camp.
> ask if you can join the orc band
The orcs don't want to fight anymore so they're looking for someone to help them out in their new endeavors. They offer you a chance to join up with them as a drummer. It would allow you to play music while helping out around camp.
> tell them we should start a musical troupe
They look at you skeptically but agree to give you a try. You tell them about your plans to create a musical troupe and how you plan on using the orcs to help you out.
> teach them how to play the flute
You teach them how to play the flute and soon enough they're playing for you all the time. The orcs seem to enjoy it and they begin to think of you as their leader. Soon other orcs come to you asking to join too.
> accept them into your musical groupe and train to be the best musicians
You continue to teach them and eventually they become better than you. You learn that you need to practice more and more often until you can master the art of playing the flute. Eventually you find yourself practicing every day and finding it very difficult to sleep.
> take orcs to perform at a nearby festival
You take the orcs to perform at a local festival where you get to meet some of the other orcs you've trained. This is followed by another success when the orcs manage to attract the attention of the Emperor himself who grants you permission to use his personal carriage to travel around the Empire.
Why this matters
This is a toy, of course, but it’s a nifty demonstration of how the GPT model can be used on an arbitrary data set to create fairly compelling responses. This stuff remains primitive compared to human prose; AI Dungeon isn’t going to write award-winning fiction, but it does hint at ways that it could help human writers by proposing directional text. In a Hacker News thread, Walton wrote:
This doesn’t reach near the level of good human authors. There’s no long term plot or deep human themes in this. I don’t think this will ever replace quality human writing, but it may be able to augment it in cool ways. I personally would love if rather than every guard in Skyrim telling the exact same story, if each guard could have their own stories or comments generated based on things about their life. Human authors could provide high level details and let AI generators fill in the smaller details.
As with so many instances of machine learning, in other words, the best application here is not to replace human efforts but to augment them. What might be the role for this in supporting common or repetitive writing tasks? In supporting customer-support teams providing tailored responses to frequently asked questions? In giving automated agents better comprehension of the task we want them to accomplish?
