Can Machines Write Musicals?

In 1992, as personal computers were beginning to reshape everyday life, Sony co-founder Masaru Ibuka wrote that computing could never overshadow human achievement, because of one missing quality: creativity. "A computer isn't creative on its own because it is programmed to behave in a predictable way," he wrote in Fortune. "Creativity comes from looking for the unexpected and stepping outside your own experience. Computers simply cannot do that."

But new projects are challenging the question of computer creativity—like Beyond the Fence, the world's first computer-generated musical, which opens at the Arts Theater in London today.

"From the beginning, it was an experiment to see how much you could get a computer to do," said Benjamin Till, who came up with the idea. Till is a composer by trade, probably best known for creating a musical rendition of his own wedding to writer and actor Nathan Taylor. For this project, Till and a team of researchers from around the world designed a series of algorithms to create the "magic formula" for a hit musical. Each part of the production—the premise, the narrative, the lyrics, the score, and even the size of the cast—was determined by a computer system. If it's a hit, it'll be the musical theater equivalent of passing the Turing Test.

"There is not a single song in this show, not a single moment, that wasn't at some point inspired by or written by a computer," said Till. Altogether, computers fully determined the premise and generated 25 percent of the music and lyrics.

The experiment challenges the categorical divide between "science" and "art," and the belief that what makes humans human is, at its heart, creativity. Computers can be programmed to do all sorts of things, but algorithms can't create the X factor—that piece of art that resonates with us emotionally, on a human level. If a computer can write a musical, or write a storyline that moves us, then what is it that separates man from machine?

The question is relevant to a growing field of "computer music" researchers, who study the constellation of music composition, performance, orchestration, engineering, computing, signal processing, machine learning, artificial intelligence, data analysis, and the general question of how technology can be used to further artistic pursuits. Ge Wang, a professor of music and computer science at Stanford University, says an experiment like Beyond the Fence is interesting because it tests the boundary between what can be made only by humans and what can be outsourced to machines.

"These things do make you wonder about the role between human creativity and machine creativity," said Wang. "I mean, machine creativity—what does that mean?"

In a technical sense, there is no such thing: Computers aren't exactly imaginative, and they don't create things on their own volition. They are, however, extremely good at replication and creating things in a particular artistic style. As early as the 1950s, computers could follow simple algorithms to compose a musical score; by the 1990s, computers could learn to paint in the style of Picasso or compose music that sounded distinctly like Bach.

"The history of music composition by computers goes back almost as far as computing itself," said Roger B. Dannenberg, a professor of computer science, art, and music at Carnegie Mellon University. "Machine learning is evolving very rapidly, and there's been a lot of recent work on applying techniques of machine learning to areas like music."

Take, for example, Spotify's discovery algorithms. The machine can't judge the artistic merit of a song, but it can use data to predict which songs you'll enjoy. Computers aren't emotional, but for years, they've been able to guess at the things that will move us, using data and replication. Part of starting a new artistic project is sourcing inspiration, and what is inspiration if not data?

Data-mining was the first, and perhaps most important, step in building Beyond the Fence. Statisticians at Cambridge University designed an algorithm to sort through nearly 2,000 musicals to find patterns that separated the hits from the flops. (It constitutes the largest study of data on musical theater in the world.) What they found was that, based on the statistical analysis, their best shot at a hit musical should take place in Europe in the 1980s, feature a female protagonist, and involve a war conflict. There had to be a high-energy musical number at the top of each act, a death halfway through the second act, a strong comedy number. They used a "What-If Machine"—a computer system that spat out various premises, designed by a team at Goldsmiths, University of London—to complete the parameters for the plot. Afterward, Till said the team just had to "fill in the dots."

Giving machines license to create these parameters freed up the team to focus on the emotional material, like the dialogue ("there is not yet a single computer program that can write dialogue," said Till). A machine listening analysis rendered thousands of bars of music, and a poetry generator created lyrics—most of them very, very bad—that the team sorted through to find phrases that might work. Machines did the churning, but it was human curation at the helm.

Till wrote the first draft of Beyond the Fence in a few weeks, and they finished the entire show in four months. For comparison, he spent over a year just researching and writing the first draft of previous musicals. And while Till said there was something slightly uncomfortable about surrendering a key part of the creative process to an algorithm, Wang points out that the process played to both the strengths of people and computers.

"Machines are good at sorting through data; people are good in a curatorial role," said Wang. "It's not like someone wrote a program and pressed a button and out came the score and script for the musical," said Wang. "There's a lot of human decision-making." (Till, for his part, said the symbiosis was reassuring: "If I'd have done this project and suddenly realized in ten years time I'd be completely out of a job, then that would be a terrifying thing.")

Beyond the Fence officially opens tonight, and throughout the show's preview week, Till says he didn't hear much conversation about the computer-generated aspects of the show. "People seem to be watching it without having that discussion, which is extraordinary," he said. "Instead, people are talking about the plot, as you'd expect [from a normal production]."

And that's exactly the point: It's not a robot's vision of musical theater. It's musical theater enhanced by technology.

"The source of material may be very different from a normal musical, but it still feels like our show, rather than something created by the computers."

Beyonce the Fence will play at the Arts Theater in London's West End until March 5. Get tickets here.

Follow Arielle Pardes on Twitter.