How Gemini co-composed this contemporary classical music piece

There is a moment in “The Twin Paradox: A Symphonic Discourse," a new piece of contemporary classical music first performed by the Munich Symphony Orchestra at the end of last year, that stands out for its breath-snatching intensity. As the fourth of the piece’s five movements nears its climax, horns begin blaring in cutting jolts, shrieking violins build and drums thunder violently underneath.

It feels more metal than Mozart — and that’s by design.

“The Twin Paradox” was composed by a unique trio: German composers Jakob Haas and Adrian Sieber, together with a newcomer to the world’s concert halls: the Gemini API via Google AI Studio. While the model is unable to write or create music itself, it was a valuable collaborator, the composers say, able to suggest narrative ideas, instrumentation and musical motifs. And get a little wild.

As part of their process, Jakob and Adrian took advantage of Gemini’s multimodal capabilities, filming interviews with 12 members of the orchestra along with several minutes of each musician improvising with their instrument. They uploaded the footage to Gemini, then asked for musical ideas for each player — something that would tap into their passions and also complement the overall piece they were building.

“Alex, our percussionist, used to play in metal bands before he started his classical career,” says Jakob, also a cellist in the orchestra. “He improvised a special kind of drumming he used in metal and when we asked Gemini to suggest a rhythm for the fourth movement, that sound came through. It became the basis of that whole section.”

“The Twin Paradox” project is a collaboration between the composers, the orchestra, Google Arts & Culture and Google Germany. Having previously worked with Google to develop a music module for a project that brought AI to students, Jakob had the idea of co-composing with AI and was quickly introduced to the team at Google Arts & Culture, who has a long history of experimenting with classical music.

“We’ve worked with famed record label Deutsche Grammophon to digitize some of the oldest records ever made, and produced a 360° of the opening performance of the Elbphilharmonie concert hall in Hamburg,” says Simon Rein, senior program manager at Google Arts & Culture. “What made this project special to me was the emphasis on how AI can support creativity and human connection: composers coming together with Gemini to write the piece, an orchestra comes together to work out how to play it and you end in a concert where hundreds of people enjoy it together.”

Gemini’s first big contribution to the piece was the title. “We started by giving Gemini the basics: the number and type of instruments available, the desired length of the piece, and that it would be a creation between human composers and AI. Then we said, ‘OK, give us a name,’” Jakob explains. In a clever nod to itself, Gemini suggested “The Twin Paradox,” the name of a thought experiment from relativity theory that posits that if one of a pair of twins makes an extended space journey at near lightspeed, they will have aged less than their twin when they return.

“We then challenged Gemini to tell us how The Twin Paradox would actually sound as music,” Jakob says. “It suggested using different tempos at the same time, as well as glissandi — or sliding between notes — to show the stretching and compressing of time. That really inspired us. It was when I thought, ‘Wow, this is going to work.’”

Over several months, Jakob interacted with the Gemini API via Google AI Studio on his laptop, sending its suggestions to Adrian, who would translate them into sheet music and play them on piano. The duo would discuss the results, refine and choose what to include in their composition.

Effective prompting was essential, and the team worked with the Google Arts & Culture lab to develop and fine-tune their inputs. Knowing that AI models can often imitate musical ideas that already exist, due to the fact they’re trained on existing datasets, Jakob and Adrian refrained from suggesting any specific styles of music or any known composers. They also leaned into something they call “metaphorical prompting,” providing concepts, images or videos to inspire musical ideas rather than directly asking for specific musical elements.

The most elaborate of these prompts were the orchestra videos that led to that thumping metal moment. The most self-referential came when they asked the model to reflect on the process of collaborating with the composers. The result? A special segment of the second movement where the oboe and solo violin stand in for questioning human voices, while percussion and brass instruments deliver metallic, machine-like responses. Call it Gemini does Gemini.

“The Twin Paradox” made its debut in October at Munich’s Prinzregententheater. Simon had been impressed by what he’d heard in audio files the composers shared during its development, and at a final rehearsal he attended, but there were still nerves. “Thankfully, on the night, it got rousing applause,” he says. “People were hooked — they were moved.”

Jakob, who didn’t play that night but watched from the front row, remembers the palpable tension in the room — especially during the fourth movement. “It was breathtaking,” he says.

And while he hasn’t settled on his next project, he says he will continue to experiment with AI.

He may even work in a more multimodal musical artform.

“Adrian composes a lot of operas, and we’ve been talking about how well that form plays to the strengths of a model like Gemini,” Jakob says. “You go beyond music alone, with elements such as lyrics, scene descriptions and visual components. That’s just one idea. And if I need more, I know one place I can ask…”