Renowned for its hilarious and painstakingly detailed stop-motion movies, like Wallace & Gromit: The Curse of the Were-Rabbit, The Pirates: Band of Misfits and Shaun the Sheep, Bristol, England’s Aardman Animations has been at the forefront of creative storytelling and technical animation for more than 40 years.

But when Google came knocking last year with a proposal to work collaboratively on a new story development kit (SDK) for its smartphone-based, 360-degree video platform, Spotlight Stories, and create the latest short film for the project, the studio’s technical abilities were put to the test like never before.

The end result of what was ultimately a 122-day production cycle is Special Delivery, a Christmas-themed, interactive animated caper, directed by Tim Ruffle, about an apartment building custodian who embarks on a comical chase after mistaking Santa Claus for an intruder.

“It was one of the most technically challenging projects we’ve worked on, but ultimately all the more rewarding as a result.”

– Heather Wright, Aardman Animations

Launched in December 2015, with design inspiration from Alfred Hitchcock’s Rear Window and United Productions of America’s animation style from the 1950s, the short seamlessly merges an old-fashioned 2D look with a modern, interactive 3D experience that features cinematic quality, 360-degree video, full-sphere surround sound and motion sensor-driven story options.

The short is currently available in two versions—a fully interactive YouTube video optimized for select Android devices via the YouTube app, and a non-interactive YouTube 360 video for other Android and iOS devices, as well as the web. Both versions, however, are compatible with Google’s Cardboard virtual reality viewer for a heightened 3D experience.

By combining non-linear storytelling with motion sensor technology, the interactive version of the short offers a unique experience every time. It contains 10 subplots, three ways to view the ending, and more than 60 moments in which viewers can follow the story in a different way.

“It was one of the most technically challenging projects we’ve worked on, but ultimately all the more rewarding as a result,” says Aardman executive producer and head of partner content Heather Wright.

With a point-of-view from the backyard of the custodian’s snowy apartment complex, viewers can follow either main character (the custodian or Santa) as they often end up in separate locations.

And tilting the phone in different directions shifts the POV to other story elements while the main action gets put on hold.

“We wanted to come up with an idea that worked within the broad parameters of how the technology works in terms of the viewers taking control of the point-of-view and effectively pointing the camera where they wanted, when they wanted,” says Wright. “We were also keen to bring our brand of silent slapstick comedy to the piece, so the idea of a courtyard surrounded by lots of doors and windows, entrances and exits was then an obvious extension of the idea.”

The four-minute film, in fact, features 77 doors and windows, 60 props including baubles, candy canes, tree lights and trapdoors, and a total of 33 animated scenes—one title sequence, 20 main scenes and 12 background scenes.

A full 20 days of the production schedule were spent building characters, 79 days were devoted to character shading/texturing, and 46 days were spent on animation.

Eleven animators, in total, worked eight hours a day for a total of 4,048 hours from development to delivery.

To make the multiple story elements work seamlessly together, Google Spotlight Stories’ technical project lead Rachid El Guerrab says the teams designed a new type of virtual camera that lets viewers look up and down between the different stories of the cityscape, so that everywhere a person looks is framed within the phone screen, functioning almost like an elevator.

The technical innovation is exactly the type of collaborative work between engineers and artists that Spotlight Stories has been delivering since Google took over the Advanced Technology and Projects R&D unit in 2011, when it acquired Motorola Mobility.

Officially launched under Google in 2013 as a means to create a new form of storytelling for mobile and VR, Spotlight Stories introduced Windy Day, a one-off 360-degree video by Ratatouille co-director Jan Pinkava, who is now the initiative’s director.

For its second short, the initiative released Buggy Night, with direction from veteran animator Mark Oftedal and most of the Windy Day team.

Before Special Delivery, two additional Spotlight Stories were released—Oscar-nominated hand-drawn film Duet from acclaimed Disney animator Glen Keane, and Help, the first live-action Spotlight Story from Fast and the Furious director Justin Lin.

One of the main differences between the first group of Spotlight Stories and Special Delivery is that the others had to be played on a dedicated app and were very large downloads. Aardman’s film, on the other hand, is the first Spotlight Story to stream directly on YouTube, and the effort to make this happen was not without challenges.

According to El Guerrab, interactive, 3D and 360-degree streaming with real-time graphics has never been done before on YouTube.

“For an instantaneous and fully immersive experience, the team used a variety of heuristic and automatic planning approaches to enable virtual worlds, with unpredictable playthroughs, to behave just like a video,” he says. “To ship within the YouTube app on mobile, for potentially hundreds of millions of phones, every bit of code size matters.”

After a lot of problem-solving, a fully interactive 360 format with advanced rendering, animation, visual effects, video and audio playback that would normally use tens of megabytes of code was compressed to a package smaller than one codec.

On a broader scale, El Guerrab says Google’s team had to build tools for the complex world of real-time rendering and animation on mobile that traditionally only game developers understand, and then bring them to a much wider and more critical audience of film creators.

“And most importantly, we had to come up with the most straightforward user experience for a general audience that couldn’t care less about the tech behind it and just wants a show they’ll enjoy,” he says. “We had to make it as easy to get to as clicking on a video link in a YouTube list.”

For Aardman, the biggest challenge was the writing process and getting away from traditional linear storytelling where a director controls what the audience is looking at.

A cardboard set helped Aardman visualize a 360-degree story

“Our standard process is to create a storyboard and from there an animatic, but in this case storyboard artist Andy Janes started with one post-it note at the top of a pin-up board—and from there the story split out into branches following the multiple stories,” explains Wright. “Getting our head around the multiple stories was really hard, but once we had the basic story structure—albeit in a very long string of post-it notes—we then had to work out the 3D choreography.”

To better visualize how the story would move around the apartment block, Janes constructed a cardboard set and made paper cut-outs of characters placed on the ends of pencils.

“It gave us the ability to almost puppeteer our way through the story, and work out what the shape of the set needed to be,” says Wright. “Google also encouraged us to start viewing the story on device as early as possible. So it was a combination of all these ideas that finally helped us make something cohesive, and most importantly, entertaining.”

With the success of Special Delivery, Google’s new goal is to scale to hundreds of millions of more devices by bringing the interactive experience to YouTube on iOS and to more Android devices.

“We developed a platform, format and SDK to enable more mobile and VR storytelling,” says El Guerrab. “And eventually, we intend to open up our SDK to more creators, who will push the toolset further, while we work to bring them to the largest audience in the world via YouTube. We will also open the SDK up to toolmakers so that they can create new or adapt existing tools for all types of filmmakers and creators.”

The innovation comes as new VR headsets are set to reach more consumers in 2016 (Oculus from Facebook in April, HTC Vive and PlayStation VR later this year), while other major companies including Apple and Activision look to develop their own VR divisions.

Notably, Oculus Story Studio launched its first original film to use real-time rendering at this year’s Sundance Film Festival. The animated film, Dear Angelica, gives headset viewers different experiences depending on how they move.

“This is an exciting space to be involved in. Companies are all experimenting to understand what works and doesn’t work in this new world,” says El Guerrab.

For Aardman, which is currently developing multiple projects in the VR space, Wright sees lots of opportunities for storytelling to develop as VR technology becomes increasingly more intuitive.

“At the moment this type of entertainment is very much an individual experience and it will be interesting to see if it develops in a more social way so we can maybe meet others in the VR space in the way that games do,” she says. “There’s a lot to play for and we are just getting started.”