Sonaria was a one-of-a-kind musical & artistic VR experiment, the result of a collaboration between our studio and the Google Spotlight Stories team. A VR experience like no other, it could be called a musical meditation on nature, as well as an experiment in minimalism. Sonaria offers a rare peaceful escape for the mind, offering space for your imagination to explore the area between images and sound.

The original concept came from the mind of musician Scot Stafford, who also provided the musical backbone and immersive sound design for the film. Our team at Chromosphere then provided overall creative direction, story, art, and technical implementation in collaboration with the Google team.

KEVIN: Scot Stafford pitched his original concept for this experience to us, which in short was a musical VR journey following two creatures in a continuous chase through multiple environments. The other central piece of the idea was that the two creatures would also be morphing or transforming into other creatures along the way. When he presented the concept, Scot also had a little doodle of a teardrop shape which added three more teardrops and became a bird-like creature. I really latched onto the concept both because I like the simple nature of the teardrop shape and because it made me think of a graphic way of looking at the evolution of life. Treating the teardrop as though it is the singular building block of life in this world, like a molecule or a single-celled organism.

JASMIN: Because of the idea to build the world out of teardrops, we looked for inspiration from artists who love to use playful, simple shapes in their work. Two artists who ended up providing a lot of inspiration were Leo Lionni for the playful way in which he handled cut-out shapes, and Alexander Calder for his bright colors and emphasis on strong graphic shapes. The palette of the world began to develop with the idea that the two teardrop characters are red and blue, so our early exploration focused on these two bright teardrops against very minimal backgrounds. The designs quickly became more complex, and Nelson Boles did a lot of fantastic exploration to see how many creatures could be built from the teardrop shapes. However we eventually decided to simplify and thought it would be more appealing if there was a very concise method to determining the number of shapes we used for each creature. We used the same thinking for the environments, where we would isolate a single item or prop that best described the environment, such as lily pads for the marsh or kelp for the kelp forest, and focus on letting that element alone flesh out the scene.

JASMIN: For the transformations of the creatures, we initially imagined that they might multiply like bacteria in a petri dish, or maybe split apart like cells to become new creatures. As we explored further, we found inspiration in some naturally occuring patterns, like the Fibonacci spiral of some flowers. This gave us a clearer idea of what to do in the spaces between transitions, allowing for more interesting shapes to occur during the sequence.

KEVIN: Even though Sonaria wasn’t a traditional narrative, we still wanted to find some natural arc to the experience to help the journey tie together. There needed to be some sense of order that led us from one vignette to the next. Because of how the teardrop reminded us of the concept of building blocks of life, our initial thoughts had to do with tying things together through a kind of natural evolutionary chain.

JASMIN: Scot had told us from very early stages that he wanted to incorporate the sounds of whale calls, so another idea we considered was starting very small with the teardrops and then working up through larger and larger creatures until we reached the whale. Eventually Nelson had the really cool idea to string everything together in levels of elevation, starting at the bottom of the ocean and working our way up to the surface, into the trees and sky, before coming back to the ocean.

KEVIN: Once we had this concept for the arc in place, planning the story out became like an artistic choreography of the creatures’ movements, almost like we were planning a ballet. We had to find an interesting and elegant way to get the creatures from one place to the next, and also start to think through the details of how we would transform the sets.

JASMIN: For the first step in planning we came up with a 4-panel storyboard frame where each panel represented 90 degrees of vision in VR. This gave us the first ideas about how much physical space the creatures would be traversing, and we tried to stage each major point of action within a single quad to keep the movement from becoming too dizzying. Theresa projected these storyboards onto a 3D cylinder that we could time out and watch in VR to give a very rough sense of the experience. These initial tests helped us to realize that we were still forcing the viewer to move far too much to watch all of the action as we continually wrapped ourselves up in cords from the headset!

KEVIN: This also led to some more advanced set planning. We created these isometric set diagrams to help us better visualize where actions and set elements would be in 3D space.

KEVIN: After some trial and error with the flat storyboards, we decided it would be necessary to make a completely 3D layout to better visualize the timing and flow of the film in VR. Based on our storyboards and rough set layouts, Lucy was able to quickly model proxy versions of the sets & characters, and Alex Grigg spent a few days making them fly around in space. This gave us a much better sense of timing, and more importantly the closest idea so far of what it would be like to experience Sonaria in VR. After some more finessing and getting ourselves entangled in cords a few more times, we had a solid layout pass and could move on to animation!

The transformations of the creatures turned out to be particularly hard to animate since they involved so many teardrop shapes moving around and rearranging themselves into new creatures. Because of this we sometimes prototyped the transformations in 2D where it was easier to plan out the shapes we wanted to hit. The animators then had a drawing they could follow in 3D, and it also helped the tech team to make sure the characters’ rigs were set up to accommodate all of the motions of the transformation.

THERESA: A big technical challenge on Sonaria was in planning so that all of the sets, animation, and most importantly music, would all fit together seamlessly in the end. We knew that the creatures would be flowing between scenes in a continuous motion from on to the other, and this needed to happen without breaking the musical structure. This necessitated setting up a spreadsheet early on which charted out the exact frame numbers on which each major action would happen. Those actions also needed to happen on frames in multiples of 12 which was the pre-determined “beat” of the film. In addition, the viewer’s gaze would determine when important actions would happen so that you would not miss vital story moments. This meant that the characters would need to enter a holding pattern and then seamlessly come out of it once your gaze returned focus. This in turn meant that during the set transitions, the set would also need to exit from a looping animation and go into a transition animation as quickly as possible without a hitch. Of course, this careful coordination also all needed to function in real time!

LUCY: The idea for the set transitions was to smoothly hide and reveal environmental elements, transporting you to a brand new location. To do this I used a gradient shader which was first written by Cassidy Curtis from the GSS team, which allowed me to do things such as animate a color change on an object that would radiate from a certain point. We used this effect to reveal the new sky color and also to do things such as animate the kelp disappearing from the fish scene from the top down, almost as if the light from above was obscuring it.

THERESA: We also used the gradient shader to create the pulsing effect on the animals. One of the big technical challenges that had to do with the visuals was the fact that the film’s style utilized so many layers of transparency. Transparency is notoriously processor-intensive in real-time projects, so we had to figure out how to optimize this as well as working to minimize transparency sorting issues. To add to the mass of teardrops on screen at any one time, I also created a particle system to make teardrop trails coming off the creatures. It was a system which would generate teardrop shapes with controls for the size and speed, and was designed to be straightforward for the art team to use and place in the scenes wherever they wanted.