Concept
Medusa is a VR play about cultural identity and adapting to changing times from a Croatian/British perspective.
The inception of the story came about from a recent phenomena on the Croatian coast of a new invasive jellyfish species appearing in large numbers.
Medusa isn't just the Gorgon of Greek legend. In the Croatian language as in many European languages, the word Medusa means jellyfish.
Initial Info
Character Concepts
Russel (Icarus) Krill
.png)
Zlatan Viskovici
.png)
Phoebe Brikhelhoff
Kevin Odinburg
Olivia Auntuandidos
Ray, Harry and Hausen
Musical Motifs
I wanted to portray Russel's journey with his identity through the musical motifs. The music alternates between D minor – the key used for a Slavic classic waltz - to D# major, which is Golden Era Hollywood jazz. It is in a 12-time signature that alternates from ¾ beats per bar (a waltz tempo) and 4/4 beats per bar (a more accessible tempo).
Writing
Writing for VR:
-
Rather than writing the script like a screenplay - I've opted for a traditional theatre approach.
-
Each scene has a diagram of the layout of the set to visualise and utilise the immersive aspect of VR to its fullest.
-
Scene transitions are specifically suited for VR. Points of interest (swirling lines) are used to streamline scene transitions and guide the viewer's view in an unobtrusive way.
-
To avoid motion sickness, each scene is constructed with a stationary viewer in mind. The scenes occur around the viewer, rather than moving the viewer through the scene.
-
I wanted to start the piece with a fish-out-of-water-experience - placing the audience behind the curtains of a theatre stage along with the protaganist.
-
A large emphasis is placed on the stage directions. With VR being a fully immersive medium, the practical aspects of sets, scenic changes and lighting all contribute to the overall emotional experience.
-
In this scene, the main visual motif is that of the sliding stage doors: Once closed, the audience is behind the stage seeing the backstage magic at work, but once open, the audience is placed in the same position as the protaganist - alone in front of a massive audience. A single space that is reconfigured to create two very different but symbiotic atmospheres.
-
The transition from Scene 1 to Scene 2 utilises the film technique of the jump cut. Russel slumping back on stage only to land on his sofa back home serves two purposes:
-
A stationary point of interest. Since Russel remains in the same place as the environment changes around him, the viewer is given a fixed point of interest to anchor the transition.
-
From the heights to the lows. Emotionally, I wanted to replicate the uncanny feeling of being on stage to returning home alone all within a single day. I wanted to make it clear from the beginning that the tone of this piece is different from most movie's depictions of creative success. Here, rather than ending with success, I'm starting with the traditional depiction of 'success' only to deconstruct the concept as the story continues.
-
Full Script
Scale:
Once I finished the script, the full run time was around 50 mins, far too long to realistically achieve by myself. Hence, I've decided to create a single scene, Act 2 Scene 5.
This is what I consider to be the heart of the piece - a conversation between Russel (a British/Croatian actor who passes for English) and Zlatan (a Croatian actor who has had to adapt in order to find and retain work).
My intention with this scene is to depict both the Croatian philosophy as regards art and work against the romantisised Western approach. Working for art or for survival - and the compromise we must make between artistic integrity and financial wellbeing.
Language:
I chose to have this scene in the Croatian language because it felt more authentic. English is the universal language of business, but Croatian is a language spoken by just 4 million people.
Using it made the scene more intimate and idiosyncratic, and it captured a tone of conversation that people just don’t have in English.
These two actors are Croatian, even if one passes for English, and when they are speaking in their native language, the audience gets a sense of what it feels like to live in a linguistic/cultural limbo.
Length:
I decided to have a segment that was around 10-20 minutes in length because from my own experience, I have found that this is the best length of time for newcomers to VR experiences.
This gives them the opportunity to acclimate to their environment but it is not so long that it causes cyber sickness.
3D Modelling
This piece explores what it means to be foreign in an adapting industry where film and theatre are transitioning to VR.
I wanted to portray the characters’ cultural identity visually through the design of the set and their puppets.
The full story starts in Britain, then moves to California where the locations include the ‘A-Studios’ soundstage, a hotel room, LAX airport and Venice Beach. One scene is set in the mythical island of Sarpedon, where the legendary Medusa was said to live, according to Hesiod.
However in the short extract, the piece shown is the exterior of the fictional ‘A-Studios” soundstage in California, where they are filming the ‘Medusa’ VR series.
To incorporate the themes of the full play, I designed the space using influences from Croatia, a country which sits at a cultural crossroads between Italy, Greece, Venice and other Slavic countries.
Croatia’s visual identity draws on many different cultures, but there are elements that are uniquely its own.
My aim was to create a look that was idiosyncratic to Croatia, but which incorporated some Californian or US references. The shape and layout of the buildings and set are based on the Warner Brothers' California sound stage.
Inside the sound stage, the buildings and waterfront draw on Croatia’s cultural heritage. They take references from ancient cities such as Split, Dubrovnik,Trogir, Zadar and Zagreb - popular locations for the US film industry. These were also used as locations for sword-and-sandals movies in the 1950's and 1960's.
Character Models
The clothes in this scene are inspired by Mocap suits. The ping pong balls on the heads are placed so that they could emulate a crest on a helmet, which Perseus might wear to fight Medusa.
Russel (Icarus) Krill
-
Character reference:
-
Unlike the concept drawings, the character reference has a very practical function.
-
This reference shows the character front on, 90 degrees to the side and from the back.
-
When 3D sculpting, this image is used as the primary reference for proportions and visual tone.
-
This is more of a blueprint for the character's design rather than an aestheic representation of the final product.
-
-
High Poly Sculpt:
-
Using a mesh with millions of faces, a high poly version of the character is made.
-
This stage of the process is most similar to sculpting with clay - however with the drawback of high computer processing power.
-
The high poly version of the model is the most detailed but it cannot be used for the final product until it is manually retopologised.
-
-
VR optimisation
-
Above is the high poly version in a VR scene. Whilst it was aesthetically pleasing while stationary, it ran at 12 fps (frames per second) in VR rather than the recommended 72 fps needed to avoid motion sickness.
-
This model had too much detail and needed to be scaled back.
-
-
To the left is my second attempt. This one was built with VR in mind and performed better, running twice as fast (24fps) but still not nearly fast enough.
-
Below is my concept for the opening scene in the theatre running within Blender. This ran even slower at 7fps due to the environment and multiple animations in place.
-
Up until this point, I was working within Blender, but it was clear that I had to transition onto a software better suited to VR - Unreal Engine 4.
-
-
3D modelling for UE4 (Unreal Engine 4)
-
In order to have a smooth performance, I decided to utilise textures to place extra detail on the characters without causing a massive hit to processing power.
-
To the left is the UV unwrap of the character puppet colour-coded texture map.
-
Below that is the hand-painted texture map of their motion capture suit (the costume both characters wear during the scene)
-
-
Placing textures onto low poly models
-
Below is the low poly retopology of the high poly sculpt with the texture maps applied.
-
This way the model can run a lot smoother and with this, I was finally able to have the character meet the required 72fps.
-
-
Normal Maps
-
Using normal maps, I was able to bake the details from the high poly model onto the low poly model.
-
To the left is the normal map for Russel's face.
-
Zlatan viskovici
-
As with Russel's model, I started with a high poly sculpt and baked the detail onto the low poly retopology.
-
Left is the high poly sculpt with the low poly retopology being built over it.
-
Below is the low poly model with the details of the high poly baked onto it as a normal map.
-
Since Zlatan's character is Russel's stunt double, I reused the same body mesh and placed Zlatan's face over it.
-
By placing custom textures maps, I gave him a different costume and hair style.
-
This way Zlatan felt visually distinct and different from Russel.
-
I wanted to represent visually that these were two people from very different worlds.
Texture and Decals
The decals and textures are a direct reference to Croatia’s cultural history. The main visual motif I play with is the relationship between Croatian and Greek culture.
I use the Greek key, Glagolitic script (the oldest Slavic script that is now only found in Croatia) and the Licitar, a traditional Croatian decoration used for religious celebrations.
Below are the hand-made normal maps I created for the stone textures based on these stimuli.
Set Models
-
Each prop was created individually on Blender before being placed into UE4.
-
This meant that every prop had to be drawn on paper first before the 3D modelling could begin.
-
For the coffee table and chairs, I had created a couple of variations before committing to my favourite design.
-
When making the models, it was important to have functionality in mind. A prop with less geometry takes up less computer RAM and hence I had to make up for this using detailed textures.
-
In order for these textures to align properly with the objects, I had to be very careful in my UV unwrapping (below)
-
The color grid below serves as an example of how a flat texture will look once it is wrapped around the object.
The coffee props are based on the coffee table and chairs from my local café, embellished to look more Croatian. The textures of the coffee cup and ashtray draw on the Greek key, the Licitar and Galgolitic script.
The coffee cart was placed in to celebrate the Mediterranean coffee culture that both Greek and Croatian culture share.
The cart has the sign 'Kafenio' (Greek for cafe) and has the decals of the Greek key.
The streetlights are based on those found in the Old Town of Zagreb, Croatia’s capital, where I grew up. Every evening, a lamplighter would come around and light the gas lamps with a long pole. Including these tall light sources was a way of creating an ambient light, and they reflected well in the water. I also added decals reminiscent of the Licitar to the silhouette of the light assets.
The palm tree is a common motif between both the Croatian and Californian Coast.
Originally, this prop was static, however after play testing the scene, I decided to place a very slight looping animating of the tree swaying.
The boat was based on Venetian gondolas and a Greek trireme. I included low lights on the boat at the front and rear. When the boat docked sideways in front of the actors, these acted as side profile lights, giving the actors a more dynamic profile. The actors were backlit by the same light that lights up the subtitles.
The building is based on the oval-topped sound stages in the Warner Brothers' Lot. Air conditioning units and pipes were added to break up the clean silhouette of the facade.
The overall silhouette of this structure, topped by a water tower, is a reference to the iconic Warner Brothers’ water tower. In this case, the top of the water tower revolves, which is a reference to the Lighthouse of Alexandria and the searchlights in the logo of 20th Century Fox.
The towering figure over the water is a reference to the Colossus of Rhodes as well as the giant Atlas statue that can be found outside the Rockefeller Centre in New York. The arches that he carries are based on arches that can be found in the palace of Roman Emperor Diocletian in Split, but also on the painting of the Tower of Babel by Pieter Breughel the Elder (the Tower of Babel being one of the earliest references to language barriers).
Jellyfish Model
_heic.png)
-
My first attempt at creating the jelly fish was to have their tentacles act as cloth that would drag behind them as they swim.
-
This however caused a lot of issues as the tentacles had little to no control and they often would clip through other parts of it's anatomy.
-
To summarise, I wasn't pleased with how it looked and it was nearly impossible to successfully animate.
-
I had to try a different design all together.
-
My second attempt was a lot more succesful.
-
Here I made jelly fish that swam like a squid/octopus.
-
This way, I was treating the tentacles as limbs with a skeleton that I could manually animate.
-
Whilst this very wasn't anatomically correct, its design was to make it look more regal and mystical.
-
This design complemented many of the other visuals in the scene.
Medusa Title
-
The proportions of the title are arranged using the Golden Ratio - a reference to classical Greek architecture.
-
Golden Ratio: a/b = (a + b)/a
-
Since this was a piece for VR, I decided to have the title of the play as 3D object that was physical in the Virtual World.
In-game Medusa Poster
-
However, an issue that arose was that, when viewed in VR, the moving poster alongside all the other moving props in the scene ended up feeling overwhelming.
-
I decided then that it was better not to have the poster in the final piece as it distracts from the scene rather than informing it.
-
Since I was only doing a single scene from the play, I wanted to incorporate a live poster for the fictional 'Medusa' project that the characters reference in their dialogue.
-
My plan was to have this video poster play behind the the characters as a billboard.
VR Development
With all the props and character designs ready, I was able to place them into UE4 (Unreal Engine 4). Here I could place the textures onto the models and arrange lights throughout the scene.
This was my earliest proof of concept, a functioning scene within VR.
The work however was far from over, I still had to record the dialogue, music, motion capture and subtitles.
The decision to incorporate a Venetian-style canal was a creative solution that guided the audience into the scene in a way in they wouldn’t feel motion sickness. It also added a very important thematic element - water. The emotional impact of this is far greater than that of an asphalt road, as per the original script. It added a mystical element and captured the spirit of the Croatian waterfront at night with the palm trees and the reflected lights.
Performance Capture
For this piece, I had the pleasure of working alongside Croatian actor: Daniel Topic (left).
In order to record the performance we recorded dialogue first. I used the dialogue as a reference for the motion capture. And lastly I used the audio as a reference for the music.
Unfortunately, we were unable to connect the suit to a laptop over wifi whilst Daniel was around, so I had to record the movements for both characters myself (right).
To make matters worse, I was unable to use the motion capture at all.
Whilst I was able to record and upload the animations onto the character models, there was no way of importing it into the game engine as the files were too large.
Above are the recordings of the motion capture performances. Below are the looping animations used in the final show.
CSM ALIVE SHOWCASE
Due to the nature of VR and the onboarding experience, I created a Menu room. This was a standalone island with a couple of props from the main scene that ran on a two-minute timer. In this time the audience could adjust their headsets and get used to the virtual environment. Once the timer was up, it would automatically take you through to the main scene where the performance takes place.
Technical Challenges:
In order to get the VR to work, I had to have a computer that could run the piece. However, I found out a month before the performance that the university would not be able to offer any computers for this.
My solution at the time was to see if I could make it work as a standalone Oculus App that didn't need a computer.
The main issue with this was that the Oculus hardware doesn't support the amount of detail that I had placed into the scene and hence when it was imported into the headsets there were many problems. Half of the textures were missing, lights were in the wrong place and worst of all, the frame rate was so slow, it caused major motion sickness.
The initial plan was to have four VR headsets running at the same time. However, after seeing the result of the Oculus App upload, that simply wouldn't have been possible. Instead, I chose to use my own computer. This was the machine that I had originally built this piece for and I knew with complete certainty that it would be the version of the play that I would want to share.
In conclusion, I chose to have only one functioning headset that wouldn't cause motion sickness, over four poorly running ones.
Below is footage of the performance running as an Oculus App.
Opening Night
Tech and environment:
I controlled lighting to match the colour scheme of the scene. White trackers were placed on the floor to calibrate the headset’s guardian. The space was being shared with another group who were using it as a changing room. The solution I found was to separate the space using a curtain.
Hygiene:
The headsets and controllers were wiped down and disinfected for each viewer. Having just one headset in use made this easier for me to achieve by myself.
The response:
The show had a slow start. At first, most people couldn’t find it due to how hidden away it was from the other shows running at the CSM Alive showcase. However, after my first few viewings, people that had watched the show returned with friends. I was then in a position where I had queues forming outside the performance area made up of people that had heard about the show through word of mouth.
Having one person at a time also gave me time to personally vet the viewers on the onboarding experience before trying it.
I had my computer monitor turned so that those waiting could see what the current viewer was seeing. I initially felt like this could act as a spoiler for those that hadn’t tried it yet – however, the music, dialogue, and the immersive aspect of wearing the headset made viewing it in VR a completely different experience. Instead, seeing the footage gave people a chance to get accustomed to what they were about to experience.
Whilst the viewer was sitting, I was able to answer all the questions that the audience had – both technical and artistic.
The atmosphere that formed from this was that of a group of people speaking in excited hushed tones, and overall made both the passive and active audience experience feel personal and intimate.
The response to the show itself was incredibly positive.
"Best thing I have ever seen in VR"
"Absolutely beautiful"
"As if you’ve invented film for the first time"
"I’ve never seen anything like this"
After the three days of the showcase, I had back-to-back viewings throughout. In the end I had around 50 people view the experience.
Below is a recording of the full experience – it shows both the 2-minute menu area where the viewer becomes accustomed to the VR scene and its functions, such as picking up props and looking around. This is followed by the full 7-minute performance.
Reflective Statement
In conclusion, this has been the biggest and most challenging project I’ve done by myself and there were many points throughout the production when it seemed like the show wouldn’t be possible. From the failure of the mocap suits, to taking out large amounts of content to streamline the experience, to cutting back from four headsets to just one – the entire process had left me feeling burnt out in the run up to the showcase.
However, after hearing the overwhelmingly positive response from the audience, it was clear that this piece was worth the effort. I got a chance to see the performance for the first time again through their eyes – and I was able to breath a sign of relief. Whilst I managed to get the tech to just about work – it was the immersive visual design, dialogue and music that captivated the audience.
This was my intention from the beginning. Not just to make a theatrical scene function in VR, but to have an engaging immersive work of art.
The audience only sees the final product. And whilst there was only one person viewing the experience at a time – I am glad that I chose quality over quantity.
In hindsight, I believe I would have benefitted from working with a team of technicians and artists alike. Whilst I knew how to create each individual element needed to make a VR play, I had overlooked the sheer workload needed to place it all together by one person alone. Working with a team could have mitigated this by allocating specific tasks between multiple specialised technicians and digital artists.
At the time I had started, I simply didn’t know any other people who knew how to use the programs that I was using. However, with more students showing interest in the new medium, I hope that we will see more students learning the programs needed for VR in the same way I did.
VR development is difficult, time intensive but not impossible. And when it comes off right, it can offer an emotive experience unlike any other. Throughout this process, I have learnt what works in VR and what doesn’t. Many technical elements can be streamlined and optimised into a more manageable workload and my own creative approach has developed as I went along.
I set out to make a VR play to showcase what a play made for VR could look like. I believe that I have met these criteria. Not only is this a VR play that functions, but more importantly, it is an immersive and emotive form of self-expression that I am deeply proud of.