Immersive Experiences
I produced content for an educational experience designed for an Egyptian tomb set.
Additionally, I designed a collaborative game produced for the CAVE where players communicated with each other to traverse an underwater environment. The story involved mythology surrounding Merlin and finding his spellbook which I modeled in MAYA. I also designed the layout of the 3D environment in UNITY.
Slay the Runway
Team Size: 4 (Zheng 'Simon' Diao, Xuefan Zhou, Qianye 'Renee' Mei)
Duration: 1 week
Platform: Microsoft Kinect
Slay the Runway was an Extended Reality [XR] experience built for the Microsoft Kinect platform. The experience was designed and built in a one-week rapid prototype and iteration process for the Building Virtual Worlds course at Carnegie Mellon's Entertainment Technology Center.
As the appointed leader for a team of four, I provided the overall vision for an experience utilizing the Microsoft Kinetic platform. Rhythm-based movement games have been designed before, but I imagined an experience where participants would pose down the runway in time with catchy music at 120 bpm.
A major challenge the team had was to overcome how to give the guest clues as to what pose they had to perform next. In addition to giving them a chance to practice the poses "behind the curtain" / backstage (where no scoring would occur), we gave them cue cards in the corner of the screen. We captured their real-time image with an overlay of the next pose, so they could be as accurate as possible. We also gave a subtle hint of what was coming up by having the "model" in front of them perform the next pose they would be asked to do.
In addition to maintaining the artistic vision, I developed the audio assets for the experience. To give some variance from existing rhythm based movement games, we hid surprises throughout the experience. This included generating a flurry of bananas when the guest performed a "monkey" pose and a surprise freestyle moment at the end which would also capture images of the guest.
VIDEO LINK
Egyptian Time Machine
Team Size: 5 (Ruili Tang, Trisha Surve, Guande 'Patrick' Lyu, Wonjae Kim)
Duration: 3 weeks
Platform: CAVE [MR] / [XR]
Developed for the CAVE platform with a motion equipped base, I produced and directed a location-based experience for the Building Virtual Worlds course at Carnegie Mellon's Entertainment Technology Center. The experience was designed and built in a two-week rapid prototype and iteration process.
Guests would enter the CAVE and approach a "time machine" which they would control. A mad scientist would guide them through a tour of an Egyptian tomb only as a ploy to trap the guests in the tomb. In a twist, the scientist would leave the room only to enter the digital environment.
I worked with the artists and programmers to build an interactive location-based experience where the guests would have agency by controlling the motion floor up, down, left, and right. The guests would also be given a "flashlight" where they could use the controller to shine a light in the digital environment to destroy enemies.
I built the physical time machine prop, produced the audio, and maintained the production & creative vision for the project.
VIDEO LINK
The Gallery
Team Size: 5 (Ruili Tang, Mohan Bai, Qianye 'Renee' Mei, Xuejun Wang)
Duration: 2 weeks
Platform: HTC Vive [VR] / [MR]
The Gallery was a Virtual Reality experience built for the HTC Vive platform. It was a location-based experience with the virtual space mapped 1:1 with a physical themed room.
As Creative Director, Producer, & Designer; I led the overall vision for the Virtual Reality experience. Part of the Building Virtual Worlds coursework at Carnegie Mellon's Entertainment Technology Center, the experience was designed and built in a two-week rapid prototype and iteration process.
I envisioned an experience in a digital environment where guests could interact with and walk into famous works of art. With my background in performance art, I wanted an experience where guests could have agency to traverse and locomote throughout the environment to "fix" paintings and interact with the artwork.
In the final activation, guests would enter the room themed to a museum gallery. Two paintings were prominently displayed: Birth of Venus by Sandro Botticelli and Mae West's Face which May be Used as a Surrealist Apartment by Salvador Dali.
Guests would put on the HTC Vive Headset and be transported to a virtual environment matching the physical one they are standing in. The lights flicker and the paintings are now awry. Guests can transport into the works of art and through a series of interactions are able to "fix" the paintings as they come alive in 3D.
The experience was very well received and showcased at the Entertainment Technology Center's Festival. I ran the production for the experience, making sure the environments matched by facilitating work between the artists, programmers, and ETC building staff. Additionally, I built props and was responsible for the audio in the activation.
The project also received press, which can be found here; And on my Press page.
VIDEO LINK (original build with audio)
VIDEO LINK (BVW Festival Composite video / no audio)
Children's Museum of Pittsburgh
Team Size: 5 (Daryl Choa, Min Pan, Weidi Tang, Jinyi Ye)
Duration: 15 Weeks
Platform: Physical Installation & Digital 3D environment [XR] / [MR]
Client: Children's Museum of Pittsburgh - Pittsburgh, PA
One Small Act was a collaboration between the Children's Museum of Pittsburgh and Carnegie Mellon's Entertainment Technology Center. The Museum has an initiative based on having children practice Qualities of Kindness. As part fo this initiative, they are retrofitting their current "Attic" exhibit Space into a new 1,200 square foot gallery loosely based on Fred Roger's Neighborhood and Kindness. The museum tasked my team to envision a new installation for this exhibit.
Playtesting with paper prototypes
As Creative Director and Producer for my team, I held the creative vision for the project while being a conduit of communication between the team and client. I also lent my expertise in design to shape the experience for guests.
Although the museum initially wanted a Virtual Reality experience; with the target demographic being children 0 - 12 (really 5 - 8), we quickly determined that would not be a compelling experience especially given the directive from the client that the installation should spark dialogue between caregiver and child.
We wanted the experience to be communal, so landed on a Mixed Reality experience combing physical props with digital elements. The inspiration was based on the imagery of a stranger holding an umbrella to shelter someone from the rain. We used this as a starting off point and quickly imagined a scenario where sheltering animals from the rain would be a compelling experience for children.
The installation will utilize a large oversized prop umbrella on a linear track that is matched 1:1 with a digital umbrella that the guest will control to enact qualities of kindness. Since we cannot control the entry and exit points of guests in the museum; The experience is non-linear, with a sunny day to rainy day cycle on a loop.
I facilitated the communication between the programmer and the fabricator at the museum to make sure installation will be seamless. I kept the production on track while carrying the creative vision and making sure all of the digital and physical elements fit our installation and the exhibit as a whole.
Prototype setup at the ETC with VR tracker attached to an umbrella
I procured the technology to playtest the experience using VR tracking hardware and short-throw projectors and utilized my design skills to create documentation showcasing all of the elements. I maintained the Google Drive, ran meetings, kept the budget and timeline in check, and developed all of the audio for the experience.
For information on the project, please visit the project website:
One Small Act
VIDEO LINK