Synesthesia Simulator was a virtual reality application that simulated three different variations of synesthesia, a syndrome that manifests as a crossover of multiple sensory experiences. I created this VR app using Unity, C#, and the Oculus Rift in the capstone project for CS 498 SL: Virtual Reality, a UIUC special-topics course that provided students with a deep understanding of the fundamentals of VR as well as practical experience in developing VR experiences.

My group created a synesthesia simulator that allowed users to experience three of the many types of synesthesia: chromesthesia, spatial-sequence, and grapheme-color. The user remained sitting with an Oculus Rift headset while making selections with an Xbox controller. To minimize vection and visually induced motion sickness we did not allow users to physically walk around the scenes. Ambient sounds played continuously throughout the application.

Inside the chromesthesia room, we placed the user in the center of a 360-degree video of a beach scene with photisms connected to the sounds of the waves and the birds flying overhead. For spatial-sequence synesthesia, we created a circle of months surrounding the viewer where each month has a specific location in space. The current month was closest to the viewer, while future months became physically further and further away. For grapheme-color synesthesia, we put the user inside a classroom and showed them a sequence of letters and numbers. The user could switch back and forth between colored and black text using their controller.

Collaborators

Doug Ebersol, Kevin Eicken, Kate Milleker