

Project
ESC_room
Exploring the interaction between virtual and physical environments. Seeing what effect it can have on collaboration and problem solving.
For a class in 3D interaction during my master's, we were tasked to design and implement an interactive installation in Unity. With access to many different technologies within the research center CAVI there were plenty of directions and technologies to explore. My group and I approached the task wanting to explore the interaction that can occur between virtual and physical spaces and furthermore, how that plays into the user's interaction and experience as well.
Overview
Problem
Exploring how interaction across virtual and physical environments can be used to and influences collaboration and problem solving.
Outcome
We created an escape-room-type game taking place in a virtual studio using both physical and virtual object clues. The catch being one person can see the virtual world the other cannot.
Role
Developer
UX designer
Timeline
Spring 2023
Tools
Unity
C# code
Virtual Reality Studio
Team
Helene Boeriis
Sophia McCulloch
Jonas Paaske Ditlevsen
Jane Clausen
End result
The end result was an escape room placed in the virtual studio and the next-door control room. One person was placed in each space and together they had to solve quests and find clues.
Control room player had access to a keyboard to write codes and the virtual studio player interacted with physical objects + physical turned virtual objects.
Get your footing
How to start a project when the brief is "just don't only use a mouse and keyboard"
We started this project wanting to explore and use the existing technologies in CAVI in a new and different way to how they have previously been used. These steps kicked us in a direction.
Technology knowledge
Throughout the semester we were introduced to many different technologies, development tools and platforms and how CAVI has used them for different research projects, events and artworks. This was our knowledge base for starting our own project.
Think aloud brainstorm
As we knew the limits aka the exact technologies available to us, we wrote them all down alongside plenty of activities/interactions. Each of us picked a random technology and activity and thought aloud about ideas and immediate thoughts on this combination which evolved into conversation of possibilities.
Inspiration
Taking inspiration from previous projects helped ideation too. Especially a collaboration game using a VR headset and an interaction table was a source of inspiration as the players had different knowledge of the game depending on what technology they were using.
Each player having a different role, opportunities to interact and knowledge of the game was a concept we wanted to incorporate to our experience as well.
THE BLUE ROOM
The virtual studio makes it possible to combine physical objects with digital 3D-objeccts in a virtual scene.
Insigths
The blue room and its virtual counterpart
Based on the brainstorm and continuos talks of possibilities we chose to use the virtual studio aka 'the blue room' as our backdrop to our interactive experience.
The virtual studio
The studio is set up as a development tool which means it is not built for an ESC-room experience. This means the solution has to be a mixture of the existing setup and new solutions in the studio.
Two worlds collide
The experience of being in the real world, the virtual studio with blue walls and objects while simultaneously being in a virtual world with a story and intended to capture your attention.
Collaboration
Having one person only experiencing the virtual world and one person experiencing both worlds (by being in the physical room with a screen with the virtual room in it), which effect it has on collaboration and problem solving.
Embodied interactions
How does users interact with and experience not only the studio but also the studio through the lens of the virtual room.
Mapping it out
Interactions between virtual space, reality and humans
Using the Virtual Studio as our point of entry together with the concept of an escape room with players having different roles, the result was as follows:
The blue room was the physical space wherein one person interacted directly with the environment. Through a screen this person also had a visual of the virtual space that they inhabited.
The control room gave another player access to the controls. They only had the view of the virtual world created in Unity. They could interact in the game via the keyboard.
Movable objects and gamification. Because of the setup of the virtual studio it was possible to combine physical objects with digital 3D-objects. Sensors made it possible for the person in the studio to move a physical object resulting in something happening in the virtual scene.
A storyline and roles. To make the experience more immersive and encourage collaboration a storyline and roles was set in place.
The two players communicate through microphones.
Test results
Virtual or reality?
With only two tests nothing can really be concluded but it was insightful in terms of behaviour and the perception of the prototype.
Scattered embodiment
Behaviour in strange places
Despite only testing two groups their approach in the virtual studio was vastly different. One was very curious and moving objects around while the other was much more cautious and needed encouragement to engage with the physical space. This player explained he felt very aware of the blue room and being afraid of breaking anything.
Embodiment supports communication
Having an embodied experience seems to be much easier for the control room player, as their perspective was limited to viewing the combined virtual and physical space through a screen.
This immersive viewpoint reinforced the storyline and encouraged clear communication. One tester noted, "I am the brain and he is the hands—we need each other to solve the tasks," highlighting how embodiment can drive collaborative dialogue.
Not knowing and sensing encourages communication
Final thoughts
While this experimental play on virtual and physical interactions is hard to replicate in a commercial sense, it brings inspiration, new perspectives and understanding on embodiment in new settings.
Iteration and learning from others unlock new possibilities.
While this page focused more on the interactions, physical versus virtual and the many different technologies at play a lot of iterations were needed in effort to make the technology work our way. Working with technology in new ways underlines the need for iterative testing, even when solutions are not yet fully formed. Trial and error kept us moving forward, problem solve and think in new ways resulting in deeper learning and practicing agile work flows.
How could this be done without the blue room?
An interesting thing in working with technology that is custom made for a space is thinking of how this experience could transfer into a more accessible format. While it would change the embodiment of the experiment, AR (augmented reality) where virtual objects can be placed in your cameraview and mix it with object recognition I think a similar escape-room concept could be achieved. Collaboration can then be created through multi-player functions and playing the same AR-game having different roles and informations.
Enabling novel interactions and insights.











