Project

ESC_room

Exploring the interaction between virtual and physical environments. Seeing what effect it can have on collaboration and problem solving.

For a class in 3D interaction during my master's, we were tasked to design and implement an interactive installation in Unity. With access to many different technologies within the research center CAVI there were plenty of directions and technologies to explore. My group and I approached the task wanting to explore the interaction that can occur between virtual and physical spaces and furthermore, how that plays into the user's interaction and experience as well.

Overview

Problem

Exploring how interaction across virtual and physical environments can be used to and influences collaboration and problem solving.

Outcome

We created an escape-room-type game taking place in a virtual studio using both physical and virtual object clues. The catch being one person can see the virtual world the other cannot.

Role

Developer

UX designer

Timeline

Spring 2023

Tools

Unity

C# code

Virtual Reality Studio

Team

Helene Boeriis

Sophia McCulloch

Jonas Paaske Ditlevsen

Jane Clausen

End result

The end result was an escape room placed in the virtual studio and the next-door control room. One person was placed in each space and together they had to solve quests and find clues.

Control room player had access to a keyboard to write codes and the virtual studio player interacted with physical objects + physical turned virtual objects.

Get your footing

How to start a project when the brief is "just don't only use a mouse and keyboard"

We started this project wanting to explore and use the existing technologies in CAVI in a new and different way to how they have previously been used. These steps kicked us in a direction.

Technology knowledge

Throughout the semester we were introduced to many different technologies, development tools and platforms and how CAVI has used them for different research projects, events and artworks. This was our knowledge base for starting our own project.

Think aloud brainstorm

As we knew the limits aka the exact technologies available to us, we wrote them all down alongside plenty of activities/interactions. Each of us picked a random technology and activity and thought aloud about ideas and immediate thoughts on this combination which evolved into conversation of possibilities.

Inspiration

Taking inspiration from previous projects helped ideation too. Especially a collaboration game using a VR headset and an interaction table was a source of inspiration as the players had different knowledge of the game depending on what technology they were using.
Each player having a different role, opportunities to interact and knowledge of the game was a concept we wanted to incorporate to our experience as well.

THE BLUE ROOM

The virtual studio makes it possible to combine physical objects with digital 3D-objeccts in a virtual scene.

Insigths

The blue room and its virtual counterpart

Based on the brainstorm and continuos talks of possibilities we chose to use the virtual studio aka 'the blue room' as our backdrop to our interactive experience.

The virtual studio

The studio is set up as a development tool which means it is not built for an ESC-room experience. This means the solution has to be a mixture of the existing setup and new solutions in the studio.

Two worlds collide

The experience of being in the real world, the virtual studio with blue walls and objects while simultaneously being in a virtual world with a story and intended to capture your attention.

Collaboration

Having one person only experiencing the virtual world and one person experiencing both worlds (by being in the physical room with a screen with the virtual room in it), which effect it has on collaboration and problem solving.

Embodied interactions

How does users interact with and experience not only the studio but also the studio through the lens of the virtual room.

How is the Virtual Studio with its embodiment and the virtual environment with its story experienced? How does it have an impact on collaboration and problem solving?

How is the Virtual Studio with its embodiment and the virtual environment with its story experienced? How does it have an impact on collaboration and problem solving?

Mapping it out

Interactions between virtual space, reality and humans

Using the Virtual Studio as our point of entry together with the concept of an escape room with players having different roles, the result was as follows:

The blue room was the physical space wherein one person interacted directly with the environment. Through a screen this person also had a visual of the virtual space that they inhabited.

room. cobalt blue walls and floor. Big boxes placed like a four steps are also cobalt blue. On the left there is a TV-screen showing the room in real-time but with the virtual version.
room. cobalt blue walls and floor. Big boxes placed like a four steps are also cobalt blue. On the left there is a TV-screen showing the room in real-time but with the virtual version.
room. cobalt blue walls and floor. Big boxes placed like a four steps are also cobalt blue. On the left there is a TV-screen showing the room in real-time but with the virtual version.
A dark room with big windows looking into the blue room. There are three PC-screens with different views of the blue room. A person is sitting at the desk with screens.
A dark room with big windows looking into the blue room. There are three PC-screens with different views of the blue room. A person is sitting at the desk with screens.
A dark room with big windows looking into the blue room. There are three PC-screens with different views of the blue room. A person is sitting at the desk with screens.

The control room gave another player access to the controls. They only had the view of the virtual world created in Unity. They could interact in the game via the keyboard.

Movable objects and gamification. Because of the setup of the virtual studio it was possible to combine physical objects with digital 3D-objects. Sensors made it possible for the person in the studio to move a physical object resulting in something happening in the virtual scene.

view of a computer screen. It shows the virtual world, where a person is holding a physical small object that in the virtual world has turned into a big cylinder
view of a computer screen. It shows the virtual world, where a person is holding a physical small object that in the virtual world has turned into a big cylinder
view of a computer screen. It shows the virtual world, where a person is holding a physical small object that in the virtual world has turned into a big cylinder

A storyline and roles. To make the experience more immersive and encourage collaboration a storyline and roles was set in place.

The two players communicate through microphones.

Test results

Virtual or reality?

With only two tests nothing can really be concluded but it was insightful in terms of behaviour and the perception of the prototype.

Scattered embodiment

While embodiment normally shapes your understanding and interactions in a space the test players in the virtual studio felt divided in their embodiment because of the setup. The continuous shift between focusing on the virtual world on the screen and the physical environment created a sense of disconnection to both. This seems to create more distance and less embodied experience.

I found it really gratifying to be able to explore and use design as part of the business strategy. I learned how powerful design can be as a strategic tool wherein not just digital products but also workflows and values can be altered for a better overall experience through design thinking.

Behaviour in strange places

Despite only testing two groups their approach in the virtual studio was vastly different. One was very curious and moving objects around while the other was much more cautious and needed encouragement to engage with the physical space. This player explained he felt very aware of the blue room and being afraid of breaking anything.

Embodiment supports communication

Having an embodied experience seems to be much easier for the control room player, as their perspective was limited to viewing the combined virtual and physical space through a screen.

This immersive viewpoint reinforced the storyline and encouraged clear communication. One tester noted, "I am the brain and he is the hands—we need each other to solve the tasks," highlighting how embodiment can drive collaborative dialogue.

Not knowing and sensing encourages communication

Being in two very different worlds encourages the players in being curious of the information each of them have. This resulted in over-explaining elements, as they do not know if it is relevant for the task or not.   

Final thoughts

The value of experimentation

The value of experimentation

While this experimental play on virtual and physical interactions is hard to replicate in a commercial sense, it brings inspiration, new perspectives and understanding on embodiment in new settings.

Iteration and learning from others unlock new possibilities.

While this page focused more on the interactions, physical versus virtual and the many different technologies at play a lot of iterations were needed in effort to make the technology work our way. Working with technology in new ways underlines the need for iterative testing, even when solutions are not yet fully formed. Trial and error kept us moving forward, problem solve and think in new ways resulting in deeper learning and practicing agile work flows.

How could this be done without the blue room?

An interesting thing in working with technology that is custom made for a space is thinking of how this experience could transfer into a more accessible format. While it would change the embodiment of the experiment, AR (augmented reality) where virtual objects can be placed in your cameraview and mix it with object recognition I think a similar escape-room concept could be achieved. Collaboration can then be created through multi-player functions and playing the same AR-game having different roles and informations.

Enabling novel interactions and insights.

The ESC_room projects, shows how experimentation opens pathways to explore unconventional combinations of technology and the play on physical and virtual elements. By default this also unlocks new forms of user interaction and experiences which challenges the understanding on how users can collaborate.

I found it really gratifying to be able to explore and use design as part of the business strategy. I learned how powerful design can be as a strategic tool wherein not just digital products but also workflows and values can be altered for a better overall experience through design thinking.

Thanks for visiting
my portfolio

Thanks for visiting
my portfolio

Thanks for visiting
my portfolio

Made with framer by Jane.

Made with framer by Jane.

Thanks for visiting
my portfolio

Made with framer by Jane.

Create a free website with Framer, the website builder loved by startups, designers and agencies.