728 x 90
728 x 90

Mixed Reality: Enjoy the View

Mixed Reality: Enjoy the View

For hundreds of years, people learned anatomy from books. Now, goggles and other mixed reality devices enable us to see particular elements of the human body like real objects.

Holograms are complex images

You may have already seen the hologram of Princess Leia from “Star Wars: Episode IV” on TV. But did you know that the first holograms were created over 70 years ago? Holograms went unused for many years, like pieces of science fiction (sci-fi) in books and movies. To create a hologram, one needs light and special equipment, like goggles. And, while old photos or digital images have two dimensions (height and width), a 3D image requires an additional parameter, like depth. Other parameters, like color, brightness, or contrast, can make the picture look even more real. 

How does vision work?

More precise than digital cameras and swifter than computers, the eye is the most specialized detector of light we have. Instead of electronic components, the eye is equipped with tiny tissues, called rods and cones, that respond to light, intensity, color, and motion. After entering the eye, light passes through the lens and reaches the retina where millions of light-sensitive cells send an upside-down, inverted image to the brain, where complex analyses happen quickly. Finally, we see the image, without even noticing the processes it just went through.

Knowing the nature of vision and mirroring it, scientists designed special goggles. Tiny mirrors are placed at its internal parts and a miniaturized projector displays the image onto the surface to view. That surface is covered with small mirrors bouncing off the light to our eyes, and also to a camera. Some of them are double side mirrors, splitting the ray of light, and some only reflect it. These mirrors are placed in different positions, so the goggle surface is not flat. Our eye receives many rays of light reflected in various positions and our brain interprets it as a 3D real image. Amazing, isn’t it? 

At that moment, the journey into sci-fi becomes real. We may see everything that we want, increase and decrease images, rotate geometric shapes, and explore the digital world.

Mixed Reality – what it is and what can we do with it?

Three-dimensional visualization, also called Mixed Reality (MR), is a combination of real-world and computer-generated data and their mutual interaction. You can overlay any 3D image to the real environment in an interactive way (Figure 1). You can not only display a red ball, like in the real world, but also play with it. Mixed Reality brings users to a completely synthetic reality, one where they are no longer limited to a passive interaction with the digital world. All they need is special goggles or any object that displays mixed realities, such as screens, smartphones, head-mounted displays, or special eye-lenses.

The technology was first used in the military for real-time overlaying for fighter-jet pilots and display of the spatial orientation data. MR was also widely used for entertainment, such as subtitles in movies or lyrics in karaoke. The first head-mounted display was created in 1968 and since this time it evolved to Mixed Reality equipment. Now, it is an integral part of the modern game’s word. Within just a smartphone, the user can catch weird creatures to complete the collection, create the quest routes, or run the virtual drone. With that technology, we can display visual information, so Mixed Reality can also bring education to a higher level.

Mixed Reality in medicine education

The body is the most frequently portrayed object in history. Medical doctors are supposed to know its entire anatomy, from its structure, organs, tissues, and cells to the chemical processes that keep it functioning. For many years, until these days, this knowledge came from books. Thanks to the development of technology, books are transferred to another reality and, more and more often, MR is being used for education.

Figure 2. What you see inside the glasses. The person using gesture manipulations changes the 3D object in the space like skull skeleton, cheek, and teeth, muscles, circulatory and lymphatic system.

Within Mixed Reality, students can see how joints work in various positions of the body, observe the lungs during breathing and watch and listen to heartbeats. Every single element of our body can be presented digitally and displayed like a real 3D image. In MR, the body is not only visualized; in the form of the holographic presentation, we may, for example, rotate the position of the heart and even change its beat. Imagine that you can see your brain, its neural connections and skull skeleton, cheek and teeth, the circulatory and lymphatic system presented at anatomy lectures, and interact with those systems. Even baby delivery or complicated surgeries can be presented in a hologram. 

In a Covid-19 era, when remote, interesting educational resources are needed, MR can break barriers. It can be used in the learning process with people in the entire world, inside and outside education and science communities.


This article was the joint work of Klaudia Proniewska (Jagiellonian University Medical College), Agnieszka Pregowska (Institute of Fundamental Technological Research Polish Academy of Sciences), and Magdalena Osial (University of Warsaw).