Applying the use of different realities | Feature | University of Miami

CORAL GABLES, Fla. – At the Miller School of Medicine at the University of Miami, a medical resident is researching how augmented reality can result in more precise surgeries. At the School of Architecture, 3D models enhanced through virtual reality are bringing projects to life.

And, as UM forms the first building block in its an academic alliance with Magic Leap, the Florida-based technology company founded by UM alumnus Rony Abovitz, the College of Engineering will be creating a new lab that will explore the use of spatial computing in developing mixed reality techniques.

Many of UM schools and colleges are on the front lines of looking at how differing realities can improve lives and how we live. Here’s a look at what is happening across UM in implementing differing realities in teaching and research.

SCHOOL OF COMMUNICATION

The Department of Cinema and Interactive Media in the School of Communication is preparing a new generation of innovators and leaders. The curriculum at the department is bringing together students from different majors to learn about emerging technologies like virtual and augmented reality.

A new class offered this year, “Building Virtual Worlds” developed by Associate Professor Clay Ewing, explores the construction of virtual environments where students are responsible for creating a world that can be interacted with on various platforms, including virtual and mixed reality.

Another innovative class, taught by Research Associate Professor Ching-Hua Chuan and offered to both undergraduates and graduates provides students with the ability to design and develop augmented reality applications.

Due to the high volume of interest in these emerging realities, the department is creating a high-tech lab where students can develop mixed, augmented, and virtual reality projects and games. The studio is set to open in January.

“The extended reality studio will give our students access to the latest XR devices and high-end workstations in a dedicated space,” said Ewing. “As new technologies evolve, it’s important for developers to be able to design in an environment that allows for testing in the space they are being designed for.”

The School is now offering a new undergraduate Interactive Media major that equips students with design skills and a technological fluency that allows them to explore the expressive possibilities offered by emerging forms of communication.

“Starting a new program is difficult, but the thriving culture within our master’s program inspires people entering into our undergrad program to strive and create something bigger and better. There’s a lot of energy,” said Kim Grinfeder, founder and director for the Interactive Media Program. 

SCHOOL OF ARCHITECTURE

The School of Architecture this year opened a new Virtual Reality Lab within its research-oriented RAD-UM lab.

“Virtual, augmented, and mixed reality have been hyped for years but new technology is now delivering the promised goods,” said Rodolphe el-Khoury, dean of the School of Architecture and expert on the Internet of Things. “We are vigorously embracing the new tools and excited about exploring the new territory with faculty and students.”

Supervisor of the RAD Lab, Christopher Chung, believes virtual reality is a valuable tool in Architecture.

“Design is a big part of architecture. Traditionally you communicate your design through plans, sections, and perspective drawings; now we can create 3D models that help represent and convey your idea. Using VR, architects can actually build a 3D model and have clients enter the life-like space,” said Chung.

Virtual reality can also help the process of research when crafting a project. VR and AR have the potential of determining how a building can be affected by the sun, or even how wind patterns interact with a structure.

Chung is hopeful the technology can influence how future School of Architecture students conduct their end-of-term final review, a tradition that has long defined architectural education in North America.

“I believe it won’t be too far in the future where students can conduct their final review at SoA inside their building using VR where critics can virtually walk inside and tour the space,” said Chung.

SCHOOL OF NURSING AND HEALTH STUDIES

There is a growing interest in how virtual and augmented reality can influence the medical arena. The School of Nursing and Health Studies is actively exploring what the technologies have to offer through its Augmented Reality Studio and AR resources within the Simulation Hospital.

 “We are always finding ways to move away from textbooks and transitioning more into a visual concept of learning. When students visualize concepts, it helps them learn faster and apply the information successfully,” said Jeffrey Groom, associate dean of simulation programs.

Groom is confident one of augmented reality’s greater uses will be to teach students how to better conduct procedures.

“Right now people in a way guess where they need to insert a needle into a vein. With these goggles, students would be able to see the inner anatomy. It would help perfect these kind of procedures,” said Groom. “The challenge is to figure out how this technology can benefit education. It will take time to figure out its full capacity, but signing on with Magic Leap is a huge stepping stone,” said Groom.

MILLER SCHOOL OF MEDICINE

A chief resident at the Miller School of Medicine is experimenting with the use of augmented reality during surgery. Timur Urakov, who is currently a resident in Jackson Memorial Hospital’s Neurology Unit, is researching how AR could potentially help the success of future brain and spinal surgeries.

Urakov is in the preliminary stages of working with a software that is able to transform a patient’s MRI into a hologram. The hologram, which is seen through an augmented reality headset, could navigate the surgeon throughout a procedure to ensure a positive outcome.

“The hologram is very precise. It could show a doctor the structures within the body without even cutting inside. It also allows a person to see all the important areas of the brain and avoid damage,” said Urakov.

Right now several modalities like X-rays are used to visualize data before a surgery. Urakov hopes augmented reality can bring visualization to the next level.

“Think about Haiti and any third world country. They don’t have the expensive equipment needed for these high risk surgeries. With AR, any doctor could travel anyplace in the world with a set of AR goggles to conduct surgeries,” said Urakov.

Urakov plans to begin a validation study at the University of Miami in a few months where he hopes to put his technology to the test. Institutional Review Board Approval from UM would allow Urakov to bring the technology he’s been researching into an operating room for observation.

“Right now we’re first validating whether this is accurate enough to use in surgery. We have optimism that we will be able to test it in patients with tumors in the future.” he said.

SCHOOL OF EDUCATION AND HUMAN DEVELOPMENT

Faculty members at the School of Education and Human Development are researching how augmented reality can help examine a person’s executive function.

Professor Joseph Signorile, Assistant Professor Moataz Eltoukhy, and Ph.D. Candidate Jim Buskirk have developed a walking test that determines how fast a person reacts to a visual cue using augmented reality goggles. Traditional testing of executive function occurs with the subject reacting to visual cues by hitting arrows on a keyboard; however, many of the tasks that require executive function in our daily life, such as crossing a street, walking down a crowded sidewalk, or hitting a tennis ball, require us to react to visual stimuli with the entire body. Once incorporated into a virtual reality space, the test can be used by clinicians, such as physicians, physical therapists and psychologists in their own unique environments.

“The beauty of this technology is we can bring it into a doctor’s office, we can take it out to the street, or even bring it onto a field or gym for athletes,” said Signorile.

During the executive function test, a patient is asked to wear programmed augmented reality goggles. The subject is then instructed to react to a set of given cues that are seen through the lens. If the cue signals a specific movement, such as a green arrow pointing left, the software has the capacity to determine whether the person made the correct movement and how long it took to do so.

Eltoukhy, Buskirk and Signorile hope their research can be used in a clinical environment in the future to treat a wide variety of patients who suffer from diseases that affect cognition.

“So many people can benefit from this. There is no end to what we can do. If we can achieve successful testing, it will thrive and move into a clinical setting,” said Eltoukhy.

The team also believes their test can be used as a rehabilitation tool.

“The implications here are really wide. We can use it to train people to make their reaction time faster and better so they make the right decisions,” said Buskirk.

Clinical trials are expected to get underway sometime next year. They hope the trials will provide them with the data they need to take their product to the next level.

Main photo: T.J. Lievonen/University of Miami