Tag: Medicine

My Notes from an XRHQ Live Streaming Event on LinkedIn and YouTube — Pixels & Pills: Breaking Research on Immersive Treatment for Mental Health, using the Apple Vision Pro and Explore POV (January 29th, 2026)

PLEASE NOTE: This is now a somewhat edited first draft of the notes I was frantically taking during this livestream, because I wanted to get the information out there on this very interesting application of the Apple Vision Pro! Yesterday I came across this announcement of how the Apple Vision Pro was being used in research to determine its effectiveness as a support for those suffering from anxiety and depression. As an avid AVP user, as a subscriber to Explore POV, and as a mental health consumer, I was definitely not going to miss this presentation, which was being streamed on LinkedIn (a first for me; usually I am on Microsoft Teams or Zoom for this sort of online event).

I was originally thinking I would go in using my AVP’s Virtual Display feature with my MacBook Pro (my usual work setup lately, what with my neck and shoulder pain), and then I thought: naaah, let’s not overcomplicate things. Apparently, this is also being streamed to YouTube, which I will look for later. UPDATE: Added the YouTube link at the end.

Any omissions and errors are my fault; sorry guys, I can only take notes so quickly!

The speakers in the livestream were:

Hala Darwish, Associate Professor, School of Nursing/Neurology/Neuroscience Graduate Program, University of Michigan (currently conducting research, still in its very earliest stages)

Jeremy Dalton, XRHQ (moderator of the event; formerly PwC Head of Immersive Technologies)

James Hustler, Explore POV (3D video creator, whose app was chosen as the Apple Vision Pro App of the Year 2025, https://exploreimmersive.com)

Event description: By immersing patients in breathtaking natural environments using the Apple Vision Pro, research is now underway to discover whether these experiences can support those suffering from anxiety and depression. Join James Hustler, creator of the award-winning Explore POV app, and Dr. Hala Darwish, Associate Professor at the University of Michigan, for a live discussion hosted by Jeremy Dalton from XRHQ. Together, we will explore the technology, the clinical thinking that inspired it, and what it could mean for the future of digital therapeutics.

(Unfortunately, I missed the first few minutes while I was fiddling with my sound settings, and trying to get my earbuds to work properly, so I missed Jeremy’s and Hala’s introductions.)

James Hustler travels the world to record amazing 3D videos and share them via his subscription service, Explore POV (which I have written about before here). He had been living in a motor home in New Zealand during the pandemic, when he had started recording 3D videos to share with friends.

Hala is in early stage research, interested in the relationship between mental health and the environment. Many people do not have access to certain environments (e.g. an urban environment with very little nature). Also, people can have access issues (e.g. a disability). Hala was looked into VR as an alternative to real-life nature experiences, and in 2019 when she started, the tech wasn’t quite ready (they tried with 360-degree videos, and she felt it didn’t really work well, i.e. low resolution; caused motion sickness, etc.). She then tried computer-generated nature graphics for patients with MS (multiple sclerosis). In 2023, the Apple Vision Pro was released, and Hala had a demo. The decision was made to switch from 360-degree VR video to 180-degree VR video.

James: The VR 360-degree video format is not new, but until recently, it hasn’t been at a high-enough resolution to create a true sense of presence. i.e., it changes from an intellectual response to an emotional response of being there. Explore POV is now recording at 16K resolution, and experimenting with Apple Immersive Video. The goal is to capture a scene so that the user feels like it’s lifelike and real to them.

Hala: transporting the individual to these natural environments does appear to have health benefits (mental and physical health, stress relief, etc.). In addition to anxiety and depression relief, Hala’s area of research, VR is also being used for the treatments of phobias (exposure therapy), performance anxiety (e.g. fear of public speaking), and as a method of pain management and distraction, among other uses.

James, when asked about feedback to his videos: Explore POV was created as a travel app, but people by the hundreds are contacting him about the mental and emotional response to the VR video scenes, telling him it’s the first time they’ve climbed a mountain or paddled a kayak. People have told James that they use the Explore POV app to relax after a stressful day’s work. This sort of feedback has opened James’ eyes to the possibilities of 3D video in VR. He had originally approached his work from a technical challenge (e.g. how do I create the highest-resolution 3D videos in VR?). He stressed that all these responses are anecdotal, but that we need scientific evidence.

Hala, in talking about her research: we want to run clinical trials (but we are currently testing feasibility and safety with a limited number of patients with progressive multiple sclerosis and depression). If we give AVPs to patients to use at home (e.g. with disability), how are they going to be able to use the headsets? The first study splits the patients into three groups. It’s a cross-over study: one group gets standard treatments first, then VR treatment, the second group gets VR treatment first, then standard treatments. The third group has just standard treatments, with no VR intervention. It is an early-stage feasibility study, with 14, 14, and 12 patients in the three categories of patients being looked at. She is also interested in researching longer-term responses to VR treatment.

Hala: in my opinion, exposure to natural scenes in VR appears to be a good adjunct to standard therapy. It’s still too early to come to any definitive conclusion. We first want to see if it has an impact on stress and anxiety levels, and then eventually expand to a larger number of patients (right now it’s a small number).

Which environments create greater impacts? James: we would expect to see what we’ve seen compared to previous academic research studies using real-life nature scenes (e.g. MRI brain scans after exposure to nature, e.g. taking a hike). There is already a good body of academic literature dealing with the impact of real-life nature on people’s anxiety and depression.

But we don’t have anything beyond anecdotal results for the use of nature in VR so far, nothing scientific; this research is still in its very earliest stages. For example, one early patient had a very good response to a desert environment (but it’s only a sample size of one!). James: if we’re aiming for calmness, certain VR video environments would probably help with that, e.g. flowing water, watching a sunset while sitting on a mountain, etc.). But again, at this point it’s purely anecdotal.

I asked a question in the text chat during the livestream that was actually asked of James, the creator of Explore POV, which was: Has James created specific VR video environments for Hala’s research? The answer was no; James has not yet created specific VR video environments for Hala’s research. However, they’ve now shot approximately 200 videos in 20 different countries so far for the Explore POV app (I think he said 200, but it was hard to take accurate notes!). He notes that they are a small, nimble team who can rapidly adjust to meet any requests from Hala’s research team, if needed in the future.

My question got asked!!

In response to a question from another user about the use of Apple’s SharePlay feature, where you can share an experience together with other Apple users via their Personas: James would love to add this feature, if he can. Yes, he would love to make Explore POV more of a multiplayer experience, if possible. He talks about people sequentially experiencing the same VR video in Explore POV, and thereby “sharing” the experience with others (e.g. a father and his daughter, if I remember correctly).

James: for people who can’t physically travel due to disability or for soke other reason, the technology is unlocking experiences that they might never experience otherwise. He thinks that it’s an amazing position to be in where we can give some of these people a taste of visiting remote places, with impacts in not just healthcare but also conservation, education, etc.

Hala: the academic research process is slow due to recruitment bottlenecks, but she estimates 2 years for the duration of the study (before results are published). he notes that most of the time, the people who most need the nature exposure do not have the opportunity to access it (for example they cannot afford an Apple Vision Pro).

(Unfortunately, Hala crashed out of the stream soon her comments, and the other two speakers wrapped it up!)

Conservation, education and healthcare are the three areas of what James wants to focus on with Explore Immersive. In addition to working with Hala on her research study, he’s also working on conservation and education applications as well. He hopes to start new partnerships in these three key areas, and wants to make Explore POV more than “just a travel app.”

Here’s the 53-minute YouTube video, in case you missed the livestream (unfortunately, you do have to actually go over and watch it on YouTube, as I am not allowed to embed it into my blogpost). Sorry! I do very strongly urge you to go over and watch it, though; it was amazing and inspiring.

Fisk University Creates a Virtual Human Cadaver Lab Using the ENGAGE Social VR Platform

Fisk University, a private, historically Black university located in Nashville, Tennessee, will launch a virtual human cadaver lab for its pre-med and biology students this fall. The cadaver laboratory will use the social VR platform ENGAGE, in a partnership with Fisk University, HTC VIVE, T-Mobile, and VictoryXR (an educational content creator company using ENGAGE as a platform).

According to the official news release:

Inside the lab, students will examine the internal organs of various human systems, and the professor can even remove the organs from the body and pass them around for students to hold and open. Students will have the ability to enlarge the organ to a size large enough where they can even step inside to better learn how it works. In addition to organ systems, the cadavers will also include complete skeletal and muscle structures.

“With this cadaver lab, our pre-med students will no longer need to rely on other universities for advanced anatomy and biology classes,” said Dr. Shirley Brown, Dean of Fisk University. “Virtual reality technology takes our university to a level equal to the most advanced schools in the country.”

In the past, Fisk University has not purchased cadavers due to the high cost and maintenance. But with a virtual cadaver lab, the university can offer state-of-the-art scientific learning that’s affordable and easy to maintain. Virtual cadavers do not degrade, and over time additional specialties can be added to the software such as surgical procedures, comparative learning between human and animal as well as microbiology at the cellular level.

Here’s a two-minute promotional video for the project:

Tony Vitillo (a.k.a. SkarredGhost), an Italian man whose blog, The Ghost Howls, often has reviews of products and interesting news reports about the VR industry, paid a visit to the virtual laboratory and reported:

The…costs to own a cadaver lab is in the order of magnitude of millions of dollars. Not all universities can afford that. There is at the moment a slightly better alternative, that is using ultra-realistic synthetic cadavers, that are also able to simulate some motions of the human body (e.g. the heart pumping), but the cost of each one of them is $60-100,000. This means that to own them a university must invest much money anyway.

We all know that virtual reality can replicate real objects pretty well, so VictoryXR had the idea of trying to reproduce a cadaver lab in virtual reality: apart from the fixed cost for the 3D elements, this laboratory would scale pretty well with the number of students and would need almost no maintenance cost. This is a very smart solution to make education more accessible for medicine students. Thanks to this, many more universities would be able to afford to have a virtual cadaver lab, even in non-wealthy countries. We always talk about VR being able to democratize education, and this is one bright example of how it can do that.

Students assemble a skeleton puzzle in Fisk University’s virtual human cadaver lab

Tony came away from his brief demo favourably impressed:

I had just a short demo with the virtual lab, and I think that it is a good start for Fisk University and VictoryXR. I don’t think that at the moment it can replace the real experience with a cadaver because you miss all the tactile sensations, the weight, and also the creep of having a real organ in your hands. But it can be a good substitute to start learning about the human body, to observe the organs in detail, to start getting confidence with having a bone or a part of the body of someone else in your hands. It could be able to offer a good course, and after that, maybe the students can have just a few final lessons with real corpses in another location. It is a good way of giving value to many medicine universities not only in the U.S. but in the whole world, especially the ones that can’t afford to have real or synthetic cadavers for tests.

What impressed me the most is the potential that this solution can have in the future. There are things that VR can give to students that are hardly possible in real life. The fact that you can enter with your teacher inside an organ and examine it both at macro and micro level is one amazing thing for instance. The possibility of organizing minigames (like the puzzle) that are engaging and improve the learning efficiency via interactivity is something that VR enables and that would be too creepy to do in real life. The possibility of doing many simulated surgeries on the cadavers with the possibility of repeating every operation at no additional cost is another cool thing. 

Studying the muscles of the human body in ENGAGE

Thanks to Chris Madsen/DeepRifter of ENGAGE for the heads up, and Tony Vitillo/SkarredGhost for his report and pictures! You can read Tony’s review in full here, and I strongly recommend you follow his blog as well as my own!

FundamentalVR’s Fundamental Surgery: A Brief Introduction to a Multimodal Virtual Reality Platform for Training Surgeons

Fundamental Surgery is a VR platform by a company called FundamentalVR, consisting of several components, and a sterling example of social VR used for a serious, practical purpose: the training of surgeons. The training program has been accredited by the American Academy of Orthopaedic Surgeons (AAOS) and the Royal College of Surgeons England.

Here’s a one-minute video overview of all five components of Fundamental Surgery:

The components are:

  • HapticVR: deep simulation and procedural surgical rehearsal with kinesthetic haptic feedback (you can see a bit of this in the first video up top);
  • @HomeVR: procedural walkthrough, anatomy and environment familiarization and testing, using a standalone VR headset;
  • Teaching Space: a social VR-based virtual training space (see image below);
  • Data Insights: a central data dashboard to track progress; and
  • MultiuserVR: a collaborative/social VR platform for surgical training (see video below).

This is an animated GIF demonstrating the Teaching Space, complete with a shared whiteboard:

VRScout reported on Fundamental Surgery last September:

Teaching Space [is] an unlimited multi-user virtual classroom designed to help medical schools around the world who’ve been impacted by the pandemic; a virtual space where they’re able to get that crucial hands-on training while working other students in a collaborative virtual environment.

COVID-19 has presented big challenges in the medical field when it comes to surgical training. In many cases, it has completely disrupted traditional training programs, which have always relied on actual face-to-face classroom environments. Zoom and Skype conferencing do provide alternative learning environments, but they’re limited. 2D platforms can’t fully replace the teaching and learning opportunities offered by in-class training.

This new VR learning space provides a safe environment for instructors to meet with trainees, no matter where they are located.

The virtual classroom environment includes a virtual whiteboard that instructors can use to present additional notes as they discuss procedures with their class. From there you can hop on over to Fundamental Surgery’s virtual operating room where you can run demos of surgeries and get even more hands-on experience. 

Here’s a short video showing you what the MultiuserVR surgical experience looks like:

In addition, surgeons in training can take their lessons home with the @HomeVR program:

@HomeVR expands the Fundamental Surgery platform, offering an easy route for residency programs to integrate the latest educational technologies into their curriculums. It supports consistency in training delivery and assessment across a cohort, and can be used to enhance the effectiveness of an institution’s curriculum…

The @HomeVR product is used on standalone headsets and can be taken home to use whenever and wherever the user would like, providing flexibility of learning. The @HomeVR product serves as a great introduction to the HapticVR product, which supports full skills development.

As the VRScout article states,

Think of FundamentalVR’s medical training system as a ‘flight simulator’ for both medical students and their instructors. If you’re going to make a mistake, this is the environment to do it. Because the experience is fully immersive—using realistic audio, video, and haptic feedback—the emotions that you experience are real.

Who knows? The next time (God forbid!) you go under the surgeon’s knife, she might have had part of her training in virtual reality!

Photo by National Cancer Institute on Unsplash

For more information about FundamentalVR’s Fundamental Surgery product, visit their website, or follow them on social media: Twitter, Facebook, LinkedIn, or check out their videos on YouTube (there’s also a ton of videos here on the Fundamental Surgery website). And I will be adding Fundamental Surgery to my ever-growing comprehensive list of social VR and virtual worlds.

Oxford Medical Simulation: A Brief Introduction

Oxford Medical Simulation (OMS) is an educational virtual reality platform for training healthcare professionals in world-class patient management using virtual patients, without risking lives. Here’s a video that explains the concept briefly:

This is one of those environments where the virtual patients have that sort of creepy, uncanny valley aspect to them: realistic looking, but something is not quite right (to me, it’s the eyes; real eyes are slightly translucent, as I wrote about here, not opaque like billiard balls):

That’s not creepy, that’s not creepy AT ALL…

Despite the off-putting uncanny valley avatars, OMS has racked up a truly impressive list of clients, including the National Health Service of the U.K. and (of course) the University of Oxford:

Oxford Medical Simulation offers not only on-site VR training, they also offer distance learning for medical professionals. In fact, the recent global coronavirus pandemic has been an unexpected opportunity for the company to promote their platform, with OMS offering free distance training to healthcare providers in three countries:

We appreciate how hard it is to deliver simulation and clinical education at the best of times, let alone during a crisis. As simulation educators, the team at OMS have experienced the chaos caused by last-minute clinical cancellations and the need to rapidly deliver simulation to fill the gap.

This same phenomenon is now happening on a global scale. In response, OMS immediately offered the OMS Distance Simulation platform free across the US, Canada and the UK as of March 16th, 2020.

…

Since May 16th, over 50 institutions – with over 17,000 learners between them – have signed up. Many have started utilizing the platform already and many more will start over the coming days. This is being done across all levels of medicine and nursing and for many different use cases:

◙ Nursing programs (BSN and NP), unable to deliver clinical placements; 
◙ Medical programs (DO and MD), fast-tracking their learners for clinical practice;
◙ Hospitals, up-skilling clinicians moving between departments;
◙ Health systems, rapidly bringing in new nurses and retraining clinicians returning to practice.

Aside from the generous offer of free courses for doctors and nurses, no pricing information is provided on their website for the platform, just a contact form to ask for a sales representative to contact you (and I’m quite sure that this platform is not cheap!). If you want more information on OMS, please visit their website, or follow them on social media: Facebook, Instagram, Twitter, and LinkedIn.

The other thing that I cannot seem to figure out, either from the website or the promotional videos, is whether or not OMS is truly social VR. Can you share these experiences with other avatars at the same time, like a group of doctors and nurses working as a team on a virtual patient during surgery, for example? Because both Road to VR and well-known VR YouTuber Nathie both lumped Oxford Medical Simulation in with dozens of other social VR apps in their overviews (which I blogged about here), I am going to make the same assumption that they obviously did, and I will be adding OMS to my comprehensive list of social VR and virtual world platforms.