Using technologies from the gaming industry to improve medicine

Using technologies from the gaming industry to improve medicine

Show Video

(gentle music) - This is Stanford Engineering's
The Future of Everything. I'm your host, Russ Altman. The gaming industry is
huge and it attracts the attention of millions
of people, gamers.

The competitive landscape
of gaming drives innovation and hardware and software,
virtual reality goggles, motion detection and augmented reality where they project things
on top of real objects. These devices though are
not just for playing games. They have real promise as technologies to improve medical care. Bruce Daniel is a professor of Radiology and Bioengineering at Stanford University.

He studies how human computer technologies such as virtual reality goggles,
mixed reality projection or even motion detection
can improve medicine. He will tell us that the
technology is almost there for routine use in medicine, and now it's time to make good interfaces where we can train physicians how to best use these technologies. We're talking about mixed reality, augmented reality, virtual reality. You're a radiologist. What, Bruce, got you
interested in all these fields and their potential impact
on medicine in the future? - Well, two things; one is that I think that
if you're going to look at a virtual world, or
possibly look at right now, and I can't think of
anything more valuable for you to look at than pictures of the inside of your own body. The other part about it is
because for a long time, we had a very unique
facility here at Stanford that the former chair of Bioengineering, Dr. Pelc had a lot of to do
with, which was the MRT scanner.

It was an open MRI scanner
where you could get in there and you could do procedures on the patient while you were with the
patient in the MRI scanner. It was like a space shuttle. It was expensive. It kind of became hard to sustain. But it was amazing. I did more cases in
there than anyone else.

I really captured this idea
that wouldn't it be great if I could really intimately
use these pictures to help patients with procedures? Then I began thinking,
"Well, well when that, can not be sustained. How do we do something simular to that in a more cost effective way. "and I think this is
one of the solutions." - That's great, 'cause that gets to what
one of my big questions was, is there a problem here
that needs to be solved or do we have a solution
looking for problems? But it sounds like in your last answer that you actually see opportunities where medicine is not delivered
in the best possible way or surgery, medical care. So can you go a little deeper
into what are the problems that maybe we just accept as problems and don't realize some of
these could be actually solved? - Yeah. Well, so the most acute problems that I can think of are the ones in actual procedure delivery. So like surgery, for example.

I do a lot with breast cancer
and you might be surprised to know that if you have a
lumpectomy done as a woman, they get out the tumor about
three quarters of the time, maybe 80% of the time. But if you went to go
get your gallbladder out and the doc told you, "Well,
we're gonna get it out "almost all the time. "But one out of five times
or one out of four times, "we're gonna have to redo it," you'd find a new surgeon. But this is the... And this is the reality
that people deal with.

Surgeons are not doing this
because they're bad surgeons. It's because it's a very
challenging operation. Similarly, we have work
working, Christoph Leuze and Match Buddy are working on transcranial magnetic stimulation. A gradient device can be used to stimulate the brain quite amazingly to treat the FDA approved,
to treat depression, but it only works in
about half the patients. Why is that? Well, one possibility is that
we're just not putting it in the right place every time we do the procedure every day for a month.

So at circuits, you're activating, while you're doing the procedures, maybe these things could
improve the efficacy of these procedures resulting
in fewer repeat procedures in a cost effective way. - Going back to the breast cancer, that means, if I'm
understanding you correctly, the main problem is
they can't find the lump that they're supposed to be optimizing, so to speak.
- Exactly. - That are supposed to be removed. Take us through how some of
these emerging technologies and a lot of them are based
on the gaming industry.

They're building these amazing, not all, but a lot of this comes
from the economies of scale of people who are playing games
and doing virtual reality. So how exactly... Could you walk us through what
would the surgeons experience be like or how would it change
in using these technologies? - Well, what we've worked on initially is that the surgeon
puts this technology on and they look at their patient and the patient has had
a preoperative scan done say in my department, an MRI scan that is extremely accurate and it provides you volumetric 3D data. That data is processed into something that no longer looks
like a bunch of slices, but instead looks like
three dimensional models that maybe you 3D printed
almost or something like that. But then these, instead
of being a physical model, you hold in your patient. So you can, right before you do surgery, do a virtual dissection
of the patient to see where it is you need to go.

Now there's a lot of
challenges to getting that to work to the level of precision that's really going to improve care, but that vision is very clearly possible. And you're absolutely right that it's all leveraged off
of the gaming technology that's coming out there. And frankly, kind of industrial
engineering technology? So, a hollow lens device like I have here, one of these things you put on. So this device would never have... Yeah, the headset. Put it on. That device is extremely sophisticated and could never have
been built for the market of say 300,000 breast cancers a year, which is the most common cancer in women.

But even 300,000 a year
is not enough for this, but the 300 million
people who might be gaming in their basement, that's enough. That's how it gets built. - So how do the surgeons feel about this? I could imagine A, they're
excited because they want to get 100% of the lumps
taken out of course.

On the other hand, they
have a certain training and a certain workflow in their life and it probably did not involve, unless they were big
gamers as kids, headsets. How have surgeons responded to this and how might training have to change? - The surgeons have
responded to this universally with being in favor of this. I think it's quite remarkable how we once presented this a few years ago to the San Francisco
breast surgery meeting.

And they looked at this and said, "This is gonna change things as much as "the robots have changed surgery." And I think that if you're a surgeon, the last thing you want to
do is have a conversation with your patients saying, "That operation we did a few days ago, "well, we didn't quite get everything. "We have to go back." They would really like to diminish that.

They'd also like to diminish
the fact that right now, what their main strategy for avoiding that is taking out twice as much
tissue as they need to. In fact, they would like
to do a better cosmetic job for patients as well. They're really motivated, but the one challenge is
that the big early adopters on this are the neurosurgeons, 'cause the brain has always been an attractive imaging target and they have these million dollar rigs of things they put in the OR to do stereotactic surgery, et cetera. They're onboard already.

But a breast surgeon has to be able to do like eight operations a day. How do we make this work reliably quickly? And when you just put it on
and I see what I need to see without having to click
through a bunch of menus and log in and whatever,
all this other stuff that using our computer
seems to entail these days. We need to get to the point where this is as easy
as putting a stethoscope in your ear and listening to the patient. It has to be that easy that
when you put it on your head, you just see what you need to see. - So this is the traditional
user interface issue that sometimes engineers
who don't immerse themselves in the actual use cases can get wrong. Which leads me to another question, which is how good is our hardware? Is the hardware ready to do...

It's ready for gaming, but is it ready for serious physicians doing
life and death procedures? Or are we needing a few
more years of development? Do you, as a physician get to impact that development at all, or you're simply at the mercy of whatever the gaming industry needs? - It depends on your
application, I would say. A thing like the transcranial
magnetic stimulation, the device creates an effect that's around a centimeter or two in size. So you don't have to be
exactly that accurate because it's not gonna matter. But if you're putting in a
neurosurgical stimulator, that you want to have
it within a millimeter or a fraction of a millimeter in order to hit the right nucleus and
treat somebody's Parkinson's or whatever you're trying to do.

It depends on the target. We're trying to go for low hanging fruit that are big procedures
that we're interested in, orthopedics we're interested in. The breast cancer surgery one...

The breast cancer surgery one, they're gonna take out a
rind of normal tissue anyway. That one's a pretty low hanging fruit. We have a project on
acetabular impingement syndrome where we gotta get down to
the millimeters of shaving. I'm not sure we're accurate
enough for that quite yet. Another big problem is that
these devices are interesting.

They're very compelling actually. And they really show you something that looks three dimensional and the history of that is fascinating. But what's interesting is that
you and I will put this on and ostensibly see some
virtual heart in front of us.

But where you see that heart
and where I see the heart with the same headset might
be order to make it appear, this virtual object
appear in the right place. They have a fixed focus
out at a couple of meters, but I wanna work at hands distance. So we gotta fix that.

The last one we need to fix is some of the depth cues
are not quite right yet. You would like to be able
to perceive something as being not just in front of you, but inside of another object. And that causes some
misunderstanding in your brain.

as you look at sort of like a surface and you try to reconcile
that I see an object, that's a surface, and I see another object that is supposed to be
behind that surface, but how do we render it so
that it really looks convincing that it's back there and
not floating in front of it? - That's fascinating 'cause we don't even do much of that in
real life, if you will. In other words, I'm thinking about when I'm swimming around, snorkeling, the 3D depth perception
becomes a challenge. That isn't a challenge when I'm in terrestrial normal situation. So here, we're almost
having to train the surgeons for a new way of viewing. Is the resolution of
these goggles good enough? - Question you have there is... I think the resolution
will improve a little bit, but the resolution's
actually pretty good there.

I think that at some level, but not all are using this goggles, they're using the microscope
itself and then trying to overlay the images into that. But the images that we take, we think an MRI scan is super resolution, but about a half a millimeter if pre-operative imaging is your goal, that what you want to superimpose. - Before we leave this
idea of the visualization of body parts, basically, and we've been talking about surgery, are there other applications that you're excited about of this? So you're a radiologist and you're looking at MRIs and CTs all the time. Do you imagine you'll
start looking at those in a more augmented way, or is it primarily the surgical
impacts that we're looking? - Well, no, I think... But we are responsible
for every box in the set. Really the best way to get
to interpret every little bit is by slicing through it and that's how we could continue to do this despite the 3D things being available for a while on a computer screen.

But I think that more
and more regular doctors are really going to be using this. And the model that I see is gonna change from where we have a
report that we dictate and then when you go see your doctor, they're not even looking at you, they're looking at their
computer over here, reading my report and trying to figure out what's going on with you. And now it's gonna change where...

The device I showed you is gonna turn into something like this and they're gonna see this
museum gallery approach that's gonna come up. They're gonna look at you. They're gonna look at your
knee when you come in, 'cause your knee hurts.

This is one that my
friend, Brian Hargraves, who co-directs the labs come up with. They're gonna look at your knee and you're gonna say, "It
hurts over here, doc." And they're gonna look at you and say, "Yeah, I can see inside you.

"That's right where your meniscus
is torn right over there." And it correlates perfectly. Or, "Oh, no, no, that's
a burst over here." I think the long term
vision is it's gonna affect the doctor patient relationship in many, many ways and make it better. The doctor's gonna pay
attention to you instead of that computer screen over there.

- That is super. As a regular doctor
myself, and by the way, I love that term, 'cause that
perfectly describes myself. I love that idea because, for example, as a general internist, I don't
always think about anatomy. Every 10 years I take a test where I have to review my anatomy. But if I'm having some kind of display, especially if I can
still make eye contact, that was very important,
what you said about, it'll be more like a pair of glasses because I could scare my
patient if I walked in with one of those things
that looks like a binoculars, but if it's much more subtle and I can make eye
contact with the patient, but I have a visual
assistance in reminding me where the muscles are, how
to do the shoulder exam, how to do the knee exam, 'cause I'll do one of those
every week or two weeks. I don't do 'em enough
so that it's automatic.

That could really be
transformative in terms of the ability to improve
the physical examination. And as you know, many doctors are mourning the loss of physical examination skills and this could really pump that back up. - I had a case yesterday, I spent about an hour with
one of the gastroenterologists 'cause this patient was saying, "Every now and then I have this lump "that shows up in my abdomen."

And she even had pictures
she had taken of this. And we are trying to figure out where's this lump compared to what's on the scans that I saw. We were talking on the phone and on Zoom and trying to figure this out. But if he had had this
ability to just like, "I'm just looking at the patient, "I can see that lump is," I don't know, your gallbladder or that's your liver edge.

I mean, it would've solved
and it would've solved it in an intuitive way that would not have required this long conversation. But I do think the radiologist
role is gonna change. We're gonna go from making
reports to being the one who sort of annotates
and paints these images and says, "In three
dimensions, look over here, "this is a blocked vessel or a
torn vessel or a tumor here." You're going to be looking at... We're already doing a little bit of that in the prostate cancer actually. It's interesting.

- This is The Future of
Everything with Russ Altman. More with Bruce Daniel next on SiriusXM Business Radio channel 132. Welcome back to The Future of Everything. I'm Russ Altman, and I'm speaking with Prof. Bruce Daniel
of Stanford University. In the last segment,
Bruce described the ways in which augmented reality
may improve surgeries, both their success rate
and also their efficiency. In this segment, he will tell us how these technologies will
motivate physical therapy patients to do their exercises, how it may help teams of doctors come up with better plans for complex patients and how it may help
patients manage anxiety about getting into MRI machines.

Sign me up for that. So we had a great chat about these methods for augmenting anatomy. But I know that you've also
thought about other things in terms of things like physical therapy. Can you tell me how do
these technologies promise to impact physical therapy? - Well, what's really interesting
about these new systems that have come out is
that they were designed to warrants on them that
can do things like track where you're looking with your eyes and they need that in order
to provide a convincing image. But interact with them, they find a way that they can track where
your hands are doing so they can actually understand the pose of your hands very well.

So all of a sudden... And they can track your
head and your body. So all (indistinct).

- Here's where we're benefiting
from the gaming industry with the dancing apps and the shooting and tennis and bowling. - But there are times when
these kinds of movements have to be done in as
prescribed by a doctor in order to help you get over a stroke or some other kind of chronic injury. And so one of our students last
fall came up with this idea of a hand app for someone who's had maybe a stroke involving their limb, where they have to move their hand. But what they wanted to
do is not just train them to be able to move their hand, they wanted to motivate them. They didn't want people just
moving their hand a little bit. They want people moving their hand the full extension amount.

What they did is they had
little virtual turtles coming across that that could be
seen in three dimensions. And it looked like you're
swatting these turtles with your fingers. And what was interesting is
the turtles would only... it's like whackamole, but those turtles would only explode into these pretty fireworks
if you had done a good job with your hand, as
opposed to if you'd done a poor job with your hand.

Then they could keep track
of how many you've done and they could kind of make it fun to do this otherwise sort of boring
exercise for the patient. I'm not sure it's there for everyone, but this general paradigm of
improving human performance, as opposed to just healthcare is something that I think is really
a great opportunity here that's just barely being explored. - You did try it out
though. Am I wrong that-- - Oh, yeah. Yeah, we've tried this app.

We now have to get to the point of trying it out on patients. It was a goal that was like
one that we knew we could do, but the body tracking stuff
is exploding all over. It will be more than just your fingers. It will be how you move
your whole body I Think. I mean, I'm very excited about this because I'm at an age when
we won't go into details, but either me or many
friends and relatives my age are in the middle of
physical therapy basically because that's what happens when you get to be a certain
age and it is boring. And anything that the
physical therapist can do to motivate is super important, especially for these long
haul physical therapy plans that are months.

It's a rare patient
that has the discipline to stick to it without some help. And so that's extremely exciting. I didn't wanna miss the
chance to also ask you about telepresence and education.

What's happening in that area? - Well, one of the things
that the vendors have realized is that a killer app for these is that you and I might in the future, not just have this conference over a video Zoom link like this, but that you and I will actually be able to wear these headsets and then
I will see a virtual version of you and you will see
a virtual version of me and we'll be exploring
this same space together. And this is actually already possible now, as long as we have... if you're willing to accept that the other person looks
like a cartoon avatar. But they're even getting to
the point where with that, I heard recently with some of the headsets where they can actually monitor your smile and your eye movements and your blinking and make it less uncanny
valley and more realistic. But if we could do that
together with a model of a congenitally deformed
heart, and you're a surgeon and I'm a cardiologist and
he's a cardiac radiologist and we have this baby heart right here, and we could talk about how does the VSD need to
be closed in this patient? And here's the blood flow
dynamics from the MRI 40 flow and how much flow is going
out the pulmonary artery versus the aorta and how
big the shunt fraction is. And it must be going across
here, what kind of closure or which kind of operation we should do.

We can have this virtual conference. Right now what we do is we
all get together in a room and have these multidisciplinary boards. But that requires physical presence, which is hard to achieve
with a big group of experts. You might wanna have the expert on this particular rare condition
who's from Texas join you. And join in...

And even for general education, we have a student here
where they've scanned some beautiful high resolution
cadaver dissections, and another student who's
working with doctors in rural Kenya, rural
Kenya medical school. They don't have the resources
to have a cadaver lab, but their doctors with a
very simple device could be looking at the same
dissection performed by the expert anatomist from (indistinct) and who has spent his
whole career dissecting out the femoral cutaneous nerve
that I couldn't even show you. I think that this will democratize a lot of medical education in that way, and that'll be great for everybody. - This is super exciting, 'cause I don't think patients
realize, maybe they do, that when they have complicated cases, it very much becomes a team sport, as you said, there's all kinds of experts, including nurses and pharmacists
and they all get together. And the ability to have a
virtual meeting not be almost as good as meeting in person,
but having it be much better. And especially when you can
have 3D models of the organs or the malfunctions that are being...

that's a very exciting
model for both efficient and improved quality. How far away are we from all this? - At Case Western, they actually... which is one of our
leading medical schools, the best one in Ohio and
leading medical schools in the country, they got rid
of their anatomy cadaver labs in favor of having virtual reality content like this delivered. They've actually transformed. - Case Western trained physician, they may have learned
anatomy in a virtual world.

- Yes, absolutely. - And they know what
they're talking about? - They do, and maybe better
than the cadaver world. I mean, my experience with anatomy, maybe you had this was that
when we did our dissections, we would go in there and
try to find these things.

And then we would leave,
and then overnight, the prosector who in our case had come from Heidelberg, Germany would come and he would fix up all these things that we had made hamburger out of. And you go in the next
day and you're like, "Wow, it looks beautiful." (Bruce laughing) - We were in charge of putting pins where we thought various things were. And you're right, the next morning, all the pins would be moved
to the right location. - Exactly. And yet now we'll have
really this great expert and also you can take it apart, but you can put it back
together if it's virtual.

You can actually see the
relationships in so many ways. - We have only a couple minutes left, but there's one other area
that I wanted to ask about, 'cause it's a great topic. It's MRI and fear. I mean right off the
bat, when you say MRI, many of us shut down because we've been in those tiny little
tubes, trying to be still, worried about whether our
saliva is gonna choke us because they told us not to move our head and try not to swallow. But it sounds like there's
some virtual reality or augmented reality
solutions for MRI phobia.

- Well it, a lot of
phobias are being tackled with virtual reality actually. Tom Caruso in the CHARIOT program here, Sam Rodriguez at Children's
Hospital have been trying to help kids get over
phobias by distracting them with video games in virtual
reality, for example. But our idea was let's make
the experience of an MRI be not something that's
unfamiliar to a child who has to have an MRI and then freaks out when they get stuffed in the tube and the gradient start banging. Let's give them that experience. And so they are guided through
with a really encouraging way that kind of incrementally increases what the experience is like with a thing that's as simple as a Google cardboard that you stick your phone
in and they look at it. It may be at their home and
they lie down on their bed and they can pretend while
they're guided through to see what it would be like
to be in the MRI scanner and to hear those sounds.

What's really neat about
it is because again, it's monitoring their head
position to make it work, we could tell whether
they're moving or not. And so we could actually show them your picture's pretty good, but if you held a little more still, the green ball would stay green and then the picture
would look even better. And so we could actually train the ideas that we'll be able to train kids. One of our students presented this at the ISMRM conference
just a couple months ago. I was very excited about
this possibility of now that you could help people prepare for what might be a
scary medical procedure. - It's great because you
describe it as a thing initially for children, but many of us
who have had MRIs are thinking, "I need this."

I mean, it's such an
obvious idea to give me a little heads up about
what it's gonna be like, because most of us just
talk to our friends and say, "How bad was it?" And then of course, you
get whatever story you get. And the idea that there would be a prep, we all prep for various
procedures and various visits, and why not prep everybody for their MRIs. It's obvious that this will
have applications for adults. You just probably have to make
whatever the distraction is or whatever the game is.

It has to become adult oriented. But that doesn't sound
like it would be very hard. - Yeah. I mean, I think what
would really be the best for it would be combining it with
this telepresence thing. People want the human connection.

If we could have somehow the technologist who's going to run your
MRI scan magically appear in this virtual training experience that a child or an adult is having, they would even begin to understand that there's gonna be another person there with them who might not
be in the room with them, but is listening to them or who will be in the
room at certain times. And they'll be able to understand
how to remain connected and that will alleviate a lot of anxiety. And that's my hunch.

I mean, just unproven,
but we have my hunch. - Thanks to Bruce Daniel. That was the future of
augmented reality in medicine. You have been listening to
The Future of Everything with Russ Altman on SiriusXM
Business Radio channel 132. (gentle music)

2022-07-27 04:13

Show Video

Other news