RiVR Introduces: Historical Preservation Using Photogrammetry and Roomscale VR
Hello, this is a recording of a live session from Spark 2021. We hope you enjoy it. (UPBEAT DRUMMING MUSIC) Good morning and welcome to day four of Spark Ignite 2021.
Coming up today at 3pm we have a takeover from PechaKucha. At 1230, We have a session for freelancers about how they can work and be supported in the district. But first, we're with Alex Harvey, co founder of RiVR who specialize in photorealistic virtual environments. And in this session, we'll be looking
at how that can be applied to arts and heritage organizations. Over to you Alex Thanks Bronwyn. So I'm Alex Harvey, co founder and creative director at RiVR . And we make virtual reality experiences for multiple clients all over the world, but mainly in the blue light services. So like Alex Foster spoke about the other day, we do a lot of 360 video production. But we also do room scale environments for them
for their training. And recently, things are moving more into entertainment for us, and historical preservation. But we're still focusing on the blue light services quite a lot. This is the team or some of the team. But my current reality and this was before COVID. So used to be that we do lots of shows and travel around the world and show people our things. But now
obviously, we do a lot more of these virtual events. And my history was Codemasters mainly sort of 10 years, they're making racing games, started in the QA team, and then went all the way through to video and ended up making their trailers for TV and cinema and online. For all the different games with a team of people. There's like four of us in the team. And it was amazing job. But in 2014, I left to start, RiVR, started off as infinite pixel, but then later changed to RiVR But yeah, 2014, we started doing video production. And I'll whizz
through these, because I'd like to get to the photogrammetry stuff. Alex Foster touched quite a lot on the video in his previous talk. Video Production 360 tours and weddings was when we started tinkering with virtual reality. And then we did a lot of things with Southam College, teaching them how to shoot their own video content, their school trips and things like that. And then we had a horrible task of going on a
cruise ship in the Caribbean with Thompson, or Tui. And we did some promotional stuff for those guys. And then more 360, which Alex touched on Transport for London, blue light services.
And now we move into more photogrammetry training, which then is going to lead into historical preservation. So this is some of the scenes that we created for the home office, the STL. We create their crime scenes for training of different scenarios. And we use photogrammetry in that scene as well. I'm going to touch on in a minute what that is. But this is
brushing over how we use it in the blue light services. So one of our main projects is fire investigation. So here what we're seeing the burnt scenes that we create, we've done eight of these now, we burn a container exactly how they do in the real world for their training. And then they go in and investigate the fire but they can't move things around because another team of people are coming after. So we scan
every item and make a virtual representation of that object and then put it into a virtual reality scenario using photogrammetry. And we use unity. And once it's in there, they can investigate the scenario and then press a button at the end and everything goes back to normal and you can play that in multiplayer. You can record it you've got torch. So that's our main investigation product. And do like video as I
was mentioning to the guys, so there is quite a lot of video in this but I'll flick through it and just show you briefly. This is Investigate running. So it's showing you inside virtual reality, the level of fidelity that we have in these scenarios. So yeah, this is Jason Dean who helps us create the scenarios.
He's a fire investigator. So yeah, more things from investigate. And we're now focusing on crime scenes. So we're going to be making Crime Scene Investigation as well as fire investigation. And this is one of our latest scenes, it's actually my mom's flat in Warwick. And we scan the outside of it using photogrammetry. And made a representation of it in virtual reality and for the crime scene trainers to go in and test out their new methods. So photogrammetry, I've just
mentioned it a lot. Here's a little scribble I did explaining it a little bit. This is the main types of photogrammetry that we use in the projects that we do. And you can do it from
the ground or from the air. But it's basically taking 1000s and 1000s of photos, stitching them all together using the software and making a 3d photorealistic model of that object, person or place. Whatever you're looking to scan. So here this was a project at Guy's Cliff house, we've done quite a lot of scanning there. Because I've lived in Warwick, all my life used to live on the Butts and now I live on the high street. And I've got a bit of a
passion with Warwick and photogrammetry. So these are some of the rigs that we used. On the left hand side, you'll see us at Compton Verney where we scan the Adam Hall, and I'll show that as long as we've got enough time. But the rig in front of us here is a six Camera Rig where it obviously speeds up our photogrammetry by times six, we used to have to do with one camera, but Ross in the team developed this, which allows us to capture environments much quicker than we have done in the past. And as you see on the left, it can be changed for different environments. Drone photogrammetry is amazing. We've just been on a project down in Cornwall, scanning some sites, same process 1000s of images stitched together make a 3d model but you can't get as close to this and see as much detail as you would if you're on the floor with a six camera rig. We
were on the gadget show. I won't play the whole video you can go and watch it online. But we taught them how to do photogrammetry at Guy's Cliff using the six camera rig. So we taught them that you can scan in the cloisters cave, and then bring the data back to the studio and allow them to make a 3d model for themselves. But then afterwards, we then went down to their studio and put them inside the virtual world. I'll skip straight to it but yeah, this is them in the studio putting the vives on so we created a multiplayer experience where there inside the cloisters cave. And we also think me and
Ross got a little bit carried away. When we went down to do the briefing scanning session in the studio, we ended up scanning the whole studio and all the items, which allowed them to drop it into cloisters cave, this is available online. Not the experience yet, we might release it. They've got some cool objects in the gadget show. So in the background, things
like the hoverboard, the guys at the studio modelled the whole set and you've got a gravity button so you can press the button and everything floats around. And that's one of the projects capturing the Abbey in Kenilworth. So another local monument Ross was very keen on this one because he's from Kenilworth so we actually teamed up with this guy Simon Shader-Boa to add some process in to the images. So we would give him the images that we took of this place. And he would
scale them up and get more resolution and more textures out of the model. And here you can see multiple camera angles to capture this one. This is another one you can see on the top right that's a screenshot from sketchfab. So all of these models now Ross is having for us to use in virtual reality, but also to have on sketchfab so you can see with your phone or tablet and the link in this presentation. You can download it and check out all the models. There's a big theme here about scanning things. This was the speed map of work that we
scanned using photogrammetry. So took into the office 1000s of photos of it. Again, we've got it in sketchfab. So you can go and have a look. But he's basically preserved it forever. And now we've put it back in its box in a cupboard, and it doesn't have to come out to be destroyed anymore or degrade. And this video, you can obviously see online as well. St
Mary's Church in Warwick, I put this on here, because it was me basically testing what we could do in a day. So I turned up and did 4000 photos of St. Mary's Church just by one camera. And then I took the photos into the software that we use reality capture churn, the model took about six hours, I've got some big machines, and then processed it and dropped into Unity, and managed to put it into VR in the same day. So this was just a test a while ago to show the fact that you can do this rapid capture it goes all the way down to there was a little, like a something by the door. And we photogrammetry that
as well. So you can lean right in and see that just shows the scale from tiny to massive. Compton Verney. I'm passionate about this project because it was funded a few years ago by Coventry University and Compton Verney, and we created two experiences. One was an owl experience that let you fly around the landscape and experience what it's like to fly around the landscape like an owl. But learn about the landscapes that Capability Brown
created at Compton Verney. And this is some footage of them using it in Compton Verney on one of the open days, pre COVID. Also, the second part of the project was to laser scan and photogrammetry the Adam Hall, but then put the model sorry, the paintings on the wall and not there anymore. So we had to
find the paintings in London photogrammetry, scan them, and then put them back on the wall inside the Adam Hall. So now you've got virtual representation of the hall with the paintings back on with the freeze on the wall as well. So if anyone from Compton Verney is listening, we'd love to get this either online for people to download an experience for as an attraction that people can come and have a go at Compton Verney, when you reopen more videos about that. Lots of behind the scenes of Scot and Naomi people that helped me create it. Now we move on to Leamington Spa, Art Gallery and museum. They came to
us with a project of a set of objects, I think it was maybe 15 different objects. You can see Tyrone and Ross here doing some scanning. Ross is doing a really old painting, 16th century and quite valuable. So I was a bit nervous watching him lean over the painting with his big camera. And Tyrone's doing
something that was I think 2000 years old, it was some piece of an old shield. And here we see the little bit of footage that Tyrone's put together of another one of the objects of a toy horse and cart, that Ross is photogrammetry in here. So the the client, the museum, would like these objects in augmented reality, so that on your phone, you can walk around different places and pop up the objects and see them on your phone or tablet, which is really good. But we want to show them what else they can do, which is if you see in the bottom of this image, there's the bathroom next door. So we're going to photogrammetry scan that and then put all the objects inside that and create a VR experience as well so that I can show them that if this is put, as an experience, people can download it and meet in this place and hold these objects as well as augmented reality. So it's just showing that once you've got a model, you can use it in so many different ways. AR VR also it
could be on a 2d screen like we talked about sketchfab and saved and obviously preserved forever. Okay, so a lot of people have been talking about tunnels recently. And I think two days ago the observer or Courier did a post about a tunnel in Leamington going from Newbold Terrace to the clock tower. It
frustrated me reading it a little bit because the article had one photo of the house and Then just a load of text. And I'm thinking, we've been doing this project for about a year or two in Warwick. Because there's always been this rumour or this myth about these Warwick tunnels. So the little bit of the backstory is 12 years ago Jarris from IP building services, he was doing some work on a house in Warwick. And he told us that there was a tunnel. So we went back. Last year, we
didn't have access until last year to go back down, or they hadn't put the story together. But here, we did find a tunnel. And we're going back at the weekend, this weekend to laser scan a little bit more with the fire service, they're going to do a training exercise in the tunnel, which is handy because we can put laser scanners on them and cameras. So there's lots of rumours of tunnels, and I'm not sure if they were true or not. But this is definitely something to investigate that
we're looking at. And we've been told it might be some sort of water tunnel. But it's a little bit of a weird one. Because it is off a freshwater Well, what was once freshwater. So I'm just
going to press play on this video as well. I really wanted to show this sort of footage because I was disappointed with the tunnels thing in Leamington. And the fact that we've actually got so much data in Warwick, that is so much clearer. And we
can see a lot more I think this should be talked about a little bit more. Anyway, the tunnels. We've been down once and we photogrammetryed the project at the moment is a 2d documentary showing us going down looking at these tunnels. But at the end of the documentary, you get to go inside the tunnel, because we've scanned it. So you'll be on your sofa, you can watch the documentary, and then you can go into the tunnel yourself. So what you're seeing here in the
bottom left is Jim, who was awesome. And he actually went down the tunnel with a laser scanner for us and a 360. Camera, he went down three times. And this is the fourth time he goes down with his phone he's filming. So he goes down the well, five meters through a really small hole. And then he's got a left hand turn or going sort of up and three meters, and then a massive tunnel going in front of him heading off towards St Mary's Church with a pipe in it. So we do want to get a piece
of that pipe and a brick and do some testing on that and try and find out. If this was any sort of tunnel on the left hand side in laser scan, which is the best way for me, we can actually view that in VR as well. The best way for me to jump inside these places and experience where you get spatial feeling of where you are in this tunnel. Me and my brother, I think we're the only ones that have actually been inside the point cloud. And it is really, it's a strange feeling to be in there. So
hopefully, we're going to scan some more, we're going to uncover some more and see if it goes back to the basement or off to any significant buildings. So if anyone wants to talk more about that, get in touch. And this is a render because we did photogrammetry inside the well. So this is the well that we just talked about the tunnel, but we're coming up from the bottom.
So we're coming up from the bottom of the well. And we see the see the entrance on the left that Jim crawled through super small hole, we didn't think it was doable at first. But as he got through it opened up. And obviously Ross photogrammetry the whole thing that was another visit we went back with a camera. And we built some scaffolding and dropped the camera down the well. And Ross did about 600 photos with a
flash and just captured the whole thing that you're seeing here. So we'll be able to go drop down this with a torch inside VR and go off up the tunnel. And I have got a bit of an addiction, which is photogrammetry. And this was me scanning multiple places. I think I took the camera out a lot with me on lockdown during my one hour walks and just took lots of photos. And so I've captured quite a lot of work
using cameras and lasers. And this is just a video going through some of those people that are local to the area probably recognize some of these places. All of these scenarios that I've shown so far are completely accessible in inside VR. And that's with the big computer and you know That's
sometimes quite hard to set up. But the Oculus quest has come out now. So it's a smaller headset. So we're trying to fit things onto that, because it's so much easier to use. But the big thing is it has inside out tracking. So you, you don't have to have sensors. But what we're doing here is doing all the
computing the cloud with Nvidia. So we haven't got to have any processing power used in the headset, we're using it off headset. So this is a bit of a look into some of the projects that we're working on in the future. There's going to be a video about this coming out soon. And we're doing more with them Cream productions about these dynamic digital humans. And these are really impressive everyone in the office because you can put a VR headset on and see like this guy in the middle Dom. And you can see him in a virtual space talking to you.
But we can record the character in studio. So in RiVR in house, we can do it all in house, it's a five megabyte file on the quest as well, which is amazing. And it is the best I've seen in a standalone headset. And this was me just talking about 3 dof and 6 dof because Alex foster did a great job of chatting about three dof and RiVR Link the other day, and 6 dof is the ability to walk around pick things up, more like the things you've seen on some of these slides. But yeah, I think we did
get to the end Bronwyn so it would be good if there is any questions, but if not, I have reached the end. Thanks so much, Alex. That was amazing. It's really exciting to see your work at the Leamington Spa Art Gallery and museum as well. And we're really excited to show the results of that with the public on our social media soon. And we have got a question in the chat. So it's about budget, basically, how can the art sector use more of this technology when budgets are often small or non existent? And obviously, as time goes by, hopefully it'll become more affordable. And this is
something that you can apply for a grant for and obviously use it that way. But I wondered if you had anything you wanted to add along that? Yeah, definitely. Well, there's a few things of the software that we use reality capture, you can actually use that for free. So you can actually just download that and practice yourself with some photogrammetry. So you can take photos with your phone as well. So the barrier to entry these
days is coming down and down. You can with an iPhone that's got LIDAR, or even not LIDAR, you can do photogrammetry on a phone, so they could scan some objects and I do it on my phone. I think when Jonathan met me in the park, recently, I showed him I just scan the tree with my phone and uploaded it to sketchfab as a 3d model. While I was just walking around the park, so there is levels that a lot of people can try and get into. If you're a museum, obviously, you might be able to find some sort of budgets. But if you're a smaller
organization, like Alex talked about the 360 camera is just this big. And that could do all of the things that we talked about with Compton Verney 360 video. And then obviously your phone can do some photos for photogrammetry. So yeah, and Unreal and unity as software programs to get these models into a VR scene. They're free to use as well. So you can just do it. But obviously, I'd say, yeah, you should come to us and
pay us to do it. But it's getting easier for the individual to do. Awesome. And where do you see this sort of technology going in
the in the short term sort of next three to five years? Yeah, we're doing some work with them. Obviously, everyone's talking about 5g, six it just keeps going on. But the real thing that we can do with that technology is we can have a 360 camera like this, that just live streams from wherever that's going to be quite soon. But the scanning of real time places is going to be definitely where we're going to be heading.
You'll be able to walk through a room scan it, and someone on the other side of the world, I'll just be able to walk around that at the same time. You know, pretty much just after but that's going to definitely be helping us for crime scene. We're developing that for crime scene so that you can walk through a scene photogrammetry the whole scene, process the data in the cloud, and someone else in the world can view that. And that can be done for anything. So this is where we
move into a minefield. And I've thought about it a lot. The digital copyright of 3d models is an interesting area to think about for me, because Warwick Castle, for example. And I'm just going to say what we've been talking, Warwick Castle allows you to take photos inside the building or in the ground, you can go and take a photo on your phone. So if I was there with five people, and we all had a camera, and we took 65,000 photos, and then left, and then made a 3d model and put it online and started using it for stuff VR and sell it to VFX companies. I don't know where the line is there. So I'm very
interested to talk to Compton Verney, Warwick Castle and say to them, you know, if you're interested, I'd like to sign you up with us to say, I'll scan your place, your church, your monument, your anything, but then we'll work as a partnership to have it as a digital model, where we can sell it and both benefit from it. If they sign up and say, We own the digital model of our place, and it's for sale, then people and the reason I'm talking about it like that is because people are using the 3d models that we make these days, on LED panels now behind the actors for big films like Mandalorian, and the Lion King. So any 3d model is got enough photo realism and can have lighting changes on it can be used in production to make you think that the person is at that place in an LED volume. So
that's the reason I think three to five years is going to be that virtual productions and 3d models. And yeah, who owns what? Can I scan a street and own it as a model? Or have I got a visit everyone in the street? But then Google are doing it now Google, drive down the street and take a photo? So and then they make a 3d model from it? So I'm not sure. Yeah, the law needs to catch up with technology a little bit. And then one final question, just sort of scaling it back to the individual or the small gallery in the small museum, maybe teams behind the scenes really excited about this technology. What are your top tips for people who are considering using this? And where would you suggest they start by calling you obviously Photogrammetry? Yeah, Yeah, like I touched on a minute ago the best way to start is I guess download reality capture, it's free. And take some photos on your phone, literally on your phone, go to a park and take 20 photos of a tree from every angle, just walk around it. And
then let the software do the rest. But you don't even have to take the photos and upload them really to reality capture, just get your phone out. Now if you've got an iPhone 12 probably or it will work on most phones, but there's different apps one that I use is scan-e-verse, nothing to do me. And you can you can just use it and upload things direct to sketchfab for free sketchfab is a website Yeah a website for 3d models. And if you upload a model that
you say is good enough, or you know imagine this cup, you can scan anything do a really good job or not so good job of the scan and you can sell those models on there like you would early days YouTube. This is early days YouTube for 3d models. So models and turbo squids another site where you can buy and sell models. But yeah, 3d assets are going to be are going to be selling in the in the future. So scan
everything that you've got and just put it online. Amazing. And so for people to find out more, they come to your website.
Yeah, LinkedIn is the main one, I guess that we post most stuff on. Okay, good to know. Yeah, LinkedIn, and obviously Facebook and there's a few Facebook tunnel groups for some of the people that want to follow that stuff. Okay. Awesome.
2021-03-14 06:31
