Virtual Reality for Visualization
So, today is the fourth. Presentation. In our scientific visualization workshop. Series and, we're hoping to do this at beyond, this semester, but here's the calendar, for. This semester and, my. Things. May happen you know in like the November timeframe to move things around but for for. The most part at least by especially for the next month or so we're pretty fixed in, the schedule so next, week, Chauncy, friend who's one of our colleagues. In the advanced visualization lab, who's stationed, at the our Indianapolis. Facilities. And he'll, be coming down or. Maybe presenting remotely, I haven't confirmed, that with him he'll, and so he'll be talking about augmented, reality for, visualization. So. I will have kind of a two, things on immersive. Technologies, back-to-back and then, we'll start we'll. Talk about some other topics after that. All. Right so so. Today is immersive, visualization or, virtual reality with for visualization, and. A quick kind, of statement, about what we're talking about then is taking. Scientific, visualization which, we've been talking about the last three. Weeks kind, of focused mostly on desktop. Tools and apply. That to what we call an immersive interface which is just kind of a fancier, way of, saying virtual reality, so, so. That's we're going to talk about today I wanted. To show kind of quickly I see we'll see how this works a quick, demo of VR. Application, or maybe a couple of quick demos to start us off get us a feeling for what's going on and so, I'm going to bring up a, little. Medium. Technology. Thing here and. Oh. I have to start photo booths again. Right. So. Just, a moment I. Have. Camera, set up so that the remote people can see what's going on as well. Oops. And. Had. It set up but then we crashed, due to the. Or. Oops, lost due to the crash. There. We go. Oops. That's not what I wanted, oh. Here. It is alright. Alright. So here's the camera looking, at a computer, which is showing us what's, going on in a head-mounted display so. I have this head mount display and, this is a one, of the commercial, tools that. Will. Kind of mention today so that one of the things we have is is, commercial. Tools and, and handwritten code so, here's one that does a molecular visualization. I. Haven't. Used this one too much I've just kind of played around with it for a little bit Joe, here from. Crain played around with it for 10 to 15 minutes before we got started today, so he's, probably more of an expert at this than I am at the moment but, here's one application, so this is a commercial one you can download it for free on Steam, and so it works both for commercial. 5 and commercial oculis tools, I. Won't. Try. To demo that while. Talking but. I do also want to show. One. Other application. If. We can get out of here. And. That. Is pair of you. Alright. So, we talked about pair of you two weeks ago and, so. I want to. Just. Quickly show you how easy it is now to do pair of you so one, of the files we looked at two weeks ago was, from Professor, Horowitz, and so we'll bring up his data real quick. Load. It in there's, the frame if we add a contour, to that at. The value we talked about back then. And. Apply, that we get this, I. Don't. Know what stage this and what stage is this Dave.
Okay. Uniform, density of. Neutrons. And. So. We'll add the connectivity. Filter which we did the other day. All. Right so I'm not going to check play with the color, or anything like that but. So this interfaces backwards, for those of you watching on the video which. Is everybody I guess there's. A button down here it says send to open VR so I just click on that button and it. Gives me a little warning and, if. I go to. Have. To turn that down. My. Display mirror, them. Here we go it'll, show me what's being shown in the vibe and. If I look, at. The right direction there. Is the data oh it's. Oh I was gonna say it looks squished but that's because there's a cutting plane on there I'm gonna put the head-mounted display, on real quick here so. There's. A cutting plane and I can grab this cutting plane and actually. I can move that data so it's closer I mean I can fly around the data I can grab, the cutting plane slice. It through any way I want really, quickly here I can, also grab the data and rotate. That around however, I want so, I can move the data relative, to the cutting plane to move the cutting plane relative to the data and, so, again it's pretty simple pair of you. Thing, we can we can do more in pair of you to get our data a different state we, can animate this one although the this, one loads pretty slowly does a lot of calculations, so it's it's. Not really pleasant to watch animated, but. But. It works it works pretty quickly so it. You. Know it has some limitations, in the speed if you're trying to do a lot of data and, we'll talk about that in a little bit but, otherwise you, know it's a commercial open source an. Open source commercially. Supported package, and if, you have a VY for an oculus you just press the button and you and you're ready to go so, so. That's something we'll talk a little bit about those. Things and we'll do demonstrations. For those of you who are here with us we'll. Do demonstrations, later. After. Basically we're, done talking and you. Can try a pair of you you can't rise the the Nano me molecular. Viewer or some, of the other things that I'm gonna show today. All. Right oops I turn my volume down I don't know let's go. There. We go all. The way no okay. So uh. They. Need to get focus here there we go. Baby. There. We. Go all right so. This. Is a lecture, series on scientific, visualization we. Don't really talk about what virtual reality is so I'm going to spend it just a couple slides. Explaining. What what, I mean when I talk about virtual reality so, here's a definition for virtual reality that's, a lot of words so I'm going to break it down into some. Chewable. Segments. So, the first thing is you want to know where the person is because you virtual reality responds to the movement of the person so we have two what we call track the user we have to know where they're at the, vibe has that built in. If. For those of you in the room you can see I'm pointing to there's a couple, of these stations.
We Call them lighthouses for the vibe and they put, out a pattern in the air of light, and then, the vibe itself, has these little, little. Dimples on it and each of these dimples is sensing, when that light pattern comes by and that's how it knows where it's at and the same is true for the, controller's. Different virtual reality systems. Use different techniques the, oculus uses cameras. Instead. Of say, the lighthouse but works, essentially. Similar. Similar, concepts. So. Once we've know where the user is we want to display, to. The user from, their perspective, right, so we want their. Senses. Usually, we're least talking about vision sometimes we're talking sound, and touch and then, finally the goal is to put, the person into that space right so we. Talked about being mentally immersed when I give my full lecture on VR intro, to view our mental. Immersion, is not necessarily, a requirement of, the arts that it's the first two it's the physical immersion that's more of the requirement, but, we want to make you feel like that everything. Makes sense to you right I mean it doesn't make sense that you're standing in a molecule but if. You're, able to manipulate the molecule scale it move move atoms around things like that in a, natural way then we'll call that kind of being mentally immersed in making sense. There's. A three. Kind of major or two major and one minor way of we do that one. Is to put the display on your head which is what we're showing with the vibe here, one, is to have a display that these are enters so we call that cave or cave style that's, this middle picture here and. The. Middle picture and, so. That works where there's screens on all sides of you or most several, sides of you and also, on the floor although, we can also do that for example on this screen that we're watching here in the room it's, a four by four child screen if we want to just see something really large and kind of walk around things we, can do that as well I have. I've. Spent the last few days getting kind, of a basic. Example running on this screen which I can kind, of fire up after we're done it's not it's not as impressive as the commercial VR, that we're gonna see so. So. Yeah don't, get your hopes up too high for that just yet but we have a lot of different software that we can install and, and that'll work out well too all. Right so, our, motivation then, for doing for bringing. This. Virtual reality concept. To scientific, visualization, so. I'll, talk a little a little bit and. A few slides about the demonstrated, value of virtual, reality is an interface, of. It, being, natural, and people being able to use interfaces. Were six degrees you can move things around it's not like you're mapping a two-dimensional, Mouse into, a three-dimensional, world and with, rotations, it's even. More controls and then, as we've been talking about this semester. You know visualization. We're looking at different tools different. Ways of exploring science. Data so we want to bring those two things together and make it even better right.
So. We, want to have we're looking for then us kind of a suite of tools that we can have that, allow. Us to. Explore. Our data with, a natural, user interface, right. You. Ready Eric I. Can. Get to that picture of one all. Right so why do we want to do immersive visualization so, we just have some little bit of motivation but, I mentioned. Already that virtual. Reality gives you this natural, interface. With the world so you can actually explore, the world you. Can actually go inside your data which, in. Some cases may not make sense but in some case you want to be able to be there be in the space and look for, relationships, between things. So like, I say sometimes. That makes sense sometimes maybe it doesn't, one, question then is where is virtual reality fit, in your scientific. Analysis. Workflow right, so. In. General, there may be some exceptions but a general VR where you go into a VR experience, you're, not there to do the analysis, you're. There to go and collect data collect, make observations that. You can then just come, back from so I like, to make the, analogy of going into the field it's kind of like a field trip and. One of the my, past. Assignments. We, we. Had people who would go to Alaska, and go to Antarctica, and go to the desert and and. Collect data find things and so. Often. They would need. To teach their, new students, who are working with them the lay of the land and so one of the things you could do is is give, them a feel for the of the space before you go there but, that's just one. Example of, what you might do. Really. You're there then to take some measurements get. Some insights, and then come back to your office, space come back to your computer with your statistics, packages, and other. Analysis, tools and that's, really where you're, going to do most of your analysis, right and in, some cases you might actually use it to be. Recon, for an actual actual. Field trip right and, so, as I say so one. Of the people I used to work with you would take people to a lot you take students to Alaska, they, needed to find a valley that had certain properties, and in, order to figure, this out kind of get a sense ahead. Of time they, would actually bring up bring it up in virtual reality and you kind. Of look at some of the valleys and things like that a lot easier to do now that we have Google. Google. Earth but in the past they would actually you know rent helicopters, fly around and that. Was kind of expensive. So. So. Virtual reality then for this whole process is more I like to think of it more of a scientific instrument right so it's, a place where you take measurements, like if you had a microscope. Thinking. To us you know kind of a basic standard. Standard, I should say a scientific. Instrument and it, lets you get different views of your data right so if you have a microscope you're getting a different view of your. Data, and. Then you can also use it to enhance. So sometimes, the data is kind of messy it. Takes kind. Of a human intuition, to, figure things out but, maybe that intuition is, hard hard to bring about when you're only looking at it in two dimensions if you can actually see it three dimensions you can see relationships, between fibers or something like that then. Then. You can make this data enhancement in cleaning, tasks a lot easier I'll show an example of that as. We go on again. You can kind, of take a a. Tour. Of some data right so you can if. You already have a bunch of molecules you want to show your students a different bunch of your molecules and kind of take a tour all those molecules and then as I've already mentioned a couple times you actually do actual. Recon for an actual trip. In the field right, so, now this doesn't apply, to most. Of you unless you want to argue let you free to your management that you. Want a bigger virtual reality facility, but, what are the questions then is if for people who have VRS facilities, is how. Much is that facility going to be used right so in a lot of cases in. Scientific, spaces. The. Facility. Doesn't, get used 24. Or, not even 24/7, but 8 by 5 even, necessarily. Right 40 hours a week, and, so. That looks like kind, of it's not being put to good use, but sometimes, really. The question is is it being used enough.
Where, People are getting good benefit out of it right so it. Doesn't have to use all the time unless of course you're doing something like training where, you want the usage if, you're just trying to get people in there so they see relationships, then, they go back to their desk and then, and then they make use if it's like if you have a microscope, your scientists don't have to be looking in microscopes, you know all the time in order to be doing their science they have to look get some data make some try. And deduce. Things and then they go back and figure things out right so. So. If you're arguing, for a VR facility, even if it's not a big one like this even if sort a lab full of head mounted displays I like. To use that as one, of the arguments. Right. So one, of the issues when we're dealing with scientific visualization in. VR is that we, tend to have a lot of data and so. If you're doing you know for they are for games and things like that they, basically reduce the amount of information. They're presenting, to the point where it renders fast. We, may not have that luxury or, maybe maybe I want to. Get. To that luxury of having, things render fast by having less data we might want to see more, or a lot a lot a good percentage of our data and. So we, might suffer, with. Lower quality, rendering. Than. The games do so right, now if you are a game, company and you want to sell your, product, on one of the VR stores, they. Basically require that your game run at 90 Hertz or your product run at 90 Hertz, and. So that's, you know a lot, of processing. That's I think that's a battle I can't, remember exactly but 11, milliseconds, you have to do all your rendering. For. A lot, of you hail a big scientific data set you. Might not might. Not get to that point right you might have a much slower thing now the. Slower you go the worse, the experience, is kind of for, your, your. Brain right. It doesn't, look as natural if it's going very slow one. Of the benefits of having if, using like a screen like we have here this tiled screen that we have a big screen like that is that.
You. Can get away with, lower, frame rates and not have your body sort, of reject that and get nauseous from that because. It. Doesn't move around it stays stable it's you that's doing the moving and so, the images on the screen don't have to react as fast if. On the other hand you have a display on your head when you turn your head that path you know what it's what you see has to change immediately and so it's actually a lot more, strenuous. On your brain if for the for, the head mounted display than it is for the the cave style displays, so, if you're doing science you have a big screen you can get away with. The. Frame rate being, a little bit lower otherwise. You have to reduce your data right you have to reduce your data so, that you're not seeing all of it maybe you're seeing a subsection. Where. There's something important happening or maybe it's just seeing it all sampled down, and. So you. Can then do your exploring, and then again go back to your desk or maybe. Once you've Sam. Kind. Of perused your data at lower resolution. You can say over there in that corner of my data is where I'm really interested let's let's, look at just that corner but in higher resolution. Alright. And. So let's talk about then the cost a little bit about immersive display so here's. Our, kind. Of cave style display, that's up in Indianapolis now, just installed. This. Year and so. It's another tile display wall it's about it's. Twice as big as the one you're looking at and it curves around so you can have data on all sides of you so, that's kind of a big facility. It's going to be expensive. Again. If, you if you have a need you can argue for that that. You it'll help your science but, we also have now, commercial off-the-shelf VR. That's. Priced pretty cheaply the. Computer, actually costs, more than the, display at this point so, you want to get a nice computer that costs you know two or three thousand dollars maybe. But. Then at some point especially, if you're doing custom, code really, the cost the main cost is the people the people doing the code development, the. People processing, the data that's, where a lot of your time is going to be and so that's where if. You can save a lot of time in development, by, having a bigger display you can also have. A win there all. Right I said earlier that we talked a little bit about kind, of demonstrated, value of virtual reality so. I have here for. Research. Experiments. Done using, virtual reality starting, back in 1990, by Fred Brooks so. In the Fred Brooks case they have a molecular viewer will. See a picture of it later it had a force, feedback display, and they wanted to have two molecules dock with, each other and so, they wanted to compare a desktop, interface, with that. Versus. The Virtual. Reality Display, and, they found that basically I don't remember exactly what the task was but the the, subjects, in this task were basically able to perform twice as good, or yeah. Twice as good using. Using, the VR display a. Few. Years after that at. NCSA. Over, at Illinois we. Had a volume visualization. Tool and we did similar, kind of tests can people in the, on. A desktop interface, do, the do the same you know how well do they compare with a person, using the cave virtual, reality near face and they, found I don't remember the multiplier but they also found that the virtual reality interface was better. Kenny, Grich Allah who's, now. At the National Renewable. Energy Lab. Did, a research project on. Figuring. Out paths for. Oil. Oil. Drilling, and so. Again I don't remember the multiplier, but definitely the virtual reality had the the bigger benefit, and, the. Last one listed is Probot at Brown, University and he was looking at. Scientists. I don't remember. The. Exact task but they were looking at blood flow through, through, arteries. So, those are research, papers but there's. Also people who I think have just as kind of good of value who. Talk about specific. Cases where they had kind of a big win from virtual reality, so. Gil Bohr who was at Duke at the time he, basically he was just taking his friends showing them their cave at Duke he. Had some visualization of, a forest canopy and they were just having a conversation looking, at it and then they kind of something, struck him and this is one of the examples I talked about where you. You. Have this kind of Eureka, moment, you. Were only in the cave for 10 or 15 minutes having a conversation you see some relationship, you go back now, you're doing statistical, analysis, with R or whatever and and. So the rest of your work is sitting at your desk doing kind of traditional, stuff but, it was that that moment of insight that you got while you were in the cave. Oliver. Krause has a paper on several different scientists, who found benefit, using. Caves. At UC, Davis and then, Gary kinslin, dhileban.
The. Cave at, the University of Louisiana, Lafayette. And. Was. Again bringing a friend just to show him hey here's my data in the cave and he had, really, kind of literally just finished a paper on this very data set and while, in the cave with his friend he realized, something that he totally missed in this other paper and so, he he went back and he wrote he got a whole, new publication, just out of this one insight that, he got out of literally, just being in the cave for ten minutes so, these are the kind of things this is you know there's no published, thing there right there's nothing that's, not a scientific, study, in it in and of itself but. That anecdote, I think still lends credence. To the the value of virtual, reality for immersive or visualization. Right. So, I want to talk about kind, of a quick. Tour of some, tools that, are available for doing, visualization. With virtual reality and we'll. Talk about some examples as we go along I've, broken them down into kind of three categories. So, one is a special-purpose tool right where it. Was written just for this, task, another. One is general-purpose, where it's like pair of you where we can load in lots of different kinds of data and then we can just kind of output it to two VR and. Then the last one I call shanghaied, which, is basically where. You have a. Computer. Graphics program that does a particular task, it's. Meant for the desktop but, there are some tools who can capture the rendering, as it's on its way to the desktop and as, they capture that rendering they send that those polygons, to a VR display, and then you can actually see the polygons in the VR display there's. Some limitations, to that you. Know for the most part the tool then doesn't even know it's being displayed in VR so, if it happens to think well they can't see this I'll just throw this away whereas. In virtual reality you could turn around and see it the, application doesn't know that and so it may have discarded some of the data and. Also there's not really going to be a good user interface for, that right because the all. The user interface is going to be on the desktop none of that's going to make it to the VR. Alright, so let's go through a little bit of example so here's some old stuff so I talked about the, Fred Brooks experiment, that was with this nano manipulator. So. And, I mentioned I had on the slide in the previous slide this notion of custom applications, you, know 20 years ago of course we didn't have off-the-shelf. VR applications, we had to write everything pretty much ourselves so, there, was a lot of effort a lot of manpower womanpower, put into doing that and so. We. Had to. Spend. A lot of time and it took it took a lot of time to get to the place where he had a tool that was usable, one. Of the goals then at that point was to use some of these general-purpose. Zatia tools and add. A module, such. That you could then do virtual reality with that much like we just saw with a pair of you at. The beginning of today and, so these were some tools I have pictures of those so we'll see those in a minute here's this molecular. Docking, application. That I mentioned with the Fred Brooks study, and so, this is at University of North Carolina the. Haptics, force. Arm came from Argonne National Lab, and so, they were able to both see and kind of feel the molecular. Forces as they interacted. Here's, one from NASA Ames where they're doing. Visualization. Of wind, flow around a Space Shuttle kind, of one of the unique things about this then is the old-style the artists place so here's a head head, base display, but you actually had to hold it to your head and like you do with your phones now and there's a count in this arm here that is extending out there's, a counterweight out here because, there's you know literally like little TVs inside, this box here it's not like we have our smartphone, displays that, we use today so this box is pretty heavy so it's not something you would have to physically. Hold or wear. On your head and, so that there's an old display called the boom. But. The tracking was really good cuz had mechanical tracking, here's. Another one that I mentioned from Rachel. Brady at, the. National, Center for supercomputing applications and. So, she has this volume visualization, tool here's a chicken embryo and. I, think this one is a horse fed lock but, anyway one of the things, I mentioned already is this notion that we can't see all of our data right and so this the chicken embryo is sampled down data so that we can see the whole embryo there. But this other one this horse fetlock. Ernie data. We're, only seeing a portion of the data and that's this notion where well, we can we can kind of choose right we can see sampled down the whole data set or we can find the data that we're really really interested in and focus, on that and of course 23.
Years Ago that was even you know more the case even. With a computer here that costs, over. Half a million dollars right so just. To do the rendering we kind, of made it so that you can only see a portion of the data and that's what we're seeing with this this square in the middle and you can grab this square and move it around your data set to see different, aspects, of it and there's, tools so one of the things that she, was working with researchers, who want say measure. Paths. Through, data so perhaps the the brain stem of this, embryo. And so, since it's kind of curly and it's a little fuzzy data you could like drop these little breadcrumbs, which is where the name comes from and drop, them along the path here and then have at the end haven't report the link that whoops live the length of that path. And. So that was the task that she had and that she did in her study to find out whether, people were better at it in virtual reality or not. Here's. A quick one here's an atmospheric, visualization. So this was a desktop. Tool which, with, the source code we could actually add some virtual reality into it there's. Los Angeles, smog there and. Then the. General, purpose tools that I talked about so, a lot of them, are. Two at least two of these have gone by the wayside right they're very popular for their time but haven't. Survived. Actually. ApS may still be around but I, don't, know anybody who uses it but anyway they have these tools where you connect things and it was kind of like pair of you pair of you does this internally, you have something that reads the data somewhere something. That processes, adds color you. Know finds, I don't, I so, surface here so our contours, and then feeds that to some renderer well. The time we were able to create a render, that, produced. Two. Side, by side left and right views and let's you then interface with them and and the views changed, based on the virtual reality system. And so here again you could take your desktop tool and bring, it into virtual reality much, like we're doing with pair of you now and and. See that and. Then also, the. Visualization, toolkit which we'll talk about later in the semester if. It's the web interface for that but. There that was kind of a popular visualization, popular, toolkit that, had lots of visualization features, and you could connect together you, could trust. Kind of that the what, it was doing and lots, of people were taking that and putting in the virtual reality nothing. That kind of lasted, because we're because. Vtk. Itself was evolving and so things would change and eventually it would break but, there's, more to that story we'll talk about later so here's some pictures of vtk, from early days. Alright. So those were some early examples now there's some some modern examples, again. Here's a 3d, visualizer so this earthquake. Data in. At. The globe yeah you're actually standing inside the earth, and looking out to where the earthquake data, is you. Can generally see that it follows where these, tectonic. Plates are it does a lot of other things though it has slicing. And ISO surfacing, much like pair of you has and streamlines. Another. Tool from UC Davis is a lighter visualization. Tool here, we're just looking at points so light. Our, systems. Generate you know literally, billions of points maybe hundreds of billions of points and so, even, for modern, computers, with modern. Graphics. Cards that's a lot of data and. So the, researcher, at UC Davis Oliver, Krause developed. A system, that loads in the data where you're looking right so it focuses in as you get closer to something it fills in the point so it does the computer doesn't have to render all this points at the same time or, the one on the Left has kind. Of an a/b sampling, so it's terrain, from, two different times of the year and you, can see where there's a vegetation, growth and you can also see where maybe some, landslides. Happened, and or some building took, place, and. Then the one on the right is from power line observations. Where power, companies, want to make sure that vegetation. Things, aren't encroaching, on their their, lines, here's. A ESRI. Based so, our arc GIS based, visualization.
Tool For a cave. Here's. A volume visualization, tool which I want to show a little video of this one just gives you a sense for that so let me break. Out - oh right. I lost my VLC, so let me start up the LC here. So. Yes. The. I'll. Talk about that a little bit but these. Couple, last couple tools the one the lighter one and the earthquake. The one that was showing earthquake data and this one there written with a tool called bruit which, at the moment bruit only works on basically. Linux or UNIX, based. Systems, so it'll work on a Mac. He. Has got it and well I've got it to work on in our lab over at tenth in the bypass he, has gotten the. Vibe to work on Linux with. His, so we can look at this these applications. In our vive one, of the things I noticed though and you'll other, people have noticed is them when, you have that much data actually notice the resolution, of the vibe isn't, as good as you met. You might imagine if you're just playing a game and you're just kind of concentrate on the game or some other stuff or. Even a molecule, there's. Not that much data and so the resolution doesn't bother you so much but once you get to lots and lots of points that's when the resolution actually starts to become. Noticeable, so you, know the next generation of headsman displays I think I'll probably address that to to a degree but, yes so some of these tools and I'll talk about volume visualization, at the end for Windows too but some, of these tools the guy who authored. The. Base toolkit, is kind of a Linux guy and so. Some. Of these are limited in that way but. We'll talk about different. Opportunities later. So. I want to show the. Video of this which I lost my I. Lost. My bookmark, on that since my machine crashed. All. Right. So. Here's that that same tool that we're seeing in the in the picture here's that tool in a, cave and. And. So it has this interface and one of the things we want to do is be able to kind of what, in, this particular case is count the number of I. Can't. Microbes, or whatever I can't remember exactly what this data is but anyway you and count the or the cell. Nuclei. Or something like that and so, normally the the, grad students would have to go and look at 2d slices and say well there's you know there's ten on this slice and then look at the next slice well there's 12 on this slice but three of them are still on that slice and you have to figure out you know which ones are which whereas, in three dimensional you kind of go and just kind of tag them and you can and do much better more accurate job of, counting. The things, and also seeing again, the relationship between things and so, here's the part, of the user interface for that there's a, transfer. Function to convert the density. Of materials, in two different ways kind of a. Pretty. Nice utility. For its time so, that's. One tool. That. We actually have running over in our lab, at 10th in the bypass I've. Lost my mouse there. It is alright so let's pause that. So. Anyway so that's that tool in virtual. Reality or in motion, motion, here's. Another example, I need, my focus here's. Another example actually using the same tool here's, a couple of researchers. Went to the Sahara Desert and they want to see what, are the wind patterns underneath. What. Were the prevailing, winds five hundred years ago over the Sahara right, and you can tell that because, as, the prevailing winds change the shapes of the dunes change, and so, what they do is they go and they they take this ground-penetrating radar and every, meter they take a measurement right and they go through and now it takes a few days to make these measurements. We. Reshape. That to the actual terrain and then you can look underneath and see the shapes of these dunes. Over time and then they then, they in virtual reality as I, said you know you can use human intuition a little bit better it's kind of hard maybe, to have a computer autumn. Automatically. Figure out you know what are the angles, of the dunes and various places what kind of produced you want to look for maybe, with machine learning today we can do a better job with computer but back, you know 10 years ago this is what we had and so, you could with with, kind of the humans, ability. To deduce things figure, take measurements, and then again once, you have those measurements, take them back to your lab and that's when you actually do your your, kind of analysis. Yeah. So. This wouldn't well when you say you guys this, is when I worked somewhere else yes. That does a Research Institute so. In, this case they were researching, a desert, which. Isn't always the case but yeah, so. Here's one that I was not involved in this is from UC de UC, San Diego where, they have several caves and different facilities, and so, they have a, program. They call kal VR and, they were doing some archaeology, so they start, with lidar data which you can see here and then, as the ground is being dug away that actually rescan, it and then they can like archive.
Where. They found different, artifacts, and and. Have captures, of that and then be able to visit that in 3d so you can go back in time you know pre. Dig pre. Pre. Levels. Of I, don't know unearth unearthed nests and see. How things were here's. A similar. Tool this is from the supercomputer, Center in Stuttgart, Germany and so, they do a lot of he has a general purpose tool where you can bring in a lot of different types of data it's, kind of like pair of you accept. Features. Were added based on what the scientists he was working with. Wanted. To do. Here's. The picture, related, to the anecdote, I talked about earlier, with, Gary Kingsland at, the University of Louisiana Lafayette, and so this is I don't know if this is the particular data that he was looking at when he had the insight but, one of the things they've done here is have these like what they call magic lenses, so, they have some general data and then inside these boxes they can see different versions of the data and they can have those overlaid, and so you can only you can kind of like cut away parts of it and see. Different things you can tell this an old picture because they actually use a CRT. Monitor. Here's, another one on my show in the, vision, in, a video form and so, this is a another, one from UC Davis and so, this is one where she's looking at. The surface of Mars and has. Looking, for different riffs and different properties of different riffs but specifically. You'll see that there's a picture of the researcher, herself, in here holding, a little Wiimote and the. Reason that this is kind of interesting is that you. Can then use this as an educational tool so she. Was captured, in. VR, doing. VR I'm, gonna show you a video clip of it as a video but it's, actually stored totally, three-dimensional, so you could look at her in three dimensions you can actually be, in VR while, she's explaining, things and see her you. Know inside, the space with you so let me go back to the movies, here. Whoops. There. We go. Right. So this is really this is she's captured with a Kinect, right so $100. Game. Device. So. And, you know that the, Kinect I think the Kinect 2 is still out and you can still use it and still get it pretty cheaply there. Are places that are using multiple connects, to try and capture things there's, more sophisticated, stuff, lidar, systems, you know capture a lot of points but they're not really great at real time right because they're spinning and so and.
And So they kind of capture from the, outside in as well. There. Are people doing video captures, so, one, of the technologies, that does this that hasn't quite got to real-time is this you've seen in football games and basketball games where they have a, suit, you know really interesting play and they, want to show you the dunk, or whatever from multiple viewpoints and so, they they take cameras, like 30 some cameras, that are mounted around the stadium we're on the assembly hall and then. They take, that data right crunch it and now they can watch it you know kind of in 360, or move around zoom in zoom out. So. That's not really real time so when they do that there's somebody actually operating oh this is good I think he's going to dunk it right so start recording and start processing and then, about five minutes later in the broadcast they'll, show you that video, but. You know computers getting faster right you know algorithms, getting better so I. Think those kind of things will happen. Right. Yeah. So I mean if you're good with this, here, let me just run the video if. You're good with Oh some for the people listening, remotely the question was well, it's kind of the state of. Right. Right. Right. So. Yeah. That exists here with the Kinect tool this this tool that we have now this, particular tool runs on Linux so. You'd have to Linux both sides but other, people are doing it with other you know other operating, systems. Let. Me go and find this data though. All right so here she is here's Mars or, that sexually the Sun it looks like. Right. So this is some you can see it's the Kinect she's talking. I don't have volume. Control on my laptop at the moment oh, there. We go it's on the big screen. Right. So, this is again a video of her talking explaining, things let, me jump ahead while she's actually interacting, with, the data right, so here's Mars. And. So her. Hand is where you know she's singing in virtual reality she's in virtual reality doing this she's. Seeing it that way and so when she touches Aperta, nice, it from her perspective we're. Just another person seeing it from our perspective and, and the two perspectives match, and again, we're. Watching this in video but I could also replay, this in a head mount display or a cave and I could walk around to the other side of her now the Kinect is only capturing you know one side of her but. I could walk around to the other side I don't have to have this particular point, of view I can kind, of zoom in and all, this is captured, and, can be replayed in virtual reality this, is just a video replay, of that, and so. Again. This. Same tool can be used to have people collaborate, remotely as well and and. And. You. Know it's not it's not commercial grade quality at this point but it certainly worked workable. Right. No. So yeah, so the question is can she interact, with us as. Well no this is purely pre-recorded. This is her talking if I stand over there she's gonna she's not gonna look at me she's gonna look at the where. She was looking when she did this right. And. So I can move around but. And she and I, can pause her so it's like oh I didn't hear what she said I can rewind her, but. In this case she's. Not live this is all pre-recorded and, it's more of an educational, thing but, like we were saying we can do back you know back to back, and forth and see each other in virtual reality as well using, basically the same technology. All, right so we'll, pause that. This. Is Rui yes. Yeah. If you see those menus that are like that that's that's really. All. Right so here's. Pair of you so pair of you has a couple of different stages for virtual reality this is from a couple of years ago maybe four five years ago where. Our. Lab teamed up with Idaho National Lab and kitware which is the company that does pair of you and we, wrote in a plugin to, make it work in caves and walls and things like that and so, we could bring it into a system that already had VR running and make that make that work we had kind of some rudimentary interface, stuff that worked. Idaho. National Lab was funding this and so once they stopped funding it we stopped developing, the. User interface for it but. It generally, worked and and, I, own, a tional lab still uses it. Separately. And will see this a little bit the folks at kitware started, working on it for the head mounted display and with. The vive and oculus and we already saw that running earlier, here's. V MD which is a molecular visualization. Tool this picture is a little coarse but you, can see there's some molecule stuff going on there in the background and again, it's another it's a desktop tool this, time where, VR was kind of added as a plug-in, feature so VMD, knows there's VR in there you can map button. Presses on your controller to do, certain things start. Play start, an animation playing. Resize.
Things Move things turn, various, features, on and off in the molecular visualization. And again, we'll revisit that one in a little bit I mentioned. Earlier this shanghaied, notion, so here's MATLAB. Working. In VR and MATLAB, doesn't know it's displaying to VR here so MATLAB is displaying to the desktop and, you can actually see that in this picture on the right in. The background here on this side screen we're not doing VR we're just showing matlab's desktop, and this, data as it's. Being rendered to this screen here is grabbed and then, we, take those polygons, and we map them into virtual, reality so we can actually then walk around that same data, and, so, that's a specific tool that was done for MATLAB by a researcher, at Duke there's, some generic tools that do that. From. Commercial. Companies so mechdyne company, called mechdyne sells a product called conduit. A company, called tech vis sells, product called tech vis or tech relates excel I guess. And. So I call, these generic, in that and the notion is you put this on your system any time you're rendering graphics, to your screen you can capture that and bring, that to your computer the, problem is there's a lot of little kind of fiddly, stuff whereas, if you're rendering it in this particular way it might not work or if you're running in another particular way it might not work so they actually customize, how, they capture the stuff and how they process it for, particular, applications. So. It's not quite as generic as as. The concept, makes you makes you think it might be, alright. So that. I'm calling those modern but now let's talk about some really recent stuff so, here's. That vtk again and now vtk, is kind, of formally, introduced. And has the ability, to run in VR. And here we've merged VR, with rui if you can recognize those menus and. So there's some viewers for there's a volume velar there's a mousse viewer which is a simulation. Product, and. It's this actually being integrated into the vtk releases, so if you have vtk and bruit you can you can just make this work, here's. A program. Called Cephas, which was recently open source so this.
Guy Right here James money was, in the Department of Defense and he started working on it but any went to the Department of Energy and open. Sourced it when he was there and so you can get that and so this has basically. Time data so it has lidar data it, has drone, flyovers, with cameras, which you can merge I think it have other data and stuff that. You want to keep track of and you can again go back and forth in time and look at your data at different points in time. All. Right so I said we'd revisit vmd so a, couple. Of features we saw vmd in the cave that picture but, VMD has had, some extra things added on where works at head mount displays and, it. Does a couple of different things one is, VMD. Is kind of known for its high quality rendering, the. The problem of course is high quality rendering you don't get that 90 frames a second that you want and if. You're doing huge molecules, so this I think I, can't, remember for sure but I think this is an HIV molecule, right here which is just massive and. The. Processing, for that if you want to kind of be semi real-time in this case we actually the rendering happened in a cluster. Of graphics, cards in, California. And we, could watch that here at Indiana or @vm. Dee's home in Illinois and what. We would do what, the program would do is render that as kind of a big rectangle, meant to be wrapped onto a sphere, so it would render it then, you in the head-mounted display, you, were just looking at the inside of a sphere and so, you could kind of look around and see what's going on if you wanted to move you could move but then it had to refigure out the sphere again so. It would actually get kind of grainy and then it would get high-resolution and. So. So. You could get kind of half. And half you can do a little compromise you have high resolution data, process. Somewhere but, you if you wanted to actually kind of move your head side to side you, got to reprocess it but if you just want it to rotate your head and look around then, you were good you could look in any direction and, if we can get a sense of that I'll show some, movies, I have I can play the movies on the.
Laptop Or you can play it on your phone or I can play it in a web browser and, so. One of the things that, came out of that you can also do 360, rendering with the MD and so you can make your own movie, pre-plan. A flight pre-planned, some features to get turned on and off or you know have the simulation, running and then you can actually record that to a movie and and, John. Stone the author of V MD has put some up on YouTube where you can actually just watch that and, actually. Let's, see. If that's still in my browser tab here. There's, my Chrome thing there. Here. We go alright, so this is the oh I, lost it, alright. So I will go and just search it it's just on YouTube. If. You search VMD, oops. BMD. 360. There. It is. Alright. And, so here it is in 360, mode and, I can take my mouse then and rotate. It around right. So I can look in any direction these, are water molecules these red and white things and they, are working their way through pores. In a membrane like. A. Skin sort of thing. And. Then. Somewhere. On here if you happen to have a VR device you can click on the desired device if, you have this on your phone you, can don't, have to use your mouse to look around you can just use your phone to look around and I can show you that after we're done talking here so I'll just get out here. For. Those of you remotely. Viewing you have a little phone here so there's a little phone viewer that you can carry in your pocket. And. Put your phone on it and just run up that same YouTube thing. Run. YouTube on your phone and you can just kind of look around like it's a little head mount display right, I'll, show this for the well. That's not on the camera so. Those who are watching remotely I'll show you later. So, this and and that we can also watch that, that. Same, video in VR and so in VR I can just kind of naturally in the head R on display and I have that up and running too so we have there's a YouTube app for, the vibe right and, on steamvr and, you can just watch YouTube, 360s. In the v. Pretty. Easily, alright. So this is I think the hiv-1. Stop. That. Alright. So so. Via vmd, in addition to have that cave view has these lower you, know consumer, oriented views, as tools, as well. Alright, we've mentioned brewery a couple times and. We already addressed this so bruit bike. As, it was developed was meant for caves, via our big screens but. As I mentioned earlier, the. Author of Rui got it to work with vibes, and, and earlier versions of the oculus. On. Linux, and so you, can do Rui stuff here's a point cloud data set, of the, Temple of Isis, in. Pompeii, that's been excavated, and recreated. Through photogrammetry. Which we talked about last week and. So you can see that point cloud set data set you have all the features of the Rui interface in this in the head mount display again. I mentioned that, the resolution you know when you're looking at this kind of data the resolution, becomes. Noticeable. Alright, and then we're back to a pair of you where we kind of where we started we're. Open, VR is valves. Interface. To the vibe, it, also works with the, oculus interface, and basically, as I showed when we got started you. Just start, a pair of you you. Tell it you want to do VR you load up your pipeline, you press the button and you're doing VR so really. Conveniently. Done and I. Mentioned earlier the, notion of volume visualization, that tool that I showed earlier was for Linux there's. Some folks the team at Idaho National Lab that we've been collaborating with they've. Been looking at using unity, which is a game engine, and. Works under windows of course as. Well as OSX. And Linux.
But. While. It's meant for games it has the ability to actually do open, GL shaders in it and with, Open GL shaders there's, a lot you can do in there to manipulate. Data. And so, they're developing a volume visualization, tool inside, a shader, which they then just use unity as their kind of the shell to hold that and and. Since unity already knows how to do VR it, all kind of works together I contacted. Them a couple weeks ago they're not ready to share it yet so I don't have a running copy of this just yet. And. Then. This. Is we, looked at this data two weeks ago when we're looking at pair of you this is the globular cluster stuff from an professor. Of every knee so. A couple of our team members took. That data and made kind of a fun application. Of it in. Order to demonstrate. It to you know kids at the at. The science fest right, and. So this if you go to this URL down, here you can actually run it again on your phone also. Though, we. Can run it in the vibe just like, there's. A program called, super, medium which, takes. Takes. Web-based, VR. Spaces. So there's a. Programming. Interface called web VR, which. Is like WebGL takes, graphics, and puts it on the web it takes those graphics, ads VR to it and. There's a product, called super medium, which, will load those web pages and just let you watch them in the vive and so this is one of those webpages you, can just watch in the vibe and I, can demonstrate that this. Afternoon as well. Alright, so. Those. Are kind of the bulk of you know tools designed, for scientific visualization there's. A handful of other commercial software, that we. Could kind of make, use of for visualization. So this one's called prospect. Pro it's an architectural, based. Tool. It'll. Load though any sketch, Sketchup. Data set so if you had science data and you brought it into Sketchup it'll, load wavefront, models so if you had a molecule, you could load that in and. You can do annotations and, you know and this one lets you share as well so this one actually, works.
Across The network so you can create, what they call a meeting multiple. People can join these meetings and all. Be in the same space and they have these little crazy-looking. Avatars here. With. A little kind, of pinpoint stick, so you can see where their hands are where the heads are and and. When you're in there you can annotate so over here in this left. Most, image these, arrows and blue lines were drawn by people in. The space they're not part of the model they're actually drawn there and those are persistence, so if you create a meeting that. Meeting any. Annotations, you have stay there and then if you join that meeting later. As, long as the person whose computer, this is being hosted on is up you, can actually see those annotations from the past. Right. So it's not cloud based so it's it's. On somebody's, computer and then, they join in, to that meeting and then it. Goes I mean, it's it goes through the company iris iris VR is the name of the company and then they kind of connect you at that, point. I'd have to I don't know you know hun percent the details on that so, that's. My understanding but I could be wrong but I've. Tried it with like up to three sites at a time and. Usually. I start the meeting one first but that may also be just from habit right i. We. Can we can investigate that and try, it out we have prospect Pro on this system but I only have it I don't have it on the back one so we can't do the collaboration like if you want to stay around long enough I can install it back there but but. We can look into this later too. Yeah. So oh yeah so Jeff is online Jeff the question, was although, I can't hear Jeff unfortunately. If. You want to type in the chat message, Jeff the, question, was how does how, do prospect, Pro share data isn't like a cloud-based thing or, does. It go through iris, VR. So. We. See if I can start the chat up here somewhere, soon. So, you're saying you can't hear me I, can. Hear you talking Jeff but I can't it's so mute it's so low I can't hear you're saying alright, I'll type, it. Something. Is screw with my settings. And. I can't find lis share either actually, someone, else are you on the chat essen. Alright. So, Jeff. If you could answer that question in the chat space, Eston. Will tell us the answer. And. If you have a question about the question we can clarify. Another. Tool is called gravity sketch so, this is more, where. You you start with, a blank slate and you start designing things and it has tools to like mirror things and do smooth. Surfaces, and create you. Know primitive. Shapes. And. So. Again, this is kind of more for as I say building stuff if you want to just sketch out some things pretty. Sure this one is not a collaborative. So this would just be you you could save the data and load it back but it's not collaborative like iris. Pro. Prospect. Pro is. Tilt. Brush by Google it's another kind of famous one, this one's continually, being enhanced, because people are actually starting to use this on. A more regular basis, and so Google's been enhancing it but it's really just a paint program with some fancy paint brushes it, does have the ability to mirror things and and and do draw straight lines and things like that, so. And. You can walk around I can run tilt brush of anyone wants to see tilt brush later and then, finally as Google Earth which. Especially. When, I as I mentioned before if you're going out into the literal, field and you, want to do some recon, on that Google, Earth is a good way to do that find out where different places are. And. So that works pretty well so that's. At, from, so at this point we can now do some demonstration, you can try, things out we have a vibe running up front we, have another vibe running in the back and. For. A lot of things like now the, molecular, viewer the calc flow, I forgot to mention calc flow. That's, like get lost I guess and when my machine rebooted. So. There's a couple of commercial products called. It. Well. Actually so there's some pictures of it here so let me just talk to it here so, here's.
Calc Flow and, use the mouse. So. This is a. Commercially. Based tool you can get it in the steam vr so run out of your rift and your, vibe and it shows different ways of doing like mathematical, expressions, and it'll plot those out for you the. Same company, did this program called and. So this is actually this was a student project to the University of California San Diego they, started this up they put it on the. Steam store and sold it for like ten dollars and. From that they decided to start a company you can now download for, free they've open sourced it and. Then they now, they're working on this molecular viewer where you can here you can see they're manipulating this DNA molecule but. It'll load basically any protein, databank molecule. And you can manipulate, it this one is collaborative, as well so people multiple people can be joined in the same session, and and. Talk to each other and point, out things to each other. Joe who had to leave early today I let him he arrived, early so, because he knew he was leaving early so he actually did the little tutorial in this earlier today so anybody wants to to try that out we can we, could see that too so. Any questions before we go to the demos. The. Molecule is called, nano, me, so it's ma n. Ome. And I've. Heard it pronounced nano MS well but. I don't, know how they pronounce it actually. So. Their company is that. Nano me dot, AI I think. Right. So the, question was about for. Those who early the question is about standard, data types that are available and. So. There's. Several standards right from. For molecules protein, databanks kind of the main standard so most molecular, tools are gonna be able to rail load PDB. Files. For. Lidar. For, point clouds there's lis which, is a standard, and so, Oliver's. Program lidar viewer will, we'll take that and then pre-process, it into his format so I'll for lighter viewer as a special, format because. He's loading in data and taking it back out again but, you can take, standard, data from lis and or just ASCII comma, separated values and process. Those into his format, and, then you can actually take his data and go back the other way too if you want. If, say you erase. Some stuff you know you go in there you process it you want to erase it some, data like if you did photogrammetry, and you had some data points that you didn't want you, go in delete the data points you don't want save back the ones you do want and then do the rest of the processing, that, way. And. Then. For like pair, of you well two, weeks ago always saw the huge list but really. Just kind of you know X Y Z plots with, data at least you know several bits, of data at each thing right. So. There, is there are a couple, of tools which I didn't show you that. Haven't been developed in a while but there is one I can't remember the name of it but basically is a, point. Edge. Visualization. Tool connectivity, tool. Cannot. Remember the name of it but there's a bruit tool that does that. Right. So, for those who are only a recommendation for, brewing. Essen. Yeah. Let. Me actually I'll, give you the microphone. I'm. Gonna relay Jeff's response, to the question the prospect, that parent that does not store. The data centrally, so, a user, who, wants to collaborate with others creates, a workspace. And all, the data is, shared, with the participants, and they, have to copy that data locally, so that everyone, has a copy of the data and they, collaborate on that data using. The local, apps on their machines and, apparently. There's some. Desktop. App it, works at prospect. Someone. Doesn't have VR they can still participate in the meetings so. Okay good. What's. That. Right. Yes you have to ship the data around and so, I didn't say so in it kind of an answer back to this question about data state so, prospect, pro will load kind. Of standard architecture stuff Revit it, also load sketchfab, or sketch Sketchup. I'd. Not familiar with that one but. Step. Files Jeff prospect. Pro.
Right. Yeah. It has a couple, of different formats that are loads yeah. Right. No it sounds like you're responsible, for sharing, the data on. Different machines I guess I I should, have remembered that because I did that myself he says what Rhino. Yeah. So. But they, are actually adding I mean they just added a new a, new file form and I think last a month or two ago so you, know they're addressing they'd say yeah it's a licensed. Software so you're paying for that one so you can get some customer. Support you know out of that and tell. Them what you want right. So. Anything Rhino can load okay which includes step I guess. Any. Other questions, otherwise we'll go and do the demos and we'll stop our recording. All. Right very good and so, next week is augmented. Reality and visualization, and chauncey, friend will be presenting that. All. Right.
2018-10-02 14:49
