- All right, next up, we have Tom Flynn and his presentation is "Developments in Online 3D Visualization." Tom is the Cultural Lead at Sketchfab and the co-chair of the IIIF 3D Community Group. Tom has also been extremely helpful with my team in launching MoMA's first 3D public model of "Starry Night," so let's welcome, Thomas. - Thanks Robert, can you hear me okay? - [Robert] Yeah. - Okay, I will just share my screen.
All right, well to quickly restart the things I'm gonna quickly cover are some context and trends in creating and publishing 3D online. The idea of what making 3D data available for download means and the potential of interoperable 3D. And I'll come to what interoperable means later in the presentation. A quick background on Sketchfab, it's a community-powered platform for publishing 3D models online. We just passed 4 million members this year, and it covers individuals, 3D artists, 3D designers, architects, as well as museums, libraries, and cultural organizations. The basic premise for Sketchfab is that you have some 3D data and you have an audience that you want to show it to.
And Sketchfab is one way of uploading data to Sketchfab and then your 3D data exists on the web that people can interact with. And that's the platform very simply. There's lot more to it and I totally recommend having a look at the site if you are interested in some of this. 3D and all 3D models in general, the idea that 3D models can be used in many, many different contexts, in many different ways. And whether you're showing them or using them, specifically the 3D data files, or if they form part of another experience, whether that's an educational experience or a commerce experience, this idea that 3D models have a use the same way that 2D images and videos do. Last year, there were some pretty big investments in 3D from some of the largest corporations on the planet.
Whether that's the ability to make 3D models more easily or to experience augmented reality in a more kind of a more solid way if I can say that as well as different ways of experiencing 3D models, whether that's through a screen or through a virtual reality headset. And obviously these devices are not cheap, they're not available to everyone, but in terms of a trend of consumer electronics I think this is something to bear in mind. Obviously, there's this idea of what a 3D model is, and maybe this is obvious to a lot of people on the call now, that you have a 3D file that you can create derivative works from, for example, here is an image of the 3D scan of "Starry Night" that Robert mentioned a moment ago. And while this is a picture of that 3D file, it's not 3D in itself or a video or an animated GIF, or also not 3D necessarily, they're derivative media.
And now considering what my laptop's been doing I'm gonna see what happens if I click this link and try and open the 3D model. No, I didn't think it was gonna work. We'll come back to that.
But if you go to this presentation, you can click that link and experienced the actual 3D model, the interactivity, that's what I'm trying to get at here. But the real reason that you might want to explore a 3D file is to do it in 3D and the idea of interactivity being quite important to what 3D offers. Going back to cultural organizations on Sketchfab, there are about 837 verified museums.
So these are institutions, organizations that I've had contact with on the platform, as well as thousands of libraries and archives. And they join under free or discounted subscriptions depending on what they are trying to do with 3D. What their needs are and the audiences that they're trying to reach. We're also seeing... For a long time, we've seen a community of individuals and enthusiast 3D scanners who have been digitizing objects spaces from all over the world via the techniques that have been mentioned by Alonzo and Carla and Mark. And these particular images are from scans made with one of the latest iPhones that they're not very high resolution, they are by no means at conservation level but they are made in a matter of seconds and they do convey the sense of an object and also the story, the context of these things and I think that, that trend that we're gonna be seeing more and more people creating 3D content the same way that we snap photos with our smartphones is something to think about.
And I would imagine the fidelity of the output, the resolution of the outputs from these devices, and the speed at which you can produce and capture sometimes maybe temporary or living objects, people scenes, is going to kind of increase and become more common. I added this slide, right just before we began having listened to Alonzo talk about crowdsourcing and we just published a blog post or rather, the Our Shurija project published a post on the Sketchfab blog about the Shuri Castle Digital Reconstruction project. So October, 2019, fire destroyed Shuri Castle in Okinawa and the project very rapidly was undertaken to crowdsource images to produce a 3D model but then also to use that 3D model onsite at the castle for school groups, in this case, to experience in some way what the castle was like before the fire. And there are other examples as well. I should say again that the presentation has links to all of these projects, if you're interested. So now onto that idea of making 3D data available.
And a lot of what I'm talking about is covered in an introductory handbook called "glam3D.org" that I wrote in collaboration with Neal Stimler and Michael Weinberg at the Engelberg Center for Innovation at NYU. And it gives a broad introduction to the topics of 3D copyright and Open Access, which is also what I'll be talking about now. So I think there's a difference between making a 3D model available for viewing, so I can access this visualization of 3D data, I can turn it around, I can interact with it, which is different to, but not mutually exclusive to being able to download a 3D file onto my own computer and make it a part of whatever project or thing that I'm working on myself. Some considerations that I have kind of picked up from my work in this area that people generally care about when they're producing 3D models of cultural objects but this could apply to pretty much anything, the idea that you would want to consider: Are you ethically able or should you be digitizing something? Are you legally allowed to create a copy of something? These points go for cultural organizations themselves and museums but also individuals, for example, visiting museums, taking pictures in museums, creating 3D models from their visits to museums and cultural sites around outside of museums, so things that aren't in a museum collection. The technical consideration file formats were mentioned earlier.
And this idea of, can people technically look at that 3D model, can your audience interact with it? Is it for a scientist in a lab or is it for a student in a school classroom? And then there's this practical idea of just because you've published something doesn't mean that people will be using it or accessing it. So, to go a step further than that technical hurdle of a lot of people can look at it if they want to, you also have to kind of consider how you can get it to the right people. At Sketchfab, aside from making 3D models available for embed like a video online for interaction, it's also possible for people to make their 3D data downloadable. So here's a picture of the download pop-up when you click download you get shown that this model is free, you can download it, it's under the CC attribution license, so the author must be credited, commercial use is allowed. And we recently added just below the description of the license, a simple button to copy-and-paste the credit.
We found that no matter what license you offer your 3D data under, it's very hard to enforce that license unless you hear about what somebody has done with it, unless you are following up on, what people are doing with your data. But we do try hard to encourage users to respect the licenses that things are made available under. We even have a Help Center article that was written about how do you credit, you know, very simply it sounds simple.
How do you credit somebody for their work and examples of how you could apply that to whether you've made it an image of a 3D model and put it on a t-shirt or you've used it in a video game or a film. Last year, Sketchfab launched a public domain initiative for cultural heritage. And this was something that had been brought up by the cultural heritage community on Sketchfab on several occasions that people and organizations didn't want to be applying Creative Commons licenses to their 3D models especially when the artwork or the subject was out of copyrights or in the public domain in the first place and they wanted to make it available under the right dedication.
And this was launched with some 23 organizations and we have about 2000 models that we're constantly adding to a collection of cultural heritage, public domain content and making them available for people to download. And you never know really kind of where things will turn up. This is a, still from the video of a music artist called Post Malone. It's been seen millions and millions of times on YouTube and it features a 3D scan from the National Historic Museums in Sweden, very briefly for about two seconds. But this idea of the ways that 3D can be reused is quite interesting where we're just seeing the beginnings of how data will be used.
I briefly want to talk about application programming interfaces and this is a technical term for being able to access the same data via code as a web developer or as a computer coder. So not through the Sketchfab interface through clicking a download button, for example, but being able to kind of do interesting things by programming, we can connect the library of openly licensed content on Sketchfab to other creative programs so there are a number of importers that Sketchfab has integrated with certain creative software. So this goes back to the idea of putting 3D data in front of the right people, in front of 3D creators. In this case, who are using Spark AR, which is an app for creating augmented reality experiences for Instagram and this kind of facilitates the use and reuse of data.
Another way might be to connect existing, in this case imagery with a 3D model so that you can cycle through different spectral views on a 3D model. If you visit this, we call them configurators. You can turn the model around and switch between the different wavelengths or rather the subject viewed under different wavelengths of light all through using the Sketchfab API. The State Darwin Museum in Russia 3D printed a skull of a primate and filled it with some electronics that you could hold in your hand and as you turn this around it would also manipulate the 3D model on a screen that was also hosted on Sketchfab. So you're, you're connecting something physical with something digital.
And other examples are a touchscreen again at the National Historic Museums in Sweden. The Cleveland Museum of Art, the touch screen on the left is an open-source project for a kind of custom interactive for museums that the State Museums of Sweden released. And then what you see on the right is in the background two people interacting with a 3D model projected on a wall, using their bodies, they're not touching anything via a connect device which kind of scans your body, recognizes your gestures, and that's how you can interact with that 3D model. So making data available for viewing in different ways and all these models, I should say also exist on the regular Sketchfab platform. You can also look at them on your phone or your laptop or your desktop. I'll try and go quite fast here, 'cause I'm sure I'm running out of time, but interoperable 3D is something that I've tried to give a definition to here.
The interoperability is a set of living technical standards and practical workflows that allow for the same piece of data to be accessed, visualized, and reused by as many different audiences as possible, in as many different contexts as possible. So you can see how that follows on from this idea of an API making things available in different ways. And the International Image Interoperability Framework or the IIIF consortium has been achieving success in this area for many years with 2D imagery and there's a 3D community group looking at developing similar achievements for 3D data through things like APIs and as I say, more practical workflows and standards. These slides are from the fall meeting of the IIIF group from last year, it meets monthly.
There are 10 calls in 2020, and there's a range of attendees from the Smithsonian to Google to Mozilla to the Met museum and more contributing their knowledge of 3D and their use cases for 3D to the group. Simply put, we're moving through stages of building a community around the professionals working with 3D for cultural heritage. We want to understand needs to help us begin technical discussions that we can then produce demonstrations of interoperability and then a long way down the road through, a kind of a repeating process, we should achieve interoperable 3D. That's the dream.
And simply put, one way of looking at this is that you have your 3D data... Going back to that early slide. You have your 3D data you keep it on your web server, for example.
You want to show it to people through Sketchfab, but also other 3D viewers, to reach certain audiences, to do certain things, and interoperability would hopefully achieve that. And it's not to say that only one 3D viewer could reach one audience. The idea would be to make it possible to reach as many audiences as possible in as many ways. And this could also be about disseminating the data, distributing the data itself, as well as making it available to view and interact with. Some very simple use cases for interoperability, is, I want to embed a 3D model on my website, I don't want to have to really pick a 3D viewer, I want to just have options. I want to add annotations, clickable information hotspots on a 3D model.
How would that look in different 3D viewers? I want to share a URL that displays a particular view on a 3D model. So, I want to be able to sort of show the underside of an object or the left side or the right side the top view in a uniform way. And there are other, other use cases for interoperability that people have supplied us.
And if you have ideas yourself, the IIIF group is gathering 3D stories or user stories, a simple way to define kind of what you're doing with 3D. It begins with, as an individual, I want to do something with 3D so that I can achieve X or Y or Z. So please contribute if that sounds like something that you're already doing. And it goes back to...
I'm just repeating this slide, but I feel like a requirement for interoperability is to have checked all the boxes that we've got there and going back to what Carla and Mark were saying and Alonzo about metadata and really being able to understand what you're publishing, or at least making it clear to audiences what they're looking at, is very important. And one way we're trying to do that is through something called a viewer comparison project. So, here's the same 3D model viewed in four different 3D viewers. And you can see kind of similarities and differences.
And that's what we want to kind of start overlapping is not just the way things look, but the way things work and the functionality the services that these different viewers can offer. And I'll leave it there. I know that there was a lot of different ideas, but I wanted to just give a snapshot of where I've seen 3D for cultural heritage, interoperability, ethics, copyright going.
So thanks for listening.
2021-02-02