hello everybody and welcome to the first pre-recorded lecture of 6837 introduction to computer graphics at mit fall 2020. this lecture will be a little bit shorter because we're also going to spend part of our class going over course policies administrative details and so on which is something that we'll do live rather than in this youtube recording so our plan for the recorded part of the lecture is to go over a few things we're going to introduce exactly what computer graphics is which is a particularly diverse and broad discipline i'll give you a bit of information about what this course 6837 will cover then we'll do a quick overview of how our semester will be structured how we plan to dig through the computer graphics world and finally i'll give you a little bit of information about the assignments that we'll be doing in this course so let's get started in particular let's talk about what is computer graphics and where does it appear so the main question to get us started today is to ask what the applications are of computer graphics i think we can all think of different places where graphics appears from you know fancy movie effects to video games and other things that basically appear on our computer screens and on the screens that surround us every day but the reality is that computer graphics is one of these technologies that is so pervasive we don't even think about it and so i challenge all of you to step back for just a minute and think about the different applications of computer graphics technology so unfortunately because this recording is uh done not live uh i can't start a discussion with you all but hopefully you've thought a bit about some different places where you can see computer graphics and let's go over a few examples so i think the example of graphics that comes to mind for most of us are in the movie industry uh of course there's both animated films like these pixar dreamworks films as well as the really crazy special effects that we see in hollywood blockbusters one of the really subtle aspects is that even in the special effects world some of the graphics that we're seeing are actually quite subtle you know i think when we think of movie special effects and computer graphics we think of you know splashing and fire and explosions and all that stuff but the reality is that almost any major production these days probably includes a pretty significant computer graphics component so it probably doesn't come as a huge surprise but your instructor is a huge computer graphics nerd and can spend all kinds of time watching movie effects breaks down breakdowns and so on one thing that you should and can do is to go onto youtube or whatever your favorite streaming service is and to take a look at some of the videos that movie studios have put out there oftentimes there's sort of promotions in the computer graphics world just to show the different special effects that go into a typical movie scene so for example uh here's one uh from ilm which is one of the uh special effects studios industrial light and magic for the avengers film let's see if it'll load on my computer here and essentially what happens in these little reels is they show you know some of these scenes from these blockbuster films and all of the different graphics that go into them right the breakdown of that frame and all the constituent parts that are needed so you can see there's a physical aspect here a digital aspect here different characters probably some of them are motion captured maybe others are animated there's a very significant simulation piece and all of these things are composited together into what i would consider to be pretty much a photo realistic scene right this is indistinguishable from real life except of course that we have a crazy camera panning around in a spaceship that doesn't exist but it's just amazing to watch these different special effects breakdowns and to see all of the different things that go into these these these animated characters the scenes around them uh and so on here's you know another one with uh another marvel character and you can see the city being built the geometry that goes into it the lighting the shading and so on and these films are just incredible there's a it's basically an entire category on youtube i would encourage you all to dig around personally i can i can watch these things for days in fact i could watch them right now but i guess i should probably be teaching class of course movie effects aren't the only place that we see computer graphics technology the other obvious answer to my question of what are the applications of computer graphics is in video games video games motivated a lot of the modern graphics technology that we have today specifically you know the graphics card which is sitting in your laptop today's video games in fact i just updated my slides to include some screenshots from new video games a few from old modern video games have reached really stunning levels of realism you know to the point where when you're playing these games you can suspend its belief altogether and it almost feels like you're looking into a window into some other universe of course not all video games are intended to have that effect for example i think these days probably the most popular one uh that isn't photorealistic might be you know minecraft which has its own look um and that's also one of these big exciting aspects of the computer graphics world is that we're not constrained to physics we're not constrained to simulating light properly we can take lots of liberties to get interesting artistic effects um which are compelling to viewers and players in a video game even if they're not particularly realistic from a physical uh perspective and so the tools that we give artists and game designers today allow them to create entirely new worlds in some cases new laws of physics and and essentially the technology is absolutely amazing when it comes to what we can create and how efficiently we can do it now the video game industry increased demand for graphics technology that's real time you know the big difference between video games and movies is that it doesn't matter how long it takes to produce a single frame in a movie right the movie plays back uh 30 frames a second but when the different frames of the movie are being rendered it might take hours uh in a video game i expect immediate feedback right i interact with a character and i want that character to respond and what that means is that video game technology has to be extremely responsive and efficient so because of that drive for reaching this magic number you know minimally 29.97 frames per second or else your eye is going to detect that the video game had a little bit of a pause modern graphics cards are these incredibly complicated and sophisticated electronic devices that can carry out millions and millions of computations per second in fact one of the big triumphs in the ai world is the realization that graphics cards aren't just useful for rendering video games but actually almost all of the state-of-the-art machine learning technology that is so popular right now is built on graphics card technology so if you're wondering where the modern advances in ai came from a lot of them actually have their history in computer graphics and the hardware architectures that were developed to render video games and make these environments that are responsive to users that aren't willing to wait you know 30 seconds per frame but video games and movies are not the end of the story those are probably the two most obvious examples of computer graphics technology but there are so many more for example uh another popular application of graphics in the 3d world is in simulation here i show you guys two screenshots from a brand new flight simulator from microsoft that was just released that are absolutely stunning and really can put people in a virtual environment to test out technology or learn a new skill before that skill becomes absolutely critical you know i would much prefer for you to learn your airplane in a virtual environment than actually flying an airplane where you can put yourself and others at risk virtual simulation and in silico testing also has really revolutionized the way we do product design it used to be that we would have to make all kinds of prototypes and that could be quite expensive but now computer graphics technology coupled with physical simulation algorithms and rendering and so on can preview how an object that i'm going to manufacture will look in the real world how it'll respond to physical forces how people will interact with it and so on at a fraction of the cost that it might take to actually build that object physically related to that are the universes of computer-aided design and computer-aided manufacturing or if you're in the know this is sometimes called cad and cam as well as design more generally pretty much every object you have today everything from you know cleaning fluid to bicycles to mechanical parts to you know your watches tables furniture went through some computer graphics visualization before it was actually manufactured you know these days if you take a look at a furniture catalog for one of the big stores like ikea chances are the furniture catalog was actually produced not just by asking artists to maybe draw simplified line drawings of the different furniture you can purchase or by taking photos but actually from computer graphics software that can be used to produce and customize furniture on demand in fact one of the really cool things that i learned recently about some of the online shopping sites like wayfair is that sometimes the really photorealistic images that you see where you get to actually test out and see what furniture looks like in different environments they're not even photographed they're actually just renderings and that's what's allowing these features where i can like swap out a chair and the room stays exactly the same and the amazing thing is that the graphics technology is so good we don't notice the difference you know if the rendering were really poor then you could imagine these companies experiencing a bunch of returns where people say you know okay it looked great on my computer screen but it doesn't look so good in person and of course that does happen but the advances in technology in the graphics world have really reduced the risk there now at a slightly larger scale the architecture world uses graphics technology extremely widely of course a building is a very expensive object to manufacture and typically only get one shot so nowadays architects often work basically digitally for a large part of the work cycle before anything is created in the real world technology allows us to apply computer graphics to previewing architecture not just in the sense of having a virtual walkthrough of a building which is of course extremely important but also understanding things like how the building will look in different times of day maybe you know when light shines on a particular location on the building it heats up too much and that's a computation that we can do using essentially the same algorithms that we'll cover in this course and so on and beyond that of course one thing computer graphics technology allows us to do is to communicate to a non-architect what the final product might look like to help people make a decision on the kind of designs that they prefer so here's a simple demo video that's being done in real time where somebody's taken an architectural design and actually rendering it with realistic lighting and reflections furniture and so on and in real time somebody's able to walk through this virtual environment and get an idea of an apartment that they might want to build before it exists in the real universe now this visualization is already several years old the modern technology is even better and can even incorporate some aspects of computer vision to combine you know virtual models of what we haven't made with physical models that are scanned of some aspects of the environment that you don't want to change in fact some of the really amazing technology just recently in your cell phone involves augmented reality where you can hold your cell phone up use the camera to input the environment around you and then computer graphics technology might put on top of that camera image you know a piece of furniture and object that you want to preview and it actually tracks as the camera moves around so the camera kind of acts as a lens into a world that almost exists but doesn't quite because you're introducing a new virtual object that really isn't there now one of the modern really popular applications of computer graphics technology is virtual reality i was just updating the slides this evening for 6837 and had to throw away the previous virtual reality slide because it's completely out of date but these days you know companies like oculus htc and so on are making vr equipment which is efficient and high quality enough that people can wear it essentially without getting sick um so virtual reality universe like putting on goggles that simulate a new virtual environment and beam it directly into your eyes for a long time it's been around but vr technology used to be pretty bad right in particular there's a big problem which is that i might turn my head really quickly but because the graphics technology and the processing wasn't fast enough or the tracking wasn't good enough when i turned my head the environment would kind of turn a second after it's kind of like wearing those those goggles that you wear in high school that tell you not to drink and drive but because of that vr technology really struggled to catch up with human perception and made it less than popular these days that's really changed and vr is becoming mainstream largely motivated by developments in the computer graphics world and of course that's everything from the high end you know the oculus rift and even fancier equipment from there but google recently made a really funny prototype with the google cardboard which is literally a cardboard box with two lenses and you just slot your phone in there and actually the phone is good enough at tracking your motion and has enough computational power to make for a reasonable if not totally perfect virtual reality device now computer graphics technology appears in other places too another universe is in visualization you know when you turn on the news and you watch the weather forecast and see the weatherman interacting with the screen behind them of course that's computer graphics technology but graphics tools have also been used to visualize physical systems biological phenomena medical phenomena and so on so for example uh here's a recent example from a computer graphics paper that's used to simulate and then visualize the structure actually going in on inside of some smoke in this case the smoke is being simulated using these interesting vortex ring style structures and so on that capture the swirliness i do not recommend you guys implement this paper for your final project because it's extremely difficult to understand but one of the things that i think we often neglect is that because the computer graphics visualization domains really require high quality physics some of the state-of-the-art solvers for physical problems on the computer have been introduced in the computer graphics world and graphics technology can be used to visualize physical phenomena and structures and underlying things that of course would be very difficult to capture in an experimental apparatus now one of the funny things about computer graphics technology these days is i can go to a conference like siggraph which presents the state of the art in the graphics community and even the negative examples like some of those rings that you just saw a minute ago looked perfectly fine to me you know those are the examples of the software failing but in any event these are extremely extremely impressive physical phenomena being captured and visualized on your computer okay now the computer graphics world also has had an effect on other industries for example the medical industry of course i would much prefer that a surgeon try out their skills and experiment on a digital model before they experiment on me but beyond that computer graphics technology is very regularly used in visualizing medical data giving people a better idea of what could otherwise be an extremely difficult to interpret scan of the human body for example from mri ultrasound or what have you this is used to visualize anatomy understand large groups of subjects at a time or even plan something like a surgical procedure computer graphics technology also appears in education graphics has really revolutionized the way that we present all kinds of different ideas everything from this famous video that tells you how to understand the stereographic projection to physics and engineering to you know biochemistry anatomy and so on in fact uh this year 2020 of course is a really fabulous test of all the computer graphics technologies that we've been using for education visualization and so on because many of you are essentially doing your learning independently at home looking into your computer screen all day now when we think of computer graphics i think we tend to think of the computationally intensive computer graphics applications but there's another application of computer graphics which is equally if not more important and that is on your phone i mean these days your cell phone probably contains as much computing power as some of the really state-of-the-art machines not so long ago at least from your instructor's perspective and of course the computer graphics demands on your phone are quite different from the demands of other applications right on the phone you can't draw so much electricity you probably don't want as high quality graphics but in addition to that you're interacting with a different set of tools so for example uh geographic information systems this is often abbreviated as gis and then gps global positioning systems are really critical uh to getting all of these directions that your phone displays to tell you how to walk around town and this is a great interaction between the physical world and the visualization and computer graphics domain right when you have your phone attached to your car and you're driving through the city uh computer graphics technology is being used to render that map and keep it updated live with uh the gis system which behind the scenes contains the map the roads all of that kind of information now gis specifically is kind of an interesting application because it's typically two-dimensional graphics right that's a roadmap sometimes contains some 3d information but often not but the scale and the amount of information is so huge that we often have to find clever ways to window it and not totally use up the bandwidth and the data of your cell phone plan but still get a reasonable display and that is an ongoing battle you know every year the technology changes and people's demand for data or their tolerance for how much data you can use on their phone shifts as this technology evolves and stepping back even further of course basically we're all surrounded by screens i mean just looking around me in my living room here i see a laptop a computer monitor uh you know a screen uh near my refrigerator and on my instant pot and my microwave and my washing machine my vacuum all of these things are displays and computers uh have to display interesting information on them and moreover while you're interacting with your laptop essentially the one piece of software that is constantly running on your personal machine is probably computer graphics software because you're looking at your screen it's updating and so on so your operating system contains a lot of at least two-dimensional computer graphics technology built in so essentially almost any content that's displayed on your screen is probably using opengl directx or some other computer graphics technology to generate the display and this includes not just 3d content but also 2d content so software like adobe illustrator microsoft powerpoint and so on are really graphics tools and built on computer graphics algorithms that we'll talk about in this course in fact probably the most dense amount of computer graphics information that we all look at every day are the fonts that are displayed on your screen i mean if you think about it the demands of displaying text on a computer monitor are extremely high right there's a ton of letters on a computer screen for most of your work day and your eye is spending so much time reading information on your screen the the graphics technology for displaying storing and manipulating fonts has to be extremely sophisticated and you know high fidelity in order to not cause eye strain issues or frustration so in any event hopefully i've convinced you that computer graphics is an important technology i don't think that's hard to do in fact i would argue that many of us probably interact with computer graphics technology more than ai or machine learning in any given day just to throw a brick over at the popular discipline right now but in any event uh we can all get along and and hopefully agree that learning the basics of computer graphics system can really help you understand how to engineer and be part of many different software engineering teams and be part of a type of technology that's constantly evolving improving and affecting our everyday lives in many different ways so hopefully now you have some motivation for why we should study the material that we do in this course so i thought i'd give you a bit of information about specifically what we'll cover in 6837 and that of course is one computer graphics class at mit and everybody has their own bias certainly mit's curriculum and your instructor have certain topics that they care about and that's probably reflected in the structure of this course compared to courses at other schools or in other programs so our focus in 6837 is on the fundamentals of computer graphics algorithms these are the ideas behind many of the applications that we just talked about this includes the basics of both slow but realistic rendering as well as real time rendering now we're going to spend most of our time talking about algorithms mathematical ideas and so on but we are also going to make use of practical libraries we'll do some basic opengl opengl is a computer graphics library that is used to display 2d and 3d content on your screen but it's not our focus we're not going to like go through the opengl api and give you every detail it's just a means for displaying content and helping you understand the basics of computer graphics systems now specifically at mit 6837 likely is your first exposure to programming in a language like c or c plus that's really challenging uh in the sense that c and c plus are already kind of difficult programming languages to learn and you're learning them in an environment that requires you to interface with computer graphics hardware which is a pretty low level thing but we're going to try and make it easy for you and the course staff and myself are here to help you learn all the ins and outs of this programming language it's not expected as a prerequisite to this course now it's also important to tell you what you won't learn so that we're all on the same page specifically we're not here to tell you about different ways to hack into opengl and directx or whatever your favorite graphics library is the reality is that this changes just about every 18 months you know opengl when i learned it which was not that long ago in undergrad has been completely deprecated and thrown out and replaced with a different version and maybe two five ten years from now there will be yet another set of graphics technologies that dominates the reality is the these low-level tools also don't matter a ton because the current trend is for graphics to become all software so these days we have assorted layers on top of these low-level tools for rendering uh that can help us by essentially taking care of some of the low-level graphics algorithms that we'll talk about in this course unless you really need to dig into it for a particular application or use case in this course we're also not going to spend a whole lot of time on different software packages this is not a course on producing computer graphics right you might be look for for that in art or graphic design but rather on the technology that's behind the graphics tools we're not going to spend a ton of time on graphics hardware we will talk about the architecture behind your graphics card as well as how some different display technologies work but it won't be a big focus for us and the main one is that we're not going to cover graphic design game design or artistic aspects again this is an algorithmic technological style course rather than a course that's teaching you how to use computer graphics software to draw interesting or visually pleasing content and trust me when i say that essentially you don't want me teaching you how to do computer art in fact this cat that i just drew on the screen is roughly the uh beginning and end of your instructor's artistic ability it looks kind of like a vacuum okay another question that we often get about 6837 is the amount of background we're going to do a ton of linear algebra in this course that's sort of unavoidable in fact in our very next lecture which incidentally is one of the hardest lectures in the course hopefully that won't be off-putting but we're going to start by doing tons of linear algebra and essentially we're not going to let off until the end of the course if your linear algebra skills are a little shaky that's okay our tas are going to lead a review session and essentially as long as you get an early start on the assignments and you spend lots of time in office hours and interacting with the course staff we're more than happy to get you caught up with any of the mathematics that you've forgotten from your courses uh from a while back in this course we're also going to cover some new mathematical concepts that i don't think you're likely to have seen before at least the typical student those include homogeneous coordinates solving ordinary differential equations both numerically and a tiny bit by hand and a tiny tiny tiny bit of fourier analysis when we talk about anti-aliasing and sampling but one thing that i want to tell you guys is that the math in computer graphics in fact this is one of the reasons i love computer graphics as a domain is extremely concrete right at the end of the day we're actually rendering frames and looking at them on our computer screen this is one of these fun areas where if your code has a bug you may actually see it on your screen manifesting itself as a weird-looking computer graphics system so if your linear algebra is a little shaky and you were frustrated by the level of abstraction in typical linear algebra courses rest assured that all the linear algebra we'll do in 6837 has really concrete low dimensional interpretation that may actually improve your intuition for that mathematical content now one of the fun things to note is that of course this whole course is on computer graphics but actually many of the mathematical and algorithmic tools that we'll cover are useful in other disciplines of course linear algebra is all over the place splines are used in data analysis and interpolation monte carlo integration which we'll use in rendering is also used in bayesian machine learning and statistics and probability there's so many different interactions between this course and other courses in the department we'll also talk about animation and how it connects to robotics for example and so this is one of these fun topics that essentially by making a computer graphics system whose job in life is to capture the real world will actually have many interactions with real world computer science applications like robotics simulation and so on our course is structured into several assignments we'll get back to these in a minute and these are going to comprise the main work that everybody does in this course we'll start with a quick warm-up and then have five different homeworks that essentially are trying to capture five key areas in the computer graphics domain incidentally these are all being revised this year so this list is a little bit subject to change but not the number of assignments uh or the timing this course concludes with a project which is extremely open-ended essentially what we're going to ask you to do is to identify a research paper or tutorial that covers some topic that's beyond the scope of one of our lectures and to implement it and present at the end of the semester this can be done in teams or individually and the instructions are already on canvas i believe we'll actually talk about it a bit in the live part of our course and as incentives to do a good job on this project not only will we give you a good grade um but we'll give the top project as determined by totally arbitrary means by your course staff a trip to the siggraph conference which is the annual graphics conference introducing the state of the art in this particular domain to get you guys started we're going to have two review system review sessions at the beginning of this course uh one will cover c plus programming and give you the basics of the syntax the features of the programming language and how to compile and run your code we'll also have a review session on linear algebra unfortunately because the technical content in this class escalates pretty quickly right at the beginning then it backs off by the way the second lecture i want to warn you is one of the tougher ones by the time we have our linear algebra review you will have already seen it in lecture but hopefully you'll be able to catch up and put it all together so now let's give you a quick overview of some of the topics that we'll cover in this course in a little bit more detail so the basic question that we're going to try and answer is how can we model and compose and eventually draw and display a scene like the one that you see on the screen here and of course that is comprised of many different aspects the first thing that we have to do is actually model the scene this includes capturing the geometry you know like the shape of the equipment here the characters uh maybe the lighting effects and uh so on the next thing we have to do is actually animate the content make it move and just essentially specify uh the different motions that we see in a scene both the primary ones that are probably animated by hand or by the ai of your graphics system or your video game as well as secondary motions which might be done using tools like physical simulation for example now all of this is done to set up a computer graphics scene but of course what we would all agree is computer graphics involves the actual step of displaying so we'll talk about rendering which is the process of actually drawing a scene on your computer screen or into a file and that's going to include many different aspects including the lighting shadows textures on different objects and so on so roughly our semester is organized exactly the same as this computer graphics pipeline we're going to move linearly through these different pieces so that by the time you get to the end of the semester you can go all the way from modeling to visual content and then a little bit about how it actually gets displayed so that more or less gives us an overview of our semester we'll talk about modeling static scenes first then we'll talk about how to animate them that's sort of part one then the next part of our course we'll talk about ray casting and ray tracing these are algorithms for extremely efficient rendering or sorry extremely high quality rendering but sometimes it's not terribly efficient essentially the trade-off there says that i would prefer a high quality image even if it takes a lot of computational time then we'll talk a bit about how to improve our graphic scenes both in slow graphics and in efficient graphics by talking about textures and sampling essentially different considerations of quality and computer graphics then finally we'll talk about the graphics pipeline really this should be the real time graphics pipeline which is used to actually draw content on your screen at a frame rate that's acceptable for interaction so the ray casting ray tracing algorithms are slow but realistic we'll see that they're basically built on the physics of how light bounces around in a scene the graphics pipeline is engineered to be fast because it has to be fast enough to interact with a user in real time in a fashion that doesn't create a perceptual blip right the problem here is anytime that your graphics system when it's meant to be interactive has a failure essentially the computation stops and you pull the viewer out of that virtual world and remind them that they're working with a slow computer and that's no good let's talk about some of these directions in more detail in the modeling part of our course we'll start by talking about how to model curves and surfaces curves are useful for tools like powerpoint displaying fonts on your computer screen and so on and of course surfaces are used to model different 3d digital characters objects to a manufacturer and so on we'll do this using uh techniques like splines uh which are essentially curves that are just built out of little cubic segments so in our first lecture we're just going to talk about cubic functions basically for an entire lecture then we're going to talk about how we can glue them together to form a longer smooth curve we'll also briefly cover some technology on subdivision surfaces which essentially takes a coarse estimation of a surface and smooths it out to a higher quality one so this was introduced in a pixar short film which i encourage you to watch it's actually super cute in addition to having neat computer graphics technology i think you can find it on youtube called jerry's game which was used to test out this technology long long ago we'll talk about transformations in this course this will be even more linear algebra so what you probably remember from your linear algebra course is that you can use matrices and vectors to perform operations like scaling but something your linear algebra professor probably snuck under the rug is that transformation translation rather is actually not a linear operation it's afine so we're going to extend linear algebra language using homogeneous coordinates to account for translation and then what we'll see at the end of that lecture is a happy coincidence that not only does homogeneous coordinate allow you to incorporate trans uh translation into the set of basic motions you can apply to a shape but it'll also allow us to do camera projection like taking a 3d universe and projecting it onto your screen which is a 2d or flat object in such a fashion that you get effects like foreshortening you know that for example if i take a photo of train tracks of course they run in parallel but on a photograph they look like they're converging together as they move farther and farther away from the camera so in other words we'll talk about how to use homogeneous coordinates to obtain perspective effects that's the modeling segment of our course then we'll talk about animation so there's one strategy of animation called keyframing where i specify some key poses of an object and then the different motion is filled in in between we'll see the many of the keyframing techniques that are used on the computer are essentially motivated by some of the ideas of hand-drawn animation that pre-date the computer technology but we'll also talk about some other techniques so for example in character animation we'll talk quite a bit about skinning the idea here is that the animator might animate the bones of an animated character but of course we're all covered in this squashy skin that's moving along with the bones and what we wouldn't want to do is to model the motion of every single point on your skin to get a realistic animation and so this technique of skinning is essentially attaching this bendy skin to a rigid skeleton that you can animate this is used just about everywhere computer graphics games movies and so on for animation we'll also talk about physical simulation we'll start with particle systems so these are just like little points that are moving around can simulate planets and sparks and fire so on and we'll also talk about some more general techniques for cloth simulation and other physical effects notice i've put physics in quotation marks here the goal in many computer graphics applications is not to have the most accurate prediction of the physical world it's to make a plausible one and to get it fast right in a video game i think if your cloth dropping on a table is or in this case on a teapot is slightly incorrect you're unlikely to notice it but you will notice that if your graphics system grinds to a halt so that's sort of the modeling animation part of our course then we'll talk about different techniques for rendering we'll talk about ray casting this is the simplest rendering algorithm where you iterate over every pixel you draw a ray from your eye and you figure out the first thing that it runs into and that's the thing that you render and then we'll extend raycasting to a technique called retracing which takes that basic operation of draw array and find the first thing it runs into but incorporates secondary rays for effects like shadows reflection refraction and so on ray tracing was one of the earliest computer graphics technology here we see one of the earliest retraced images by turner whittad and it can achieve some extremely realistic uh graphics uh results but the problem is that it's often quite slow so from here we'll start talking about different ways where we could get realistic looking graphics but much faster for example we'll talk about how to apply textures to 3d objects allowing us to use a pretty coarse model of this i guess what rhinoceros but then sort of stretch an image of its skin along the surface to simulate that high frequency detail speaking of frequency we'll talk a bit about sampling and anti-aliasing your computer screen is a big grid of pixels and every pixel can take on at most one color but if you think of that pixel as like a little window into the world of course there's a lot of things going on but underneath those right there could be millions of colors sitting underneath a single pixel and the graphics system has to decide on just one to display so we'll talk about the considerations for how to do that in a fashion that doesn't create too many visual artifacts as well as the physical limitations of your display technology and how those interact mathematically we'll talk a bit about how to extend ray tracing to be even more realistic at the cost of efficiency so here for example it's kind of a regular ray traced scene but we'll talk about how to incorporate global illumination which for example takes this red wall and simulates that slight red hue that you see on the ground or this light blue hue that you see near the blue wall here global illumination effects also include things that are very difficult to retrace for example notice this highlight underneath the sphere which is caused by the sphere sort of acting as a lens taking the light on the ceiling onto the ground so these sorts of effects we'll see don't really fit into the conventional ray tracing algorithm but with global illumination techniques we can get them but these techniques tend to be slow they are high computational cost meant to make nice looking images not necessarily efficiently then we'll start moving on to more real-time computer graphics technologies so one of the interesting things that we'll compare is how to get shadows on your graphics card versus shadows and ray tracing shadows and ray tracing is about three extra lines of code shadows on your graphics card turned out to be really tricky for example this is one of the old images that propose these techniques lux of the lamp here is casting a shadow on luxo jr and essentially the way that we're figuring out what's shadowed and what's not is to render the scene from the light bulbs perspective and then essentially whatever the light bulb sees is what uh is going to get lit in your scene at the end of the day this is a technique called shadow mapping which is really part of the real time graphics pipeline really shouldn't just say rendering rendering is just the procedure of drawing an image on the screen this is real time graphics we'll essentially see the at a very superficial level the difference between ray casting and real-time graphics is essentially just flipping the order of two for loops but then what'll happen is that the way we make it more efficient by eliminating computation from the inner loop is extremely different for these two algorithms essentially your computer graphics card is really good at doing the same computation a million times over like one per pixel and we're going to leverage that to come up with very efficient algorithms that look more like painting a scene one object at a time then simulating light as it bounces around in the 3d universe so we'll talk about sort of a typical computer graphics again this should be real time graphics pipeline which includes things like transforming the objects in your scene clipping them to the frustum of visible content rasterizing them which is like figuring out which pixels are actually being displayed and figuring out what objects are invisible so that we don't have to draw them finally in this course we'll conclude with a bit of a grab bag of other important but you know less linear graphics topics that are worth a call out so for example we'll talk about color both the physiology of your eye an extremely coarse level i am not a biologist as well as some of the typical systems that we use to capture and display color content and we'll talk about a little bit of hardware both displays you know like computer monitors virtual reality augmented reality as well as a tiny bit about how your graphics card is actually architect build to carry out graphics computations extremely efficiently so as a tiny last preview i thought i'd tell you a bit about our assignments in this course so we already mentioned uh here's our basic list we'll start with a warm-up and then we'll do assignments that more or less follow the outline i just presented to you so we'll do one on curves and surfaces one on hierarchical modeling one on physically based simulation one on ray tracing and one on real time graphics our curves and surfaces assignment is going to have you implement different bases for cubic curves and then extend those into 3d to model surfaces in the hierarchical modeling assignment you're going to make a tool that can deform a skeleton in response to changes of the joint angles right so that way for example when i bend my elbow here of course my hand moves along with it but if i think of the tip of my finger it's sort of the composition of many motions right there's the elbow there's the wrists the fingertips and so on so you're going to implement a hierarchical modeling system that composes all those transformations together moreover you'll implement techniques for skinning which allow you to get that interesting deformation of the outer surface in a way that doesn't just bend sharply with the bones that are underneath it in assignment three you'll do physics in particular you'll implement a simple math spring model for simulating a piece of cloth this one's a lot of fun in assignment four you'll implement ray tracing this is probably the easiest one for extra credit because there's like a million different ways to extend a ray tracer that aren't that much code and then assignment 5 you'll implement some algorithms in the real-time graphics world in particular you'll use vertex and fragment shaders which are things on your graphics card um to render a 3d scene that has shadows casts textures and so on this one's probably the trickiest assignment so i definitely recommend that when we get to this point you start it as early as possible so as a quick recap uh in this uh sort of half recorded lecture we've given you a quick overview both of where computer graphics appears what you can expect to cover in 6837 and a bit of an overview of the semester and the assignments now as we get started with this course i know there are many different moving parts and that it's sort of a unique time right now if you have questions either about what the course is going to cover the content or anything administrative do feel free to reach out to me or your tas for additional information so with that i'll see you all next time and we're going to dive right into the technical content of the computer graphics world
2020-12-31