Everything You Need to Find CLEAR SKIES
[Music] From the dawn of time, we astronomers have had a mortal enemy. This enemy is devious because it can show up when you least expect it. The forecast says it will be clear. and then out of nowhere So how do we astronomers avoid that fate? well we have to get better at interpreting weather forecasts so that we can get better at knowing both where and when we will have clear skies and in this video I'm going to equip you with lots of tools to do just that and I got some help in the making of this video from a true expert on astronomy weather Daniel Fiordalis who is the creator of Astrospheric this video is not sponsored by Astrospheric but it does have a sponsor which is Brilliant and I'm going to tell you more about brilliant about halfway through so the first thing that we're going to go through here and it will do it through a series of questions is breaking down some of the terminology and to make sure that once you start digging into these tools and data sets and forecasts that you really know how to interpret them to get the best results out of them and I'm going to start with this question the forecast model because it's something that I admittedly didn't fully understand the last time I made a video on this topic a weather forecast model is really complex to make it involves lots of investment in things like satellites and weather balloons and supercomputers so it's really only governments that are making these weather models and it is the model that creates predictions about the weather including cloud cover and then all the different websites and apps are taking information from a model or sometimes from an API that's sort of a middleman between the model and the and the forecast and the website or app chooses what to display how to display it and so forth so that's the differences you see but it's important to understand this because there's no point in taking time to consult multiple websites and apps if their predictions are all coming from the same weather model what we really want to do is compare predictions coming from multiple models and so I'm going to break down some of the different weather models out there and before I do that let's hear from Daniel from Astrospheric on what exactly goes into creating a model because I think you'll find this super interesting like I did on the left right there there's a few ways that gets assimilated into the system um they're certainly kind of like you know radar systems around the continent that are scanning uh for for cloud patterns intense weather hail Etc the the other way is weather balloons it kind of blows me away like Noah launches like a hundred weather balloons every 12 hours from dedicated spots around the US and the same goes on in Canada and other countries and these provide really accurate um vertical data that is also accessible if folks want to look at they they're generally shown in something called a skew tea graph uh they're really popular um for like Pilots right like the these are usually launched relatively near an airport and can provide some really good information about the stability of the air then just scattering the landscape are uh you know NOAA weather instruments um and so these kind of provide a set of data that's um is at ground level and then yeah there's satellites orbiting us or in geosynchronous orbits that are sending back data that the models can use there are other places like um airplanes uh commercial airplanes flying to the sky uh actually send in some weather data um and that can be incorporated into the model and so um let's see if I can if I can draw on this here we'll just pull up a blank sheet um all of these will basically go into what is called Data assimilation I don't know I have some chicken scratch handwriting but we'll we'll go with it for now and so like what that means is you end up with a ton of data points that are just really sparse relative to the land mass and the amount of uh um you know atmosphere above you and then it's up to these data assimilation processes which are just you know complex statistical mathematics that run over this um to then get you into this which is a nice grid that covers the entire area that you want to look at and um you know depending on how much uh compute resources you have how big your super computer is kind of determines then when the model runs how much you can break this down and how detailed you can get and so there are models that um divide the Earth every half a degree right of latitude or longitude and then there's models uh like the high resolution rapid refresh which I'll show a bit of today that can divide all the way down to like uh you know kilometer scale and so you can get a really high resolution and I think you know interestingly um let me um see if I can quickly turn this into kind of like an isometric View not only the grid directly over but this extends up uh many miles and in fact like it'll go up uh they basically go off of what the atmospheric pressure is and it would go down to like you know 50 bar which is really way up at the top of the atmosphere and so this then turns into a really a three-dimensional grid of Cubes that you can then pull weather data out of or predict weather data into as it goes forward there's some Advanced there's like you know super Advanced things going on where it's not always a cube that gets generated it might be you know like a dodecahedron or some other like shape that helps them increase resolution and and kind of optimize compute power um and so I from assimilation you go into really what the model is um and so that's going to take all that intro data it's going to initialize the model and run some Physics over it and produce the next hour or you know may go out three hours or something like that and then fundamentally it's just going to feed that back into itself re-initialize and keep doing that as far out as the as the model runs and and different models have different lengths uh the GFS goes out something like 240 hours like hour by hour more Others May only go out um like 36 hours like and again it's it's all about just optimizing to commute compute and accuracy of the forecast out of these models comes the data that uh astrospheric uses and a few other services use and um there are these binary files that are called uh grib 2 which stands for gridded binary file um and it's kind of like the common currency within uh the weather Community to pass data around and save data into um and and this is where it's like okay do I get my data directly from those or are there apis in between us kind of like what distance is in between because I think you know in your weather video you called out really accurately check multiple sources I actually think another big one is like try to get as close to the source as you possibly can um and so from here on out there's there's going to be a few things that happen we could go up into uh some company that runs a weather API and this would be something like dark sky um I need to stop calling that it's Apple now it's Apple weather that runs that and there's just tons of people that that feed off of those uh they'll do a bunch of interesting things and and be able to give you a point forecast really easily but one thing they're not great at is delivering map data and so some of these will go to services that will generate Maps um and others I will will go to a service like astrospheric um where we'll generate both Maps endpoint forecasts okay so looking at the the important things in a forecast for astronomy cloud cover is number one right because if it's cloudy none of the other things matter um and the way that cloud cover is typically presented is is it's the percentage of the sky that's going to be covered in clouds at a particular time in a particular location so if it's zero percent that means that that particular time and location when you look up it'll be completely clear if it says 25 they're predicting that a quarter of the total area of the sky will have clouds now they're not saying where in the sky those clouds are going to be but what I see some people misinterpret this as is 25 meaning there's a 25 chance of clouds that's not what they're saying they're saying there is a high likelihood that there will be some clouds somewhere in the sky and they think that it will cover about 25 of the total Sky area now that might not be so bad if it's just passing clouds right like sometimes you'll see a forecast go from 0 to 25 and back to zero and that just means there's a few passing clouds to deal with could still be a perfectly good night so there's you have to really look at this and um depending on the website or app you're using it might not even give you a percentage it might instead give you a phrase like clear partly cloudy mostly cloudy so here's the cheat sheet for what those terms really mean and there's a third column here with a lesser used unit for cloud cover measurement called the OCTA OCTA meaning eight and so zero octas means it's perfectly clear eight octas meaning completely overcast now I've long felt that cloud cover predictions are pretty good A day or two in advance but a week out they really can't be trusted and something pretty interesting that I've uncovered in the making of this video is the weather models actually agree with me there if you can actually find some weather models that will give you error bars showing the level of confidence in any given prediction so for instance here I'm on the European Center for medium range forecast website and if I click on the map I get this handy chart and you can see as we get further away from the present these error bars are getting bigger to the point where a week out the likely range of outcomes is huge here it's saying that a week out it's basically a coin flip whether it's clear or mostly cloudy now we've all probably been in a situation where you're literally outside the forecast on your phone says it is clear right now yet it is cloudy and so that sucks it will probably happen to you if you're in this hobby long enough but there is an extra step that I suggest taking basically for extra assurance that that doesn't happen and I'd say this step is probably more critical if you plan to spend time driving to a dark location and that is to look through um both real time but also forecasted map data when you zoom out and you sort of see how the clouds are predicted to be moving that is sometimes can give you a much better handle on just where the forecast may be going wrong and also if you can expect the clouds that you're currently under to stick around or to pass over and so I'll be showing various tools uh later in the how-to part for viewing map data so but to wrap up this part about cloud cover a few takeaways number one cloud cover is typically presented as a prediction of the percentage of the sky that will be cloudy number two it's very difficult to predict more than a couple days out at a time which is uh you know because clouds are unpredictable which way they're going to move and so for a particular location it's really hard to say and the hours leading up to a trip to a dark site Beyond just a point forecast I would also start looking at map data to get an idea of how the clouds are going to be moving for transparency and seeing which are two other important factors for astronomy and an astronomy forecast let's go back to Daniel for an explanation there aren't models that produce transparency and seeing um just directly out of the model it's a bunch of post-processing on variables that the model did produce that help us that help us understand and put them together in interesting ways and so transparency you actually nailed it in your weather video what we're looking at is from the base of the Earth all the way to the top of the atmosphere sorry the base of the atmosphere all the way to the top of the atmosphere we're going up that column and um looking at water vapor primarily um because exactly as you called out the the higher the concentration of water vapor the more things like light pollution from a nearby City will reflect off of that and the more um the more that that's in the air it also is occluding uh light coming through the atmosphere um the other big factor in our transparency forecast is smoke which is uh unfortunately kind of like a necessity now if you really want to have an accurate transparency forecast um and so we pull data in that comes from the the wrap model which predicts and forecasts how aerosols move through our atmosphere and so we're able to take that and and kind of integrate it in with transparency and produce something hopefully a little bit more accurate although we're like constantly tuning it and it's it's one of those things where each summer smoke season outcomes we're able to tune it a little bit further um and so that that ends up being transparency and the value you end up getting out of transparency like out of these uh functions are are basically something akin to an aerosol Optical depth which is a value that just indicates how much light can pass through uh the gas of our atmosphere and the higher the number the worse it is and the lower the number the more transparent it is I think one thing to call out with transparency uh because I see it on some weather websites is people use transparency and visibility interchangeably and it's it's not the same thing transparency is is a measurement of uh how clear it is directly above you and visibility is saying at the surface how far can I see at the surface level and the reason this is important is because it can be very clear at the surface um but a few miles above your head there could be you know serious clouds floating over you or something like that that are actually going to really destroy transparency um so that's just like yeah visibility is generally the horizontal measurements of how clear the air is and transparency is the vertical measurement of how clear the air is for seeing um I actually I was gonna do a little more drawing for this one I set up something for this um because seeing is a really complicated variable um and it's it's one that combines so many that I think the A good rule of thumb when you look at seeing is to say like let me find large spots of good seeing um and and you know if I just see a tiny really good spot of seeing like maybe ignore that because there's so many variables and if there's any error anywhere in the forecast it just kind of compounds all the way all the way up the stack and so um I think the easiest way to illustrate uh seeing and this is similar for transparency um is uh you know I'll use one of my favorite sites I like to go out to the Olympic Peninsula to take photos uh it's just ridiculously dark out there and seeing can be okay um but what happens is uh behind the scenes you know when when astrospheric or whatever you know astronomy software that's producing the seeing data is is coming from it's kind of looking at a set of cells above your location so in that case you know I'm out on the coast these are the cells above me um and so let me just draw a little bit there's there's a few factors um a few variables that we look at here and so let me zoom in just to help with how I draw the two primary things we're looking at are um the wind speed and direction and then the temperature um because what seeing is is a measurement of uh of turbulence in the air above your current location and the reason that's important is because turbulence is caused by you know mixing of low density and high density air or warm and cold air and as light travels through that it uh you know it refracts in funny ways and we end up going from what used to be a beautiful just point of light for a Star to something that grows out um you know much larger and so let me just do this so so like if I were to draw on this and say like you know I'm on the coast definitely at the surface I'm gonna have a wind that's kind of like coming right at me um it's usually coming in from the west and then as we go up this it may be a little bit similar it may start turning there may be a few Pockets where things get real wild uh maybe there's a total reversal and airflow in one of these cells and then as we go back up it kind of normalizes again and uh maybe as we get real high we're starting to really look at the jet stream right there's these arrows are getting much longer hopefully they're going in a similar Direction the ideal situation is they're all pointing the same way and you have like laminar flow above you um and and seeing through that type of atmosphere should be pretty good the other attribute so this is kind of like a you know wind is creating a two-dimensional look and we're looking at where there's wind shear the other uh the other element we have to look at is what is the temperature in each one of these you know at the surface it may be uh real high and then it will begin to drop off very quickly and I'll just you know use little dots to show that it's getting cold as we go up although there may be a little inversion it gets a little warmer um than the than the cell below it as we go up and so what we do at this point is we combine these together um and produce what is called a c squared n profile of this column and that's really just saying like Okay given these wind speeds and these wind directions and these temperature changes all these gradients let's um weight this and produce a number that will allow me to say like okay I had that point that is a star it's either going to turn into a little bit bigger of a star or it's going to turn into just you know chaos um and and that's that's kind of the final number that comes out of one of these scene forecasts so this is a bit complex I'm not going to go into every model out there in general though there are two types of models Global models and Regional models and Global models have to be more low resolution because they're covering the entire Earth but they're good for predicting large weather patterns extreme weather like the path of hurricanes and they can also be helpful for clouds and astronomy Regional models though can be even better because they're typically higher resolution so you can dial in like your specific location they usually combine more data sources like the hundreds of NOAA weather balloons that Daniel was talking about and they also update more frequently sometimes but the downside to these Regional models as the name suggests is that their domain it usually only covers um you know one country or a few countries usually the country trees are always probably the countries that are making these models that are paying into them so in the USA where I live we have the high resolution rapid refresh it's a weather model that's updated hourly down to a resolution of three kilometers but its domain its coverage area doesn't extend much past the USA in terms of global models the European Center for medium range forecasts often shortened to just the European model is probably the best one out there in terms of a global model and I'll be sharing more on how to use their website when we get to the how to's another Global model is noaa's GFS Global forecast system and it's another one that's considered pretty accurate now the reason astrospheric hasn't expanded Beyond North America is because Daniel draws from Regional models in addition to Global models and those Regional models that he's drawing from only cover North America but he feels some of these especially the Canadian model are the most accurate in the U.S and Canada for cloud cover you know the thing that made me go with the rdps model from Canada is that they really focused on cloud coverage and their prediction model is attempting to predict what you'd see if you just looked from one of the satellites uh like goes 16 right looking down on the planet and seeing those clouds and so what it was trying to do is predict that which I think is really good for astronomers because um we kind of want that detail and what clouds are coming up so in this video as you've seen I wanted to really dive into the models behind the forecasts but of course there's always more to learn about any topic and to truly understand something as complex as a weather model I think today's sponsor brilliant is where I'd go for a deep dive on everything you do need to know including statistics and probabilities brilliant.org is a Hands-On way to learn science and math in a really fun and interactive way and there are thousands of lessons with new ones added each month the lessons that I think could really enhance your understanding of whether modeling are the ones on probability and what I'm showing here is the course intro to probabilities that I've been really enjoying it has all these interactive activities that are really fun so to try everything brilliant has to offer completely free for a full 30 days visit brilliant.org nebula photos or click on the link in the description the first 200 will get 20 off Brilliance annual premium subscription after the free trial so hopefully you've made it this far because now that you know all the needed background on weather models and all that uh stuff you know cloud cover seeing transparency what all these things mean we're gonna actually dig into where to find the data and how to use these tools to more confidently predict clear and steady skies for your location okay so the first tool I want to show you here is the ecmwf European Center for medium range weather forecasts and of course I'll put all of the links in the description so on here there's actually a lot of cool information that I've been reading but I just want to show you uh the the most important chart here so to get to it go to forecasts and then scroll down to charts and then click into medium range forecast charts okay and then over here on the left hand side you can see there's different parameters you can select and so if we just select cloud it gives us seven different charts that we can look at and these simulated images are might be what you're looking for those are are really good for just sort of seeing the cloud fronts but I find them a little bit hard to see like the the thinner um low altitude clouds on these images and it's a little bit easier with this total cloud cover map so I would suggest this one it's the one I'm going to show here and because this is the European Center it starts with the European area the European region but this is a global forecast so you can change it to wherever you live or you can look at the whole the entire Globe um there is a legend down here it's a little bit confusing but basically what this is saying is that um low clouds are this beige color medium clouds are the magenta color and uh high clouds are the blue and so where they're all uh where you have all three types of clouds you get this sort of dark grayish black and where it's perfectly clear in the you know anywhere that's perfectly clear it's going to be white you're just seeing down to the base map so I can quickly see here that right now or pretty close to right now um it's clear throughout much of India throughout the Midwest area of the United States uh through a big part of Argentina many countries in Africa looks like all of France is clear so it's pretty interesting just to look at a Global Perspective here but of course you can click in here into the region where you live so let's just pick uh Northwest Europe and I will say for Europeans I'm guessing that uh this is more useful than for some other places because for instance I noticed like in Europe it breaks it down by many different regions while uh in North America it's just all of North America so uh this is probably the one I think to look at uh if you are in Europe or the UK okay so what is this telling us here we can see sort of the cloud patterns um and then we can advance it by three hours at a time to see how those uh clouds are going to move every three hours and so I can see 12 hours from now ish now I I should stop saying now because this is actually today at uh 12 universal time so you'd have to do a little bit of math to figure out uh you know what that means uh if you're actually in the UK it's very close to the universal time uh but let's just say for the ease of this tutorial that right now it is 12 noon universal time so in 12 hours at around midnight I can see that the south of England here is perfectly clear now what happens if I actually click on a location so let's say I live right there okay so let's look at how to actually read this in a little bit more depth here so we're going to be looking at this first chart right here total cloud cover in octas and remember from earlier OCTA is a term that means uh zero octaves would be perfectly clear sky so that's at the bottom of the chart eight octas is perfectly cloudy overcast and so we can see right now in that part of the UK it's perfectly clear if I go out one hash mark six hours there's just a tiny chance of a cloud or a few clouds if I go out 12 hours they're very sure they're going to be no clouds so that's great because uh six hours from now would be 6 PM so that's sort of the start of the night 12 hours from now would be midnight so this looks like it's going to be a clear night then 6 A.M
we're again showing some probability of some clouds coming back in now how do you actually read this probability um it's easier to see down here how the boxes work but basically the bigger box contains 50 percent of the outcomes from 25 to 75 percent in a confidence interval and then this smaller box extends down to 90 percent of the outcomes and then the line coming up from that is the maximum number of outcomes 100 so what this one is saying right here is that foreign in most outcomes and about the majority of outcomes it looks like it's going to be clear or maybe just uh one OCTA so just a few passing clouds like 10 clouds um and then in 90 of the outcomes it's clear or scattered clouds you know like 20 percent of the of the sky covered and then in all outcomes it's either clear or up to 5.5 octas so 5.5 octaves would be like half more than half the sky covered in clouds so that's not good but there's only about a 10 percent or less chance of that happening by 6 am okay so hopefully that made sense um you can see then as we get further out here they really just don't know uh you know all of these ones are saying it's more likely to be cloudy than clear but you can see the range of possible outcomes gets pretty crazy well in here uh and then you know the next few days they're saying that uh it's it's going to be they're pretty sure it's going to be cloudy it's only like a 10 chance uh that it won't be or something like that so I do find this useful I do think it is a little hard to read Because you if to actually find like what time a certain uh probability is talking about you basically have to just sort of count the hash marks each one representing six hours and for some reason they're not marked so uh but I wanted to show you how to read that because I think it is pretty cool that they even have this data available but the main way that I think uh this website is really useful is to be able to go into your area and just use this map data to see how the clouds are moving and at what speed okay next I want to show you astrospheric and I just have up the free edition here just to show you how useful even the free edition is if you live in North America I live here this isn't my actual location but I'm just going to pick up spot here in New Hampshire I'll click get new forecast and now you can see that red target Mark is what this forecast is for this is the location marked and then if I want to save that I can just go over here and type in H for New Hampshire click save and it's that easy I now have one of my favorites marked and whenever I log back into astrospheric that favorite will come up whether it's on the app or the website so how do we read this so one thing I really like about astrospheric is that it presents a lot of information in a compact way so once you get good at reading it it's really nice just to glance at it and understand what's going on so up here of course we have the map of the clouds and we can animate that to show what the clouds are going to be doing down here we have a point forecast with cloud cover transparency and seeing and then he's also put in uh the what the Sun and Moon are doing so when they're rising and setting along with ISS passes and we have dew point and temperature and if you click on any spot so let's go to Tuesday night here at 10 pm it gives you the stats down here cloud cover zero percent transparency average seeing average wind 1 miles per hour temperature 21 degrees Fahrenheit and if I look up at the map I can see how far away that cloud bank is and then if I Advance it I can see this particular model rdps predicts that it will be hitting me at right around 1am and by 4 AM I'll be in the thick of it now one thing to keep in mind with these predictions is I found that a lot of times they don't get quite when the clouds will reach you right but it this at least gives you an idea that probably some part of the night on Tuesday is going to be clear and then at some point maybe it'll be 1am like predicted or maybe it'll be 11 pm This Cloud bank is going to hit my location right so I really like the combination of uh the point forecast down here with the map up here which updates when I click on the point forecast okay uh what else to say about this so the the free one just comes with one model loaded uh rdps which is the Canadian model but uh if you update to professional which is 299 per month or if you belong to an astro society that meets the conditions to get the Astro Society Edition you get all the additional models uh like let's see here what is available today rdps Nam nbm and GFS so you get all of those and you can do some really cool stuff that I'll let Daniel explain here if I just look at live data right now this is for a spot I love to go view in Central Oregon um you can see uh if I click this Cloud later this is just the rdps model and let's look at uh you know right right about now um and so this is already ps's prediction um but I can quickly switch over to The Ensemble here and it kind of turns into a like a Christmas tree of lights here but what we're seeing is each model and it's showing what it thinks the the clouds are going to be at that time and so the legend which is maybe a little bit small off to the side is red is the GFS green is the North American mesoscale and then blue is the rdps model um so we can kind of blow this up look at this and it lets us like pretty quickly take a look at this and say uh yeah like things are green versus things are pretty out of whack they're like very much out of agreement let me back up a little here and so um you know this this large structure that's moving through kind of um coming off the West Coast likely we'll be moving East here we can see how each of the models feels that's going to move across you know you have the GFS kind of having its Leading Edge a little bit further out than um the the other two and then where they all agree it'll turn white it's just like yep we everyone agrees and then where they all agree there won't be clouds it's just uh clear all the way down to the down to the ground one this is really I think crucial um in various parts of the country and I think the west coast is a is a great place to kind of show it off I found over time the rdps model doesn't do an awesome job predicting uh the clouds that exist just over the Marine layer basically that exists over the Pacific um here and I I actually am curious if we just kind of flip over and take a look at the satellite right now Marine layer is kind of like these sets of clouds that form and it can get absolutely massive it can be a shelf that exists and kind of like covers an entire almost uh continent size area off to the off the west coast and I found over time you know the rdps Model cycle maybe let me sync these up it's uh 5 30 here right now um won't always catch these things so if I kind of like toggle quickly between these okay interesting right like these these kind of clouds here we're kind of seeing those pop up um however I found that the North American mesoscope does a great job of certainly yeah um picking out and and trying to find these Marine layers which end up impacting folks that are trying to uh uh view down in like Los Angeles areas where it's like you're low enough that this Marine layer can actually flow in a ways and that totally you know ruin a night of viewing and so that's pretty interesting right like each model again has kind of like um a sets of things that's really good ads that's a thing that it's it's still learning I guess I'll say okay this uh last one uh this last tool that we're going to show in this video is Noah's weather and climate toolkit it does feel a little bit U.S Centric when you first open it but it can open these grip 2 files from any um data source so it it does have maps for the whole world if you find uh grip files on you know non-us sites but and I'm mostly going to let Daniel explain how do you actually use this uh program but I just wanted to explain something really quickly here which is this is a Java program so you do need to install a Java runtime on your computer in order to use this and if you happen to be on Mac like I am it's a little bit confusing because if you don't already have the right run time installed you might do what I did and just Google Java Mac and this first result is not what you want um it will bring you to java.com with downloads
and it's the wrong thing what you actually want is on oracle's website um and to get Java 17 go down here go to Mac OS and either install this x64 or arm 64 DMG installer now if you are on an M1 or M2 Mac like the mac Apple silicon you're going to want the arm one if you're on an Intel Mac you'd probably want this x64 one um but anyways you just then download this DMG file installed the Java 17 runtime and then when you download noaa's weather and climate toolkit it should actually run giving uh if you just follow the instructions that are included in the zipped bundle there so hopefully that helps get you set up if you're on Mac I assume that uh it's a similar situation with Linux you can get the right Java runtime from oracle's website and then run it this way and again Windows uh maybe that's the same thing I think Windows is usually better about already having a Java installed all right so now I'm going to turn it back over to Daniel to show how to actually use this toolkit which I think is really something I've never seen and it seems very powerful uh Noah's weather and climate tool kit and this is a a great program I think for being able to get directly to the model data and it does a bunch of other stuff too it's a really kind of like versatile tools to slice and dice and look at data and generally you know before we introduce any new data onto astrospheric it's been months in uh in the weather and climate toolkit and looking at it and animating it and changing uh Legends ever so slightly and making sure that we're like really dialing it for for public consumption so this is definitely I think a great tool to use and it has some built-in functionality that makes uh things kind of easy to get at so like it can connect directly to NOAA big data which apparently is stored on Amazon these days um but you could if you wanted um just pull up a Nexrad radar of your local area and um just grab the latest data from it and look at it you know directly on the map and so um I don't know we could we could look at I wonder if there's any out on the East Coast I mean I'm sure there are unfortunately it's here let's take a look at Albany here so I can say yep I want to look at data from today let's list these files and still go out and it'll chew through everything everything in here is usually in like UTC time so you'll have to convert to whatever time zone you're in but generally at the very bottom is the newest data and then going back all the way um to like the initial data from that day and yeah a quick double click on that we'll download it um and usually Zoom you right into where you need to be looking and so as you can see um looks like some thunderstorms rolling across uh Massachusetts and uh Vermont main area right now so heads up uh the the other thing is um you can change things like the elevation of the radar you're looking at so this is like one thing it can do the other is it can pull apart grid files which is like the handiest thing because um unless you're writing special software to help uh automate a bunch of stuff it can be kind of a pain in the butt to view these things and so where I go for grid data is somewhere you know anyone can go it's the nomads um kind of like Central distribution center of many many models um and so this whole list each one of these represents either a different model that's running or some sub-domain of a of a larger model or maybe just a model running a different uh resolutions overall and it's broken out into things like there are Global models and in here you'll see you know the GFS is is represented here several times over um and a lot of the differences are the resolution of its um this s flux is usually that's the highest resolution that you can get from GFS right now um as well as um uh parts of the model that they run separately like uh the chemistry section of the GFS which is more predictive of um aerosols and things like that in the air which I think is kind of interesting folks not in the US or in Canada you could technically grab data from this and look at what the GFS thinks kind of the aerosol Optical depth is above your location based on uh forest fire smoke Etc then we get into the regional models and and um this is just a fantastic list of everything that you could possibly want to look at we'll focus on a few here and most of the time when you go to a weather website or or get weather data in the US at least you're probably looking at this which is the high resolution rapid refresh um you may be getting data from uh the North American mesoscale model um or um let's see the the rapid refresh is also one that's uh sometimes used and so these are kind of like three big models that you'll end up seeing over and over again um there's a few new ones that are popping up here and there and I'll show some data from them um I think a really interesting one and one that we're excited to start working with just at astrospheric is um the national blend model so this is the NDM and this is a a model that runs hourly but incorporates data from like over 40 other models and builds up an ensemble based on that and so I think that's actually a really interesting thing let's start with um the national blind model because I think this is one that probably not many people see um really ever uh that that's out there kind of like looking at either civilian weather or certainly astronomy weather so the way to do this is I think for starters at least you can use this um nice script that knows written that allows you to look at the grid file and kind of pull apart and pull out exactly what you want um over time especially if you're writing scripts or code against these to grab the models automatically you may switch over and just go directly against they just have big directory listings where you can get this for today we're gonna uh use the grip filter and again so this is the national blend model it's being computed hourly and it goes out 36 hours hour by hour and then they kind of decrease the temporal um resolution of it so okay we clicked there and we say all right we want um this is like common formatting you have the first four digits of the year follow that a month followed by the day um let's go with 12 because there'll be plenty of data in there nope sorry I take that back let's go to 13. today's the 13th um and then we can look at this and say like okay subdirectories now we're getting into the hour right so this this is run a lot uh I'm going to choose the 18th hour um and we're basically just traveling through a directory structure at this point so this is where um the the meat of it is and so what this program has done or this website is it's gone through that grid and it's pulled out all the available variables inside of it and you can select just one of them and download it or I can click Start download so if I don't know what these variables mean like if I don't know what uh tcdc is I can just say start this download uh this will be for the model that started at 18z so what is that that's like it's 11 A.M Pacific time um and it's the first hour of that model and if we click this we can see like 18z then produced a second hour or a third and so on and so forth out into the future um and it goes aways and you can see at 36 it stops being hourly by hourly and skips to every three hours this is again this helps just save on compute and making sure they can go out of ways here so we'll just take the first hour here and I'll start the download and we're basically going to download the whole thing um annoyingly it didn't say how big it was but I think if I open up my folder here so this is like 105 megabytes okay um and so from here um I'm just gonna put this on my desktop and then take the handy dandy weather and climate toolkits and say I've got a file on the local disk it's actually just on my desktop let's take a look and list those files and so there it is there's the you know blend.t18z etc etc here down to the group 2 file and I can open that and it will just load up and so what I end up with is this grid file contains this list of variables and we'll pick out a few interesting ones here in a moment you can see the domain that it's covering it covers basically the uh the continental US minus Alaska here and pretty big chunks of Canada as well at least um covers a lot of the population of Canada at least um and it really lets us quickly zoom in and say like oh you know what this is actually a pretty high resolution model right I'm actually seeing like valleys in the mountains that this is predicting we're looking at apparent temperature height above ground right now um and so we can just go up to the filter and say like okay well [Music] um maybe I want to know just what the temperature is right like tell me the temperature and we're going to take a look at the height here and so it's saying the temperature at two meters above the ground and just to kind of put things in perspective this is at 12 o'clock so this is noon uh Pacific time so the sun should have been pretty high in the sky for us and if I take my mouse now and hover over any part of this you'll see down in this cell that that totally just disappeared you'll see latitude and longitude and then the value um and then we can look at the legend and know that that value is being reported and Kelvin so then at that point you can convert 296.4 Kelvin to your your favorite unit of measurement for temperature
um and just like that uh we we can look at data directly out of this model there's been no kind of like processing in between by any websites or apps or anything like that um and and just see what the model is producing and I think I think this is really cool I'm very much a data nerd so I get the this won't be cool to everyone but um I hope that someone finds it's exciting at least as we go into the dark cold months there's just gigabytes of data out there to be played with okay so we we originally went into this model to look at it's it's clouds um and so uh we are going to look at a variable uh called total cloud cover at the surface and so this would be if I'm at the surface looking up what percentage of like cloud is going to be directly above me it doesn't really care about where that cloud is it could be fog it could be it could be really high up in the upper atmosphere and so this is going to show me that um and again we'll take a look at the legend dark blue is clear um there are a lot of options for what layers you use and so like these colors actually kind of like hurt my eyes um so you could easily go into grayscale here and say great now you know if it's black it's actually um totally clear and then white is predicted to be very thick clouds so this is producing a very high resolution prediction of what it thinks the clouds are going to be at um or we're at at 12 noon [Music] um and so to save us the pain of uh me downloading tons of files I I preemptively downloaded uh gigabytes of data the other nights and produced some animations because one thing that we can do um is we can actually tell this program to look at a bunch of grid files and kind of run through them in a Time series and produce an animation so let me uh let me tee that up so I have weather data nbm cloud view data if I open this and list um I have a just a ton of NDM files that I've stored I think these were from yesterday so unfortunately I'm not predicting anyone's weather tonight although if this video comes out in the future it doesn't really matter anyways um and again I can click through these I'm actually gonna disable this reset zoom and you can see if I if I slowly click through them you'll see okay the clouds kind of move cool this is this is great this is you know okay how can I make an animation of this so in this software if I click the first one and then uh shift click the rest I can select them all and then this animation option becomes available to me um and I can say okay do I want to load an animation directly here or do something else I can produce kind of like a video right away um I'll do here this may take a while um and it's just going to run through each of those files and generate an animation I will say as this animates there are some interesting things that you can see just based on what's going on here and the the first one that catches my eye the primary thing that catches my eye these clouds look really high resolution like they're very detailed I can't zoom in while it animates but if you look up just above this kind there's like a line of things here things get a little uh a little fuzzier maybe lower resolution and this is um this is a property of this forecast model the national blend of models is uh literally taking different models um and statistically blending them together to make a ton of sense and not all models are of the same resolution and so something that's covering um say the US here is probably pulling from the high resolution rapid refresh where something up here um maybe pulling from a lower resolution model so that's kind of why you're seeing those lines and so a lot of these models come with little little quirks like that that you can either see right away or you learn about as you begin using it and so um you know relatively quickly I can build up an animation and see what the NVM is projecting and so if I were using this data for the night if I was you know really traveling somewhere and I this is very important to me and um you know let's say uh this is the area that I was gonna go somewhere in Virginia here um I can then you know take a quick look at this model and uh exactly how things uh how these clouds may be moving and again you're probably not going to find this model on any other website at this time or really any other service um so this I think is a kind of a cool thing to be able to do but the time of this is off I I don't think this is at night but regardless conceptually it's the same you just download the appropriate ribs that that um that cover the time period that you'd want to be able to look at and is this uh is what we're seeing a visualization or is this using actual photography total visualization okay can I I love that question because it's like this model is getting high res enough and like obviously if I zoom in we can we can start to see um the pixels and I'll do that actually over here but it's getting high enough resolution to where this you know you could pretty easily fool someone into thinking this came right from a satellite you know just from one of the ghost satellites um if we zoom in we'll start to we'll start to see the pixelation and I guess technically you can do this with satellite data as well but the interesting thing maybe to look at um again because this is a grid if you look uh right here in this cell when I hover over it's going to tell you where in the grid I am I am I'm 162 grid um cells from from the left edge of this and I'm 686 uh cells up from from the bottom of this and so this is the native resolution of this model um which again I think is really cool because you know even on astrospheric it's computationally expensive to produce imagery at Native resolutions and so sometimes we have to we we de-resident a bit you know to make the to make the system go a little bit faster and kind of decrease the amount of data we have to send to everyone's phones but uh with this app you can absolutely uh tear your computer apart and zoom in and see kind of like actual native resolutions of of the weather models so you're now seeing the names of everyone who supports this YouTube channel over on patreon.com nebula photos it's an excellent community of dedicated amateur astrophotographers just people who want to learn and are very willing to share their own expertise we have over 800 members now there's an active Discord that you can get involved in and I can't thank my patreon members enough because I'm now doing this full-time thanks to all of you and it is what has allowed me to make these videos and to really pursue this as my own business so thank you so much to all my current patreon members and if you enjoy this channel I think you will get a lot of benefit out of joining my patreon community it starts at just one dollar a month and for that you get a bunch of perks including direct messaging support with me on Zoom chat with the whole Community a monthly Imaging challenge organized on Discord where we pick different targets every month and a whole lot more so if interested head over to patreon.com nebula photos till next time this has been Nico Carver clear skies [Music]
2023-02-17 22:32
