AI is now in the world’s top 200 programmers. Who cares?
you might have heard that an open AI model just became one of the top 200 programmers in the world and it's true at least according to one Benchmark and while it's always worth questioning a model's performance really anything's performance on just one test and situating it in a larger context especially when it seems like that test being passed cost a million dollars in compute power alone but overall open eyes claims about 03 do pass my smell test I've actually been hearing really positive things from programmers about the 01 model which was 03s predecessor the previous generation just on Computing task and programming tasks uh already in anyway I I find the trend that 03 represents way more interesting than any one Benchmark could possibly be and so today I want to walk you through what that trend is why I think it's profound both for the technology and for the world and I want to tell you a story from a previous era that I think holds really valuable lessons for us today broadly speaking there have only been three leaders in the history of the personal computer industry the first was Apple and famously the Apple 2 was the computer that Steve Jobs and Steve wnc designed to be built from their garage for nerds and not to sell them short radical nerds but a small group of nerds that they hung out with at The Homebrew Computer Club when the Apple 2 broke out to the tune of hundreds of thousands of customers the next leader IBM took notice IBM built their Apple 2 competitor using an unusual process for the company at the time they outsourced key components to other companies in an effort to hit the market sooner faster before that startup Apple could make things any harder than they already had notoriously the two components that IBM outsourced were first the microprocessor to a young company called Intel and second the operating system to a tiny little startup called Microsoft the IBM PC quickly forced Apple's hand in the PC space and as is true in almost all Tech History Apple was forced to innovate or face death and their answer to the IBM PC was a cute little computer that they called the Macintosh but it wasn't enough the IBM PC was too successful too quickly and despite the technological superiority of the Mac the PC completely rested control of the market the most important part of that success was developers devs were the reason that apple is credited for igniting the personal computer revolution in the first place they' used the Apple 2 to vent the spreadsheet but there was also they were also going to be the reason that Apple was going to fade into irrelevance there was a new sheriff in town and their name was big blue throughout the early 80s the PC industry and its fate seemed inevitable the IBM PC is what businesses were going to buy and thus the IBM PC was going to be the computer that developers were going to make apps for and thus the IBM PC was going to be what businesses were going to buy a virtuous and ridiculous L profitable cycle but there was one problem these businesses weren't buying the ibmc because they absolutely love the IBM PC they were buying it because it could run the apps that made their businesses more productive as it turned out developers weren't writing apps for the IBM PC because they absolutely love the IBM PC either in fact most developers weren't writing apps for the IBM PC at all they were writing apps for MS DOS the operating system that Microsoft was supplying for the computer smart entrepreneurs in the valley noticed this and they saw an opportunity startups like compact they started to talk to Microsoft and started licensing msos for their own IBM PC clones that could run all the same software that the IBM PC could at a much lower price and suddenly there were dozens of options on the market that could run IBM PC software at a fraction of the price that you pay to buy an original that was the moment that big blue fell from the top of the food chain and in tandem Microsoft Rose to the top of it but here's what you should remember about that story in the beginning of the personal computer Revolution the most important component was the operating system because it provided an application model for developers to make apps and people don't buy computers they buy apps but there's also a structure here that we can use to understand open ai's 03 model and what it means for the future of the a AI industry and even more profound than that the structure points to some key strengths and weaknesses of open AI itself as a startup competing in the Land of the Giants in the time of the IBM PC a computer was little more than a processor an operating system and a user interface in fact you'll remember that IBM outsourced two of those critical components in an effort to compete with apple as it turned out the processor was crucial but over time it was effectively a commodity as long as one company was willing to do the heavy lifting of swapping from One processor to another they could go from Intel to Motorola to nowadays a inhouse design like apple does with their silicon and while microprocessors aren't identical they're mostly interchangeable as a component of the larger computer and tech companies actually end up incentivized to Interchange between them based off of things like cost speed and efficiency on the other hand operating systems are much less interchangeable especially when a Computing Paradigm has been around for a while developers make apps for one particular operating system a particular platform and often it's difficult for developers to Port those applications to other Platforms in a time and cost effective way that means that over time operating systems tend to gain and keep momentum as developers make apps for the platform which causes customers to buy the device which causes them to access those apps which causes developers to make apps for that platform virtuous cycle software is a scale game and developers are almost always drawn to the biggest game in town at the expense of of any other platform success in short processors are mostly Commodities and operating systems are mostly not now before we go on I need to explain the basics of how a large language model powered chatbot like chat GPT is built and of course to do that I need to explain what a large language model is and so don't worry I'm going to keep this as short as I can but uh let's take a look at what an llm actually is so an llm is the technology that enabled the latest wave of AI products it basically takes massive amounts of Text data and predicts and generates language you can think of it as like a machine for example I'm going to represent this machine by a little box here that takes in a bunch of text data so here's our machine this is our little input take some text that maybe I've written and given that fragment of text predicts the next word so out here it just gives you one word now once you have this basic building block you can do all sorts of really fun things so let's say you have this llm machine and you give it this piece of text let's be more specific here uh you might recognize this one the quick brown fox and then the llm machine takes a look at that list of words and it looks over what it thinks are most likely to come next and so in this case it might guess over now I want you to imagine taking this whole machine and hooking it back up to itself so we take this word over and we put it at the end of the list and then we wait and then we feed it back into the llm and so what might it guess next well the quick brown fox jumps over the possibly and then you rinse and repeat you bring thatth over and until you go along then you've got the lazy dog and then maybe a period and suddenly we have this machine this llm that has become the most powerful autocomplete that humans have ever invented W be right big deal but it's kind of impressive and you might be able to see how this could be useful and so there's one more trick that you can use to take this sort of fancy autocomplete and turn it into a chat bot using just this little machine so instead of passing in a fragment that's sort of random like the quick brown fox jumps over you might pass it some instructions or a description of a scene so something like what follows is a chat between a helpful assistant and a user and then what you might do is you might prepend or append to this but sort of prepend the user's input with this sort of user tag we'll just say right kind of like you might see in an instant message and then of course what the person does is they see a piece of user interface that looks a lot like a little text box up here with a little arrow that says send and they type in whatever they want and so the user types in whatever they want and then you prepend or upend depending on your perspective something that says bot or assistant right so this is the user's message that they're about to send and when they hit send you send this whole chunk of text including what the user typed and including these little tags and then what it will do is generate the next word and then what you do you hook it back up to itself and you send it back around again completing the loop and you keep going until it's done generating suddenly when you hook it up like this you've got a chat bot and of course this is a simplification but at a high level it actually works chatbots are really just thin wrappers around an llm this thing here is the chatbot this thing here is the chatbot and the llm is the chatbot this is really the only thing that the user has input at this point right and ultimately outside of the llm it's really just this that is the thing that turns a llm into a chat bot it's a really clever user interface and so we end up with something like this because chatbots again are just really thin wrappers around llms chatbot if we take a look at our picture is really just the same thing as an llm plus a system prompt plus a user interface for the user to input their text and as it turns out early oper systems looked a lot like this they were actually also just thin wrappers but back then this wasn't an llm this would have been the microprocessor right when you look at it this way things start to look a little funny in fact the way I've set it up sort of implies something profound a new llm is actually a lot like a new microprocessor but it's only part of this whole system and so if you're like me and believe that history doesn't tend to repeat but does tend to rhyme you realize something else which is that llms are part of the system that like microprocessors will probably mostly be commodified so we've talked about how llms work when they're being used but we haven't really talked a lot about how they're built and so maybe you've heard something like we've used all the text on the internet to train these llms and for our purposes we're going to kind of treat that at face value and I won't get into the details but let's imagine that we have this other machine over here that takes a bunch of text and spits out an llm in return what open aai did is give that machine pretty much every piece of text that it could which is pretty impressive right but it's also a problem for open AI you see if you just took all the text you can find and gave it to the machine that means that any other company with enough money and time can build the same exact llm as you because every other company has access access to a similar set of Text data and the same set of training tools and for a while that was okay you just stay one step ahead of your competitors to find more data to train the next model and you'll see improvements so you're golden there's always more data and so so the model will always get better and when people say that we used all the text on the internet to train these things and when we think of it as actually true what that means is that we've sort of run out of data and this is such a problem that we've started calling it the data wall but you might have actually noticed something else which is that these problems only occur when you're making the llm not necessarily when you're using it and so as we've started hitting the data wall AI researchers started to take a look at the other side of the equation and they started looking at it seriously if the llm has nothing else to study in order to get smarter why doesn't it spend more time thinking before it gives you an answer right the whole point of open ai's 01 model was to try that exact strategy to give it time to think before it answered and just to see what happened and what happens is that o1 works by breaking user requests down step by step working through each step behind the scenes and then only sending back the response to the user like after it's done and for the first time we see this system in the chat bot where the llm isn't so exposed we don't see every single word that the model generates it only shares the result at the end of its thought process which is a lot like humans sort of Might approach uh a problem like this and as it turns out it works really well and so far it also seems like it gets better the more that the system thinks that's why it cost a million dollars for 03 to pass those tests and as we move into the future it seems that that system is going to be the next Frontier of chatbot Improvement which ends up giving us a new equation but because the system prompt and the thinking system are kind of both abstracted away from the user like we talked about I actually think it's really useful for us to think of them together as the Chad Bots operating system a lot like an operating system would happen on a PC or even on a phone and voila there we have it I bring us through this exercise because it really highlights the similarities between the early personal Compu computer competition and the current AI competition in particular how it highlights the place of an llm in the system as a whole as the natural language processor and not much more so what does that mean for 03 then in that case well I think what we can kind of do is substitute in chat gpt's components the modern components for 03 and C you end end up with 01 plus the operating system plus the chat interface is what chat GPT is right and it turns out that 01 and 03 don't have access to a lot of the operating systems tools it has the Chain of Thought and the system prompt and that's pretty much it so after the 12 days of open AI the only thing that changes is 03 and that makes it very clear now the natural language processor gets a spec bump the same way that your computer's microprocessor might also get a back bump in that way the improvements that you see are iterative they're not necessarily Paradigm shifting they might be in certain ways the same way that a super fast speed bump might be Paradigm shifting they might enable you to do a little bit more but they don't fundamentally change how the system is architected and what you can do with it if that makes sense as a comparison I want you to think of upgrading from an Intel Mac to an M4 Mac you might expect You' be able to do more with your spec bump but really you'd probably expect mostly to do things faster or with more battery life or more smoothly those improvements are evolutionary not revolutionary and that is how I think that we should think of model upgrades going into the future at this point in the AI race as spec bumps and not a ton more by the way this is bad news for opena AI their core competency which is training llms likely won't be a differentiator for their products in the long term there are some nuances here but because everyone has access to the same fundamental data and training techniques the Competitive Edge provided by open AI llms is really eroding rapidly in my opinion just as processors became interchangeable components of the PC market llms are on a trajectory to become more standardized commoditized components of AI systems as a whole there are other players out there with sufficient resources that can replicate or even outpace open AI in llm development now we haven't seen it yet but it's possible particularly as The Innovation cycle slows down on the training side due to constraints like the data wall by the way the realization that llms will become Commodities has become an underlying assumption for some of the other Giants that open AI is competing with meta has open sourced llama because they will benefit from the commoditization of large language models their core competency is an application of these models to social media and thus llms are a cost to their business that they want minimized in order to maximize profit Amazon has also pounced on this perspective actually they think that the real benefit to their business will simply be the need for more computing resar sources and the choice between models will just increase the amount of compute that developers need which is Amazon's AWS in a nutshell it conveniently uh solves that problem now if meta is looking to increase their margins by reducing the cost of llm related compute Amazon is looking to increase demand for their Computing resources if that makes sense open AI is in the exact opposite boat though if llms become commoditized and especially if they lose their lead in creating better llms they lose a crucial form of differentiation in the marketplace all else equal there if there are other models out there that are good enough fast enough cheaper more accessible why wouldn't someone else choose someone other than open AI That's a problem for them Amazon's interest in in increasing comp Ute demand is also a hint at something open AI needs to be really scared about in my opinion they can't compete with the scale of compute at Google or Amazon or Microsoft and at the moment they have to continue renting time on other company machines and that's going to continue to happen in the into the foreseeable future at least in my eyes I haven't heard anything other than that if Google actually gets it act together by the way there is a very real chance that the huge amount of compute that they own gives them the edge to being first to creating a Next Generation Frontier Model potentially even faster than open AI now they haven't done it but it's possible but the news gets even worse because the new think before you respond sort of Paradigm which is by the way known as Chain of Thought in the industry also scales with compute which could give any of those other companies with their own massive compute Stacks The Edge when it comes to the use of llm so not just training but use so open AI is in short at the mercy of these Giants by sheer nature of their size given that commoditization might actually end up being a net neutral for open AI their sort of core Advantage isn't in Ma isn't in managing those large amounts of compute power so it might be best for them to leave that to somebody else frankly but if they can't meaningfully differentiate themselves using their Frontier models like 03 what should they do I think it's helpful to look back again to the very beginning of the chatbot era as a clue towards what the future of open AI might look like in the beginning you'll remember that a chatbot looked like this an llm a system prompt and a UI the first time I remember learning about chat gbt the discussion was all around that fancy new natural language processor the llm and the growth that it was seeing based off of the amount of data that it was it was Su it was sucking in to learn the amount it was learning and that was for a good reason it was a Quantum Leap in that technology I remember studying some of that stuff in college and thinking oh this might not ever happen in my lifetime but I would argue that the real Innovation was in user experience and computer interaction not necessarily in the llm itself of course tricking the llm to act like a chatbot was such a clever turn in the use of the technology that it was the thing that made chat gbt go viral and I think that that means that the thing we now know is mostly a commodity even when it's one of the top 200 programmers in the world was not really the Innovation that sparked the chatbot Revolution it was a human interface innovation in the llms operating system and the user interface llms like the microprocessor are actually enabling Technologies for Innovative human interfaces in fact when you look at chat Bots the only really novel parts of the system are the prompt in the UI and looking at things now that means that the real differentiators are going to be exactly that the operating system and the UI by the way I think that you can take that idea to a logical conclusion you can take it really far and especially when you compare it to the early personal computer days for example um I want you to remember the thing that dethroned apple and IBM which was MS DOS with the IBM PC acting like a trojan horse Microsoft made their operating system the most important platform in the world why because operating systems create the world that apps run in and people don't buy computers they buy apps in the world of chatbots I think that that is going to hold first I think that the personal assistant Paradigm is here to stay and I also think that the most valuable Vector of competition in the personal assistant Paradigm is going to be the application space because that's what we saw in PCS that's what we saw in Mobile we see it over and over and over again take chat GPT actually as the best example of that its operating system and its user interface are already very robust in fact off the top of my head I mean I can think of UI Innovations like voice mode and like video mode there is web search and uh they have that canvas thing they've got image generation code execution memory plugins uh they've got like user customization oh I mean the other one is actually a native Mac app they have a great native Mac app which is not a guarantee in fact there's a lot of people who have talked about the native Mac app being the thing that draws them to chat GPT and it is for me too and ultimately those are all applications running on top of chat gpt's operating system they are tools that either do something for a user or help the user do more [Music] I think that that is unique in the AI space By the way almost unique other platforms have some level of attention to detail when it comes to apps but even anthropic and Google and Microsoft and meta they don't have anything near this sort of extensive catalog that openai does in chat gbt meta might be the closest but you'll note there's one exception there which is Apple Apple intelligence is completely Arch architect it's architected around the premise that a llms are Commodities and that b apps are the most important part of the equation which I think reveals something unique in itself those two companies with the most focus on product and user experience have partnered together to form an axis of power in the AI race which sort of makes me wonder how long it will last before they become bitter competitors in reflecting on the ascent of 03 and its place within the broader AI landscape it's clear that we stand on a brink of a new era where the True Value lies not merely in the capabilities of individual models but in the ecosystems that they inhabit and the applications that they enable a lot like the personal computer Revolution being ultimately shaped by the operating systems and the vibrant developer communities that thrived around them the future of artificial intelligence will probably be defined by these powerful language models but also integrated into diverse user Centric platforms even even when we think of them as being able to code like the top 200 programmers in the world that Paradigm Shift emphasizes that while foundational Technologies like llms provide the necessary horsepower it is innovative interfaces that drive widespread adoption and meaningful impact ultimately and I think it'll look a lot like it rhymes with the story of the personal computer at the end of the 20th century
2024-12-27 17:53
