New Breakthrough: Light Speed Computers Take Over!
over the past few years we witnessed an incredible AI Revolution which has been driven by AI chips in fact the demand for computing power has never been higher meanwhile the scaling of classical computer chips has slowed so what's next while Graphine chips Probabilistic Computers and Quantum Computers are still in the making light based computers are already arrived in this episode I will break down a new light-based computer chip which is on its way to data center right now and I can't be more excited about this let me shed some light on it Photonic Computers have been in the making for decades it all started 60 years ago with the development of optical fiber for communication and over time we got excellent at sending information with light now if it works so well why not to use light for computing in fact researchers have been long working on building light-based computers by now you've likely heard this idea that light-based computers are faster than digital computers because light is traveling way faster than electrons well it's true and not true at the same time let's take any conventional chip NVIDIA GPU for example during computation there is an electron that travels through a copper wire and this wire acts as a conductor and this is how it always works in fact the problem here is not the speed of electron but the medium itself the wire one light travels at 300,000 km/ second in this case we are talking about mm/ second and again here it's not a problem because wire is a conductor so it's full of electrons so here we can reach speeds way faster than mm/ second now you see we can't simply say that photons are faster than electrons it's way more complicated than this in reality the real reason why digital computers are slower than light-based computers because in digital computers we need to switch from zero to one from one and zero and this switching requires us to charge and discharge a capacitor and this takes time and this is where the real slowdown is coming from I explained this concept much more in details in my previous episode on Reversible Computing a great episode make sure to subscribe to the channel right now and watch it right after this video so by now we understood that the real slowdown is coming from this switching from charging and discharging a capacitor which is slow so that's where the light-based chips save the day because nothing like this is happening in the photonic world in photonics we compute data without stopping it basically we are computing as a data is flying by and this computation on the fly happening in the range of femtoseconds which is one quadrillion of a second so it's very fast the main feature of light is not light as itself but the main feature of light is that you can realize an Analog Computer and this is the difference it's not so much the light part when it comes to the math it's more the analog nature of light that you can natively exploit that's also why we call it Native Computing and the main advantage here is that you can carry out complicated mathematical functions without digitalization and that's very interesting in fact if we want to perform a simple summation on a digital chip to add up two numbers we need roughly 200 transistors those tiny devices all the digital computer chips are built off so then when we want to do a square root of this number we need another 7,000 transistors and then when we want to do a Fourier transform on this we you need roughly 1 million transistors so you see the more complex function you want to implement on a digital chip the more devices the more transistors the more chip area it will take what's so interesting when we want to implement a Fourier transform with light we can do it on a single optical device so you get much higher computational density and you might be wondering how is this even possible you know people who are wearing glasses if you are wearing glasses you are wearing every day a Fourier transformator on your nose and it performs this function using no energy at all once you understand this you can use the same principle to implement such complex operations on a light-based chip using special photonic elements just think about it we can replace 1 million devices with just one optical device device and it's passive so it means light just passing through it allowing you to do complex math without spending any energy at all and the same applies for multiply operation where on a digital chip we need roughly 1,500 transistors on a photonic chip we can do it with just one device so we get much higher computational density that's the reason why the interest in light-based chips is growing at light speed in practice it took many decades since this concept of computing with light emerged till the time when we figured out how to actually use it for computing purposes one of the main challenges is that light is really hard to control it tends to spread out and scatter and it has taken industry really long time but Q.ANT has finally built a fully functional commercial light-based computer their new computer chip is called NPU (Native Processing Unit) and it's powered by light rather than electricity we are already shipping first service to high performance computer centers and we've decided on that the first processor generations are coming on the standard interface of of the CMOS world mainly PCI Express and what we actually deliver to the customer are fully equipped servers which are compatible with x86 structures so in the end you get a server module you plug in the ethernet cable you plug in the plug power plug and the system operates what's even more interesting their breakthrough technology relies on a special material they're using so called lithium niobate essentially they deposit a thin layer of lithium niobate on top of silicon dioxide which sits on top of silicon and this particular material is Q.ANT proprietary technology which is fundamental for the success of their computer chip in several ways first of all it's the only material which allowed them to build all the required optical components in the chip in one material and this is fundamental for avoiding losses losses of light because losses of light results in the drop of accuracy in computations so we want to avoid it at any costs what are the fundamental features of lithium niobate well the first thing is that the modulators so basically whenever you want to interact with the light we can realize modulators that can operate in the gigahertz regime so very fast we can realize these modulators that no light is lost in the modulators and the last thing is the switching so in the end at the technological granularity level what you're doing you're changing the refractive index of the material this can be done only using a voltage and I know this sounds super technical but it's elementary because when you only need to change a voltage there is no electricity on the photonic part of your processor meaning there is no heat there is no heat dissipation leing again to a very clean signal and we already talked about that clean signals are fundamental to reach for instance an 8-bit precision so this is why lithium niobate is not just another material it's basically the fundamental source of success for building a Photonic Analog Computer in fact the Q.ANT
chip is the first photonic chip which is able to achieve the accuracy of 8-bit precision now to be honest what striked me the most about Q.ANT is that they're having their own fab so they are manufacturing their own chips and basically they own the entire pipeline from design to technology then they manufacture the wafers dice them package them write software stack for them that's a lot of work this is very untypical situation for a startup especially owning manufacturing because this is very assets heavy a question is how this upstart start up is managing it all and the most important why why do they need this fab light chips the structure of the light chips are per physical definition so by the laws of physics are pretty large you can't realize a photonic circuit with a 50 nanometer width because then the light would not be guided so in that sense what we have is we have access to a CMOS foundry an old CMOS foundry from the 90s and we repurposed it with strategic investments of a few tools to serve for the production of our own photonic chips so in that sense yes it's not cheap but in comparison to what you need to invest in the CMOS world it's easy and and this is a big advantage to the future as well because think about there are a lot of outdated CMOS foundries in the world which could be repurposed to build high performing chips for the AI next generation AI Supercomputers but using mature technology from the 90s I mean this on its own is a production paradigm shift this is indeed a paradigm shift very interesting example of turning so to say obstacle into opportunity and seeing all the investments governments are making into the photonic technology and into the photonic fabs and also seeing all tech giants like NVIDIA TSMC AMD going all in this fab's might have bright future let me know your thoughts in the comments now before we discuss how this new light- based chip works what it's capable of and how it compares to the state of the art GPUs for example have you ever wondered how much of your personal private data is floating around online your name address even information about your family members unfortunately it all gets out there thanks to the data brokers that spread this information online and this exposes you to risks of data breaches and of course personal security you've probably heard about cases where databases containing information about millions of users are sold online and sadly this is happening more and more frequently that's where Incogni the sponsor of today's episode comes in Incogni helps you remove your personal data from databases used by data brokers I used Incogni to remove my personal data from those databases and it's surprisingly easy you create an account give them permission to act on your your behalf and they send data protection law compliant requests to these companies forcing them to remove your information from their databases and you can even track the progress of these removals day by day on your dashboard as someone who values privacy I highly recommend that you try out and Incogni and put an end to annoying spam emails and calls use my code INTECH at the link below to get 60% % off an annual plan thank you Incogni for sponsoring this episode back to the bright new world now it's time to discuss applications and how it compares to state of the art GPUs and here honestly I spent quite some time looking into specs trying to make apple to apple comparison but it's really challenging one thing is clear that this Analog Photonic approach offers way better efficiency just think about it there is no wires so no resistance no heat generation so these chips require much less power to operate especially at high frequencies and here we are talking roughly about 30 times better efficiency compared to the conventional digital chips now with respect to scalability I think the fundamental question is can we compete with a GPU cluster because this is in the end what our basically what our direct competitor in the present data center is and to give you a bit outline to the future so in two years from now we're going to have Native Processing Units so processors coming on a PCI Express card that have the same performance than a graphic card in two years on the AI relevant functions but on the same side we anticipate that these systems have a 30x smaller power consumption than a graphic card in the future now what does this mean if you look on a server today you can bring eight graphic cards into one server rack and then you're at the edge of what's being reasonable in terms of power consumption we can bring much more cards into the same space and by that increasing the computational density in the server and since we still have energy budget left we can bring much more servers into a server rack and by this increasing so this is the forecast of today and I might be wrong and it's even better tomorrow but the forecast says that if we equip one of those servers and we plug the same electricity in as they plugin today already we can exceed the computational density in the server rack by a factor of 10 what's even more interesting according to Q.ANT their chip is built for both inference and training of AI models and this is very interesting you know typically we distinguish between two different kind of workloads a more simple inference when we have already pre-trained model we apply new inputs to it and we ask to recognize an object an image for example to recognize a fox and on the hardware level this typically reflects into performing many multiply accumulate operations in parallel and we've decided on that we are fully concentrating on the AI Inference and the AI Training so the layout of our chips is always that an a chip can basically serve both purposes so we can run AI Inferences which in the end is fundamentally saying similar to a vector matrix multiplication and on the AI Training we are basically going a different route because we can in contrast to what training or how training is established using a CMOS equivalent GPU architecture but the chip layout is always the same and this is very interesting because in order to do training we need to constantly update the model weights we need to constantly adjust it to improve its ability to make better more accurate predictions and to do this in photonics might be really challenging as we discussed in the beginning of the video in photonics this concept of capacitance or storing in intermediate results does not exist so nothing like in The von Neumann Architecture where we have this local memory in photonics no storage available in fact it works entirely different here the longer we can make the light to propagate through the chip without stopping it the more we can benefit from the properties of light let's say you want to train a neural network first you encode your weight into the phase of light and as the light propagates through the chip you modify it along the way one by one and at the output you get the final value and only then you convert it back to digital and then save it to memory for that the Q.ANT chip features a small electronic part on top of the photonic engine and keep in mind that there is this special fundamental property of light that it can carry a wide range of frequencies within the electromagnetic spectrum to put it simple we can encode many inputs many data at once at different colors of light and comput it all in parallel and this is very attractive when we are dealing with large data sets like in case of AI applications we knew Photonic Computing is new we know that we very soon understood that this technology can really turn the AI world upside down but on the same side to be allowed to enter the ecosystem you need to be compatible with the electronical interfaces so if we had our own proprietary interface there would only be a minor change that we would be adopted into this ecosystem and the second one what was also very clear from the very moment is that the programmers the coders of the world they should not have to change their source code in order to experience the features of our technology at least in the first instance and this is why we have a whole architecture that we call LENA (Light Empowered Native Arithmetic) and this includes the photonic world this includes the electronical world which in the end is the processor that comes on a PCI interface but at the same side we also develop the drivers the compilers the interpretors that are then seamlessly adoptable by the libraries that are used from all the programmers out there from TensorFlor from PyTorch from Keras from ONGs you name it and in that respect it's the easiest way to step into this ecosystem because the customer doesn't have to change anything to be honest I'm really grateful to my channel for this opportunity to talk to the most visionary people out there and this is a very interesting chip and very interesting startup with a big vision and of course there is still a lot of work to be done but to me it seems like we are closer than ever to the light era in computing let me know your thoughts in the comments and I would love it if you could share this video on social media with your friends and colleagues I would really appreciate it still I felt this video wouldn't have been complete without me mentioning another huge transition happening in the industry right now using light for interconnect and here we are talking about interconnecting parts of the chip so chiplets with photonics as well as moving data between the racks in the data center so at the large scale as we've just discussed light has has a much higher bandwidth or if you want a much higher capacity because here we can access frequencies in the terahertz range and that's a lot of course this attracts a lot of interest from tech giants like TSMC NVIDA Intel and AMD recently NVIDIA and TSMC have announced a collaboration in this space they've together developed a silicon photonic- -based chip prototype interestingly TSMC is making this project this innovation a top priority among all their other R&D projects and they call it COUPE which stands for Compact Universal Photonic Engine this new technology will allow TSMC to integrate optical components closer to the processor course and combine multiple electrical chips with with the photonic engine and fiber optic connections into a single package and with these they will come to more compact and more efficient designs you know modern data centers are quite complex and very generally speaking there are two main parts to it computing clusters and networking clusters so when we train a large neural network we need to distribute this workload across the data center and if we try to fit as much as possible into a single cluster and when we are talking about one of the latest GPT models which is roughly two trillion parameters this is really a challenge the thing is it simply won't fit on a single cluster it means we will have to distribute it across many of them and here the efficiency will come down to the wiring to the wiring between clusters to the networking on this channel I talk a lot about computing power of a single chip or a single GPU but at the scale of data center actually networking and wiring makes a lot of difference for example according to Meta about 30 to 50% of a overall elaps time for AI workload is spent in the network waiting for the network just imagine what if we could replace all this complex networking with photonic interconnect startups like Lightmatter and Ayar Labs are working on solving this problem by replacing all these networking switches with photonics Ayar Labs for example is developing a solution that can be applied both to chiplets and data center networking just last December they closed the $155 million funding round that valued the company at more than 1 billion dollars and no surprise that NVIDIA Broadcom AMD and Intel were among the investors so in summary all we discussed today points out to the future where light will play a pivotal role in computing so for the moment we are not focusing on Quantum Computing if this is the question and it's not because I'm not believing in Quantum Computers I believe the future compute ecosystem is going to have a multiple chiplet architecture in my words meaning you're having a CPU you're having a GPU you're having NPUs from us and you're also having QPS Quantum Processing Units but what I realized two years ago was that the time to a commercial product is way faster if we focus on these Analog Photonic Computers because we understood them very well and they're at the heart of a Photonic Quantum Computer so the mathematical operations that we are using on a photonic space are not so much different to what we've been using when we build Quantum Computers but for Quantum Computers it's really unclear you can't predict when you're going to have a system that has an economic advantage not a scientific one we all the time seeing scientific advantages with every new system but it's hard to guess when there's going to be a system on the market that has a clear commercial advantage and on the same side what we realized is that a lot of computations that were linked to Quantum Computers can already very efficiently carry it out by using an Analog Computer I think in the future each of the computing paradigms I cover on this channel whether it's Analog Photonic Digital Probabilistic Quantum or Reversible chips all of them will find its own niche application for example for matrix multiply accumulate operations for AI Inference we are likely to use Photonic engines for Quantum problems we will rely on Quantum Computers and for problems that require high accuracy high precision like banking transactions will be still done on our classical digital chips I hope this video lightened up your day let me know and now watch this video where I explain how computing backwards and Reversible Computing works this episode got a lot of attention and I got a lot of positive feedback on this one so check it out or watch another episode on Probabilistic Computing where I explain how we can harness noise for computation must watch thank you for watching till the end and I will see you in the next episode ciao
2025-02-08 14:48
