The Race For Chip Dominance | CNBC Marathon
The Chips Act set aside more than $52 billion to incentivize chip companies to manufacture on U.S. soil. Samsung's building a huge $17 billion fab in Taylor, Texas, promising to make its first advanced chips in the U.S. next year. What's going on is just remarkable. It's enormous. Really want to be a bedrock for U.S.
industry. Now, Apple's Silicon team has scaled to thousands of engineers, working across chip labs, in places like Israel, Germany, Austria, the U.K., Japan and the U.S., in Austin, San Diego and Silicon Valley. I think one of the most, if not the most profound change at Apple, certainly in our products over the last 20 years, is how we now do so many of those technologies in-house.
For nearly 30 years, Nvidia's chips have been coveted by gamers, shaping what's possible in graphics and dominating the entire market since it first popularized the term graphics processing unit with the GeForce 256. Will you be moving any of your manufacturing to Arizona? Oh, absolutely. We'll use Arizona, yeah. I'm here inside Amazon's Austin, Texas chip lab, where it makes its own custom microchips to compete with those from Intel, Nvidia and other giants. And it's actually a way for them to save money and boost performance, because it's one of the biggest buyers of data center chips in the world.
This is the chip that's inside. For nearly 30 years. Nvidia's chips have been coveted by gamers, shaping what's possible in graphics and dominating the entire market since it first popularized the term graphics processing unit with the GeForce 256. Now, its chips are powering something entirely different.
ChatGPT has started a very intense conversation. He thinks is the most revolutionary thing since the iPhone. Venture capital interest in AI startups has skyrocketed. All of us working in this field have been optimistic that at some point, the broader world would understand the importance of this technology. And it's actually really exciting that that's starting to happen.
As the engine behind large language models like ChatGPT, Nvidia is finally reaping rewards for its investment in AI, even as other chip giants suffer in the shadow of U.S.-China trade tensions and an ease in the chip shortage that has weakened demand. But the California based chip designer relies on Taiwan Semiconductor Manufacturing Company to make nearly all its chips, leaving it vulnerable. The biggest risk is really kind of U.S.-China relations and the potential impact to TSMC. That's, if I'm a shareholder in Nvidia, that's really the only thing that keeps me up at night.
This isn't the first time Nvidia has found itself teetering on the leading edge of an uncertain emerging market. It's neared bankruptcy a handful of times in its history, when founder and CEO Jensen Huang bet the company on impossible seeming ventures. Every company makes mistakes, and I make a lot of them. And, you know, some of them, some of them, puts the company in peril. Especially in the beginning, because we were small and we were up against very, very large companies and we're trying to invent this brand new technology. We sat down with Huang at Nvidia's Silicon Valley headquarters to find out how he pulled off this latest reinvention and got a behind the scenes look at all the ways it powers far more than just gaming.
Now one of the world's top ten most valuable companies, Nvidia is one of the rare Silicon Valley giants that, 30 years in still has its founder at the helm. I delivered the first one of these inside an AI supercomputer to OpenAI when it was first created. 60-year-old Jensen Huang, a Fortune Businessperson of the year and one of Time's most influential people in 2021, immigrated to the U.S. from Taiwan as a kid and studied engineering at Oregon State and Stanford. In the early 90s, Huang met fellow engineers Chris Malachowsky and Curtis Priem at Denny's, where they talked about dreams of enabling PCs with 3D graphics, the kind made popular by movies like Jurassic Park at the time.
If you go back 30 years, at the time, the PC revolution was just starting and there was quite a bit of debate about what is the future of computing and how should software be run. And there was a large camp, and rightfully so, that believed that CPU or general purpose software was the best way to go. And it was the best way to go for a long time. We felt, however, that there was a class of applications that wouldn't be possible without acceleration. The friends launched Nvidia out of a condo in Fremont, California, in 1993. The name was inspired by N.V.
for Next Version and Invidia, the Latin word for envy. They hoped to speed up computing so much everyone would be green with envy. At more than 80% of revenue, its primary business remains GPUs. Typically sold as cards that plug into a PC's motherboard, they accelerate - add computing power to central processing units, CPUs, from companies like AMD and Intel.
You know, they were one among tens of GPU makers at that time. They are the only ones, them and AMD actually, who really survived because Nvidia worked very well with the software community. This is not a chip business. This is a business of figuring out things end to end. But at the start, its future was far from guaranteed. In the beginning, there weren't that many applications for it, frankly, and we smartly chose one particular combination that was a home run.
It was computer graphics, and we applied it to video games. Now, Nvidia is known for revolutionizing gaming and Hollywood with rapid rendering of visual effects. Nvidia designed its first high performance graphics chip in 1997. Designed, not manufactured, because Huang was committed to making Nvidia a fabless chip company, keeping capital expenditure way down by outsourcing the extraordinary expense of making the chips to TSMC. On behalf of all of us, you're my hero.
Thank you. Nvidia today wouldn't be here if and nor the other thousand fabless semiconductor companies wouldn't be here if not for the pioneering work that TSMC did. In 1999, after laying off the majority of workers and nearly going bankrupt to do it, Nvidia released what it claims was the world's first official GPU, the GeForce 256. It was the first programable graphics card that allowed custom shading and lighting effects. By 2000, Nvidia was the exclusive graphics provider for Microsoft's first Xbox. Microsoft and the Xbox happened at exactly the time that we invented this thing called the programable shader, and it defines how computer graphics is done today.
Nvidia went public in 1999, and its stock stayed largely flat until demand went through the roof during the pandemic. In 2006, it released a software toolkit called CUDA, that would eventually propel it to the center of the AI boom. It's essentially a computing platform and programing model that changes how Nvidia GPUs work, from serial to parallel compute. Parallel computing is let me take a task and attack it all at the same time using much smaller machines. Right? So it's the difference between having an army where you have one giant soldier who is able to do things very well, but one at a time, versus an army of thousands of soldiers who are able to take that problem, right? And do it in parallel.
So it's a very different computing approach. Nvidia's big steps haven't always been in the right direction. In the early 2010s, it made unsuccessful moves into smartphones with its Tegra line of processors.
You know, they they quickly realize that the smartphone market wasn't for them, so they exited right from that. In 2020, Nvidia closed a long awaited $7 billion deal to acquire data center chip company Mellanox. But just last year, Nvidia had to abandon a $40 billion bid to acquire Arm, citing significant regulatory challenges.
Arm is a major CPU company known for licensing its signature Arm architecture to Apple for iPhones and iPads, Amazon for Kindles, and many major car makers. Despite some setbacks, today Nvidia has 26,000 employees, a newly built polygon-themed headquarters in Santa Clara, California, and billions of chips used for far more than just graphics. Think data centers, cloud computing and most prominently, AI. We're in every cloud made by every computer company.
And then all of a sudden, one day, a new application that wasn't possible before discovers you. More than a decade ago, Nvidia's CUDA and GPUs were the engine behind AlexNet, what many consider AI's Big Bang moment. It was a new, incredibly accurate neural network that obliterated the competition during a prominent image recognition contest in 2012.
Turns out, the same parallel processing needed to create lifelike graphics is also ideal for deep learning, where a computer learns by itself rather than relying on a programmer's code. We had the good wisdom to go put the whole company behind it. We saw early on, about a decade or so ago, that this way of doing software could change everything, and we changed the company from the bottom all the way to the top and sideways. Every chip that we made was focused on artificial intelligence. Bryan Catanzaro was the first and only employee on Nvidia's deep learning team six years ago. Now it's 50 people and growing.
For ten years, Wall Street asked Nvidia, why are you making this investment no one's using it? And they valued it at $0 in our market cap. And it wasn't until around 2016, that after ten years after CUDA came out, that all of a sudden people understood this is a dramatically different way of writing computer programs, and it has transformational speedups that then yield breakthrough results in artificial intelligence. So what are some real world applications for Nvidia's AI? Healthcare is one big area. Think far faster drug discovery and DNA sequencing that takes hours instead of weeks. We were able to achieve the Guinness World Record in a genomic sequencing technique to actually diagnose these patients and administer one of the patients in the trial to have a heart transplant.
A 13-year-old boy who's thriving today as a result, and then also a three-month-old baby that was having epileptic seizures, and to be able to prescribe an anti-seizure medication. And then there's art powered by Nvidia AI, like Rafiq Anadol's creations that cover entire buildings. And when crypto started to boom, Nvidia's GPUs became the coveted tool for mining the digital currency. Which is not really a recommended usage, but that has created problems because crypto mining has been a boom or bust cycle.
So gaming cards go out of stock, prices get bid up, and then when the crypto mining boom collapses, then there is a big crash on the gaming side. Although Nvidia did create a simplified GPU made just for mining, it didn't stop crypto miners from buying up gaming GPUs sending prices through the roof. And although that shortage is over, Nvidia caused major sticker shock among some gamers last year by pricing its new 40-series GPUs far higher than the previous generation. Now there's too much supply, and the most
recently reported quarterly gaming revenue was down 46% from the year before. But Nvidia still beat expectations in its most recent earnings report, thanks to the AI boom, as tech giants like Microsoft and Google fill their data centers with thousands of Nvidia A100s, the engines used to train large language models like ChatGPT. When we ship them, we don't ship them in packs of one. We ship them in packs of eight. With a suggested price of nearly $200,000.
Nvidia's DGX A100 server board has eight Ampere GPUs that work together to enable things like the insanely fast and uncannily human like responses of ChatGPT. I have been trained on a massive dataset of text, which allows me to understand and generate text on a wide range of topics. Companies scrambling to compete in generative AI are publicly boasting about how many Nvidia A100s they have.
Microsoft, for example, trained ChatGPT with 10,000. It's very easy to use their products and add more computing capacity. And once you add that computing capacity, computing capacity is basically the currency of the valley right now. And the next generation up from Ampere, Hopper, has already started to ship.
Some uses for generative AI are real time translation and instant text-to-image renderings. But this is also the tech behind eerily convincing and some say, dangerous deepfake videos, text and audio. Are there any ways that Nvidia is sort of protecting against some of these bigger fears that people have, or building in safeguards? Yes, I think the safeguards that we're building as an industry about how AI is going to be used are extraordinarily important. And we're trying to find ways of authenticating content so that we can know if a video was actually created in the real world or virtually.
Similarly for text and audio. But being at the center of the generative AI boom doesn't make Nvidia immune to wider market concerns. In October, the U.S. introduced sweeping new rules that banned exports of leading edge AI chips to China, including Nvidia's A100. About a quarter of your revenue comes from mainland China. How do you calm investor fears over the new export controls? Well, Nvidia's technology is export control.
It's a reflection of the importance of the technology that we make. The first thing that we have to do is comply with the regulations. And it was a turbulent, you know, month or so as the company went upside down to re-engineer all of our products so that it's compliant with the regulation and yet still be able to serve the commercial customers that we have in China. We're able to serve our customers in China with the regulated parts and delightfully support them. But perhaps an even bigger geopolitical risk for Nvidia is its dependance on TSMC in Taiwan.
There's two issues. One, will China take over the island of Taiwan at some point? And two, is there a viable, you know, competitor to TSMC? And as of right now, you know, Intel is trying aggressively to get there. And, you know, their goal is by 2025. And we will see.
And this is not just an Nvidia risk. This is a risk for AMD, for Qualcomm, even for Intel. This is a big reason why the U.S.
passed the Chips Act last summer, which sets aside $52 billion to incentivize chip companies to manufacture on U.S. soil. Now, TSMC is spending $40 billion to build two chip fabrication plants, fabs, in Arizona. The fact of the matter is TSMC is a really important company and the world doesn't have more than one of them. It is imperative upon ourselves and them for them to also invest in diversity and redundancy.
And will you be moving any of your manufacturing to Arizona? Oh, absolutely. We'll use Arizona. Yeah. And then there's the chip shortage. As it largely comes to a close. And supply catches up with demand, some types of chips are experiencing a price slump. But for Nvidia, the chatbot boom means demand for its AI chips continues to grow, at least for now.
See, the biggest question for them is how do they stay ahead? Because their customers can be their competitors also. You know, Microsoft can try and design these things internally. Amazon and Google are already designing these things internally. Tesla and Apple are designing their own custom chips too. But Jensen says competition is a net good.
The amount of power that the world needs in the data center will grow. And you can see in the recent trends it's growing very quickly, and that's a real issue for for the world. While AI and ChatGPT have been generating lots of buzz for Nvidia, it's far from Huang's only focus. And we take that model and we put it into this computer, and that's a self-driving car. And we take that computer and we put it into here, and that's a little robot computer.
Like the kind that's used at Amazon. That's right. Amazon and others use Nvidia to power robots in their warehouses and to create digital twins of the massive spaces and run simulation to optimize the flow of millions of packages each day. Driving units like these in Nvidia's robotics lab are powered by the Tegra chips that were once a flop in mobile phones. Now, they're used to power the world's biggest e-commerce operations. Nvidia's Tegra
chips were also used in Tesla model 3s from 2016 to 2019. Now, Tesla uses its own chips, but Nvidia is making autonomous driving tech for other car makers like Mercedes-Benz. So we call it Nvidia Drive. And basically Nvidia Drive's a scalable platform whether you want to use it for simple ADAS, assisted driving for your emergency braking warning, pre-collision warning, or just holding the lane for cruise control, all the way up to a robotaxi where it is doing everything, driving anywhere, in any condition, any type of weather.
Nvidia is also trying to compete in a totally different arena, releasing its own data center CPU, Grace. What do you say to gamers who wish you had kept focus entirely on the core business of gaming? Well, if not for all of our work in physics simulation, if not for all of our research in artificial intelligence, what we did recently with GeForce RTX would not have been possible. Released in 2018, RTX is Nvidia's next big move in graphics, with a new technology called ray tracing. For us to take computer graphics and video games to the next level, we had to reinvent and disrupt ourselves, basically simulating the pathways of light and simulate everything with generative AI. And so we compute one pixel and we imagine with AI the other seven. It's really quite amazing.
Imagine a jigsaw puzzle and we gave you one out of eight pieces, and somehow the AI filled in the rest. Ray tracing is used in nearly 300 games now, like Cyberpunk 2077, Fortnite and Minecraft. And Nvidia GeForce GPUs in the cloud allow full-quality streaming of 1,500-plus games to nearly any PC. It's also part of what enables simulations, modeling of how objects would behave in real world Situations. Think climate forecasting or autonomous drive tech that's informed by millions of miles of virtual roads. It's all part of what Nvidia calls the Omniverse, what Huang points to as the company's next big bet.
We have 700-plus customers who are trying it now, from the car industry to logistics warehouse to, you know, to wind turbine plants. And so so I'm really excited about the progress there. And it represents probably the single greatest container of all of Nvidia's technology: computer graphics, artificial intelligence, robotics and physics simulation all into one. I have great hopes for.
Samsung is a brand name that's everywhere, in more than 100 million U.S. households. Android phones, TVs, refrigerators, microwaves and unconventional displays. So the 13 inch Display can be as big as 17 inch. This is a feature of display. But there's a huge, lesser known side of Samsung that lately has made it one of the world's most important and valuable companies. It's not just making devices.
It's making the chips that power them. People probably don't know that we've led memory for three decades. Samsung is the leader in memory chips. Think long and short term data storage.
They are the Titan. They have, you know, nearly 50% share in both DRAM and NAND. And it's the world's second biggest maker of the most advanced logic chips, the kind in Tesla's, supercomputers, AI, smartphones, and so much more. We recently went inside Samsung's Austin Chipmaking Factory, or fab, in the first in-depth tour ever given to a U.S. journalist.
And how many chips are you pumping out every day here? A lot. Now it's gunning to overtake the massive advanced chip leader, Taiwan Semiconductor Manufacturing Company. We do not settle to be number two. It ended 2022, with $245 billion in annual revenue. For context, that's 47 billion more than Microsoft.
But since then, prices for memory chips have taken a dive, and they're expected to fall up to 23% more in Q2 2023. In April, Samsung reported dismal earnings for the first quarter of 2023, with profit plunging 95% to its lowest level since 2009. In response, the company cut production of memory chips, but it doubled down on foundry, the side of its business that makes custom logic chips for outside customers. It's building a $228 billion mega cluster of
five new fabs in its home country of South Korea, scheduled to come online in 2042. And in the U.S., where the $52 billion Chips Act aims to reshore chip manufacturing, Samsung's building a huge $17 billion fab in Taylor, Texas, promising to make its first advanced chips in the U.S. next year. What's going on is just remarkable. It's enormous.
Really want to be a bedrock for U.S. industry. CNBC got a rare interview with the head of Samsung's U.S. chip business, Jinman Han, and brings you inside its Texas sites to find out how the Korean powerhouse plans to dominate not only devices, but U.S.
chip making. Samsung dates back 85 years, to 1938, when founder Lee Byung chul started it as a trading company for exporting fruit, vegetables and fish in Korea. His vision for a company to be eternal, strong and powerful. So he chose the name Samsung, which literally means three stars. To survive two major wars, it diversified into sugar refining, construction, textiles, insurance, retail, and it remains a multifaceted business to this day. Samsung Rising author Jeffrey Kane has been covering the company from Korea and the U.S.
for over a decade. If you had transported yourself back into time 60, 70 or 80 years ago and asked the average person about Samsung, they'd just shrug their shoulders and say, I guess it's a little grocery store in Korea that no one's really ever heard of. Samsung Electronics, the division it's known for most, was established in 1969. The first Samsung TV came out in 1972, and just two years later, Lee bought Hankook Semiconductor in a bold move toward making it the vertically integrated consumer electronics giant it is today. The Lee family is, you could call it, the most powerful family in South Korea, one of the most powerful families in tech. Samsung's first U.S.
offices opened in New Jersey in 1978, and by 1980, Samsung Semiconductor was born with a fab in Korea. By the early '80s, it was making 64 kilobyte DRAM memory and had a new U.S. office in Silicon Valley. Lee's son took over after his father's death in 1987, and its first mobile phone came a year later. Now, Samsung is the world's biggest smartphone provider, often neck and neck with Apple. Just a decade after
making its first memory chip, Samsung gained international notoriety with the world's first 64 megabit DRAM chip in 1992, placing it squarely at first place in memory, where it remains today. Its presence is so ubiquitous in South Korea that they call their country the Republic of Samsung. In 1996, Samsung broke ground on its Big Fab in Austin, and it opened another one there in 2007. It got a new U.S. headquarters building in Silicon Valley in 2015, designed to look like a three layer stack of flash memory chips. This is based on three nanometer, which is the most advanced technology we have.
And this price is almost same as mid-size car. Han has been with Samsung for more than three decades, while its primary chip manufacturing still happens in South Korea. It, of course makes them in Texas as well as China. Besides devices, the biggest part of its revenue,
some 57%, comes from memory. But as shoppers cut back amid rising inflation, demand has weakened sharply, especially for memory chips. That comes in the footsteps of a pandemic that involved peaking demand and supply chain disruptions, eventually culminating in a global chip shortage. It was really painful when you look at your customers asking "more chips", but there's no way you can provide that.
It was so painful. But the new reality is far less demand. Smaller memory chip makers like SK Hynix and Micron cut production in late 2022.
Samsung waited until April 2023 to do the same. We're now going through the very worst slump in terms of semiconductor demand, and we believe that the market will rebound possibly by the end of this year. Micron and SK Hynix started laying off folks.
They've cut their spending on new fabs. Samsung is pushing forward, though, and they're not cutting back on spending despite it being unprofitable today. Instead, Samsung is shifting focus to foundry making computing chips designed by fabless chip companies. A big difference between Samsung and top foundry player TSMC, is that Samsung makes its own chip designs for its own products, as well as for thousands of others. This includes Tesla, Sony, NXP, STMicroelectronics, Intel, Soon AMD, IBM is also a customer, Qualcomm is of course their biggest customer, but they're moving significantly towards TSMC.
Samsung's stock has been trending down since the peak of the chip shortage in 2021, although it just hit a 52-week high despite dismal Q1 profits. This may be a reaction to the latest move in the geopolitical chip war between China and the U.S. In May, China banned products from U.S.
memory chip maker Micron, which in turn could boost demand for Samsung. And Morgan Stanley recently named Samsung a top pick. In October, the U.S. did place big restrictions on chip companies exporting their most advanced tech to China. But for now, Samsung and SK Hynix were given a one year waiver to operate their existing chip fabs in China.
The Department of Commerce really crafted these rules to make sure that those existing fabs aren't impacted, but Samsung and SK Hynix don't build new fabs. When it comes to foundry, Samsung is one of only three companies in the world capable of manufacturing the world's most advanced chips, ranking second between TSMC and Intel. And with mounting U.S.-China-Taiwan tensions, the U.S. is eager to entice all three to make more chips on American soil.
Good motivation for President Biden's visit to Samsung in South Korea on his first presidential trip to Asia last year. By uniting our skills and our technological know-how, that allows the production of chips that are critical to both our countries and our essential, essential sectors of our global economy. The first factory that I started working in, we did four inch wafer fabrication. I moved on to five. I've done six. Our factory here started at eight.
John Taylor joined Samsung 26 years ago as part of the team at the Austin Fab that broke ground in 1996. Now he heads up the whole Austin site. Everything is supposed to be bigger and better in Texas. Since first coming to the U.S. 45 years ago, Samsung says it's invested $47 billion here and has some 20,000 U.S. employees. Now it's expanding to a 17,000-person Texas city
about 30 miles north of Austin. Bringing Taylor on board is just going to increase their ability to source their chips domestically, and not have to go into areas of the world where they may have some discomfort. Construction began here at Taylor, Texas, less than a year ago, and Samsung says it's on track to be operational by the end of 2024. It's a 1,200 acre, $17 billion site and it's going to be bigger than Samsung's Austin fab. It's also going to be producing the most advanced chips that Samsung makes in the U.S.
Samsung says this big new growth in the U.S. comes down to customer demand, largely due to the geopolitical risks swirling around Taiwan, where more than 90% of advanced chips are currently made. Chips such as the current self-driving chip in the Tesla cars is made in their Austin campus, but that that foundry in Austin currently is for 14 nanometer and older technologies, right? So it's not the leading edge technologies yet. Samsung's seven nanometer, five nanometer, three nanometer, that is all in South Korea.
Over the last 30 plus years, the U.S. share of global chip production has plummeted from 37% to just 12%. That's because it costs at least 20% more to build and operate a new fab in the U.S. than in Asia. Labor is cheaper there, the supply chain is more accessible and government incentives are far greater. The Chips Act aims to change that, setting aside $52 billion for companies like Samsung to manufacture in the U.S.
The Chips Act is helping us to overcome the differences in construction costs that we get out of Asia versus the United States. And there definitely is a difference. That's also why it's Samsung's goal to bring more of that supply chain to the U.S. Of the 17 billion total price tag for Samsung's Taylor, Texas fab, 11 billion is going to machinery and equipment like the $200 million EUV lithography machines made by ASML.
The only devices in the world that can etch with enough precision for the most advanced chips, and the massive machines made by Applied Materials, the world's next biggest microchip equipment company. Every chip in the world made goes through our machines a few times at least. So inside this machine, you are building billions and billions of transistors in a small chip under 100km of wiring. Applied Materials is a key Samsung supplier already based in the U.S. and its growing U.S. operations at the same time as Samsung, with the biggest semiconductor project Silicon Valley has seen in 30 plus years. Why Santa Clara? This is where the collaboration happens between our customers, leading universities and our partners.
But all this growth for Samsung in the U.S. hasn't come without concerns. First, there's water. About 80% of Texas remains in drought.
In 2021, Samsung used about 38 billion gallons of water to make its chips. Where will that water come from here, especially in periods of drought? So we have the Texas Water Board that's working on that and legislation that we're working on this session to make sure that, with a growing population in Texas, we will be able to provide for the water needs, not just for businesses, but also for our growing population. Now, what you see here are the cooling towers behind us. Does. And, you know, we've got a very aggressive goal
this year in Austin, on our Austin campus. We want to reuse over 1 billion gallons of water this year. And we take our sustainability goals very seriously. And even on our Taylor project, which we have starting up, our goals there are to reclaim over 75% of our water. And then there's power. Texas has a uniquely independent grid, largely cut off from borrowing power across state lines.
In 2021, that grid failed during an extreme winter storm, leaving millions of Texans without power and causing at least 57 deaths. So electricity is the lifeblood of a semiconductor fab in a sense, right? There have been multiple instances where electricity has gone out and companies have had to scrap months of production. Samsung told CNBC its Taylor fab will mark its first use of advanced chip etching EUV machines in the U.S., but each
of those machines is rated to consume about one megawatt of electricity. That's 10% more than the previous generation. One study showed Samsung used more than 20% of South Korea's entire solar and wind power capacity in 2020. Already signed 12 laws to make the power grid more reliable, more resilient and more secure. And so we can definitely assure any business moving here, they will have access to the power they need, but also at a low cost.
U.S. expansion aside, Samsung has also faced major scandals at home in South Korea. Corruption charges have kept Samsung's founding Lee family in the headlines for decades.
This is real life succession. That is what Samsung is. It's got the whole shebang. It's got the the shareholder battles, the generational intrigue, the spying. The most recent member of Samsung's founding family to lead Jay Y. Lee served over a year in prison for bribery and
was officially pardoned in August. He took the helm as executive chairman in October. Every major company out there, Apple, you know, they have to bend the knee to Samsung. They have to get their chips, their displays. This is a company that everybody has to go through at some point to get what they need, because they're so influential and they're run by a convicted criminal.
And then there's the big seven year legal battle between Samsung and Apple. Samsung was arguing that its phones were simply using a form factor and a design that would be generic, this rectangle with rounded circles. Apple said that they copied them, so they settled, Apple got a payment from Samsung, so Apple technically won. But when you add up all the the legal costs, all the fighting, all those years, it was just a neutral zero on zero for both sides. To this day, it remains a tricky relationship. They're supplying to Apple, but they're also competing with Apple. And on the flip side, Apple is buying their
chips but then competing with their smartphones. That creates a really weird situation. At the end of the day, controversies haven't impeded Samsung's forward momentum in 2022. It announced an ambitious new roadmap that would, in theory, put it ahead of the far bigger market leader. So is the ultimate goal to catch TSMC to surpass TSMC? You know, one of the things I love about Samsung since I joined Samsung is never satisfied with number two as a business. As a company, we're very aggressive. Now, Samsung's goal is to triple its capacity of leading edge manufacturing and to make industry leading two nanometer chips by 2025 and 1.4 nanometer by 2027.
I mean, if Samsung hits their targets, they'll leapfrog ahead of TSMC, but that's a big if. TSMC is the only one that the industry trusts to hit their roadmap. As geopolitical tensions mount around China and Taiwan, customers are eager for a second source for advanced chips beyond TSMC. Intel, the next biggest advanced chip maker, is also adding manufacturing outside Asia, building big new fabs in Ohio and Europe. We can't be relied upon hostile countries for our everyday needs. And so the United States of America needs
to make sure that we are manufacturing everything that we need. We learned that during the time of Covid, and we shall not make that mistake again. But as Samsung races into leading edge chips, will it lose focus on legacy chips, the kind that saw the biggest shortages during the pandemic, slowing down production of everything from cars to game consoles? This factory that we're in right now is a mature node factory where some people would call that legacy. But there's no, there's no pulling back here.
It's really full steam ahead. But now the AI boom means entirely different chips, namely GPUs from Nvidia, have taken center stage. Nvidia relies primarily on TSMC to make its chips, giving shares of the Taiwanese giant a boost. There are more and more people around the world who can make memory chips, and to stay ahead of the game, you've got to get into the newer some of the newer logic technologies. Samsung's decision to pull back on memory and focus more on foundry, which is all it makes in Austin, now means more custom chips for customers, including perhaps those driving the large language model craze.
There are going to be diving deeper into the logic chip segment. So the AI chips, the, you know, the future applications for semiconductor technology. I think that this would place them more in a segment with Nvidia. But the question remains, is this truly the future for Samsung chips, and can it be achieved in Texas, where Taylor says making three nanometer chips in 2024 is only the start? We currently just have one fab announced there, but plenty of room for more.
And really the what next is looking at what the market needs, what our customers are asking for and being ready to deliver and hopefully right out of Taylor, Texas, with more factories and more investment there. In an unmarked office building in Austin, Texas. Come on in.
There are two small rooms where a handful of employees peer into microscopes, soldering irons, or tiny tweezers in hand. They're designing two types of microchips made to power data centers and more recently, the AI boom. When we get this, what do we do? We test it. First thing that we do, we test it. But these chips aren't coming from Nvidia, AMD, or any of the chip companies that have been hitting headlines and market milestones since ChatGPT burst on the scene late last year. I'm here inside Amazon's Austin, Texas chip lab, where it makes its own custom microchips to compete with those from Intel, Nvidia and other giants.
And it's actually a way for them to save money and boost performance, because it's one of the biggest buyers of data center chips in the world. AWS CEO Adam Selipsky told CNBC that the chips that we saw here today are powering large language models and more for the AI boom. The entire world would like more chips for doing generative AI and whether that's GPUs, or whether that's Amazon's own chips that we're designing, I think that we're in a better position than anybody else on Earth to supply the capacity that our customers collectively are going to want. Amazon Web Services is the world's biggest cloud computing provider and the most profitable arm of the retail giant, with an operating income of $5.4
billion in Q2. Although that number has been down year over year for three quarters in a row, AWS still accounted for 70% of Amazon's overall 7.7 billion Q2 operating profit, giving it the cash it needs for the huge undertaking that is custom silicon and a growing portfolio of developer tools that could eventually propel Amazon to the center of all the AI buzz. Many of our AWS customers have terabytes or petabytes or exabytes of data already stored on AWS, and they know that that data is going to help them customize the models that they're using to power their generative AI applications.
And yet others have acted faster and invested more to capture business from the generative AI boom. Think Microsoft's reported $13 billion investment in ChatGPT maker OpenAI and Google's chatbot Bard, followed by its $300 million investment in OpenAI rival Anthropic. AWS's profit margins have historically been far higher than those at Google Cloud, but those margins have been narrowing.
And although AWS's growth is still impressive, that's happening at a slower pace, too. Amazon is not used to chasing markets. Amazon is used to creating markets, and I think for the first time in a long time, they are finding themselves on the back foot and they are they are working to play catch up. CNBC sat down with top AWS executives and analysts to ask about custom chips, and how it plans to make strides in generative AI to catch Google and Microsoft, and perhaps give a needed boost to AWS, too. We end up with a package part like this.
And this is an actual machine learning accelerator that was designed and you can see Annapurna Labs on it. In 2015, Amazon bought Israeli startup Annapurna Labs to accelerate its dive into the chip business. In July, we went to AWS's Annapurna location in Austin for an exclusive look at the chip design process with lab director Rami Sinno.
AWS also designs chips in Silicon Valley, Canada, and at a larger lab in Israel, then sends them off to be made by chip manufacturers like TSMC in Taiwan. AWS quietly started production of custom silicon back in 2013 with a piece of specialized hardware called Nitro, now the highest volume AWS chip with more than 20 million in use in every AWS server. Then, at AWS's big annual customer conference, re:Invent, in 2018, Amazon launched its ARM-based server chip, Graviton, to rival x86 CPUs from giants like AMD and Intel. That's probably high single digit to maybe 10% of total server sales are ARM, and a good chunk of those are going to be Amazon. So on the CPU side, they've done quite well. We're into our third generation of our Graviton chip that provides acceleration in terms of speed and cost efficiency and power for very general kind of web-based workloads.
After announcing Graviton in 2018, AWS announced its first AI focused chips, VP of Product, Matt Wood, showed us the two AI chips it has today. This big one here is called Trainium, and this small one here. It's called Inferentia. Inferentia, Amazon's first AI chip launched in 2019.
Which was on our second generation of which allows customers to deliver very, very low cost, high throughput, low latency machine learning inference, which is all the predictions of the when you type in a prompt into your generative AI model, that's where all that gets processed to give you the response. With Inferentia, you can get about four times more throughput and ten times lower latency using Inferentia than anything else available on on AWS today. Trainium came on the market in 2021. All right. So this is a packaged part. And then let me show you the other side.
What you see here is all the interfaces. Machine learning breaks down into these two different stages. So you train the machine learning models and then you run inference against those trained models. And so we see a lot of customers that are interested in training their own machine learning models and their own generative AI models. And so that's where Trainium really, really helps. Trainium provides about 50% improvement in terms of price performance relative to any other way of training machine learning models on AWS.
But for now, Nvidia's GPUs are still king when it comes to training LLMs. AWS itself just launched new AI acceleration hardware powered by Nvidia H100s. Accelerating performance by up to 6x and reducing training costs by up to 40% as compared to EC2 P4 instances.
Nvidia chips have a massive software ecosystem that's been built up around them over the last 15 years that nobody else has. The big winner from AI right now is Nvidia. That seems clear. Still, Amazon is not the only non-chip giant getting into custom silicon. Apple has its M-series of chips. And a couple of years before Amazon had AI chips, Google launched its own cloud tensor processing units, or TPUs. Nobody's at the same scale as Google.
Google has been deploying this stuff for like eight years. My assumption is all of the hyperscalers, whether they've announced it or not, are all working on their own accelerators, and many are also working on their own CPUs as well. But when it comes to custom chips, Microsoft is lagging behind Amazon and Google. Microsoft has yet to announce the Athena AI chip it's been working on, reportedly in partnership with AMD. I think the true differentiation is the technical capabilities that they're bringing to bear, because guess what? Microsoft does not have Trainium or Inferentia.
Generative AI is the current craze, but Amazon was building out a broader AI infrastructure for machine learning with dozens of services long before it made chips or used them to train LLMs. Late 1990s, we were the first ones to actually leverage machine learning-based technologies to reinvent our recommendation engines, and we leveraged machine learning to do things like better product search and then automating leveraging robotics and computer vision in our Amazon FCs, or fulfillment centers to help products ship faster, to actually reinventing completely new customer experiences with things like Amazon Alexa. But when OpenAI launched ChatGPT in November 2022, Microsoft was suddenly dominating the AI headlines, followed by Google's Bard in February. Two months later, Amazon announced its own large language model called Titan and Bedrock, a cloud service to help developers enhance software using generative AI. I think ChatGPT and Microsoft rollout of their initiatives were so fast, so aggressive, so quick, it caught a lot of the market participants flat footed. Amazon is trying to educate the market in order to close the gap, but frankly speaking, it's going to take a couple of months.
Let's rewind the clock even before ChatGPT. It's not like after that happened, suddenly we hurried and came up with a plan. Because you can't engineer a chip in that quick time. If anything, it actually accelerated some of the customer conversation and their keenness to actually move forward with generative AI deployments. Meta also recently announced its own LLM, Llama 2, the open source ChatGPT rival is available on Microsoft's Azure Cloud platform.
Now, a leaked internal email shows Amazon CEO Andy Jassy is directly overseeing a new central team that's building out expansive, large language models, but so far, AWS has focused on tools instead of building a ChatGPT competitor. So if you look at the Bedrock strategy that they are trying to focus on, they are betting the farm on the fact that enterprises might not necessarily be building out their own GPT models. Bedrock gives AWS customers access to LLMs made by Anthropic, Stability AI, AI21, and Amazon's own Titan. Titan is actually a family of foundational models.
We have text-based models, which are great for generative texts, so creating marketing copy and advertising, chat bots, those sorts of things. And then we have an embedding model, which is great for personalization and ranking those sorts of use cases. Amazon says its AI products are being used by numerous customers like Philips, 3M, Old Mutual and HSBC. In the Q2 earnings call, it said a very significant amount of AWS business is now driven by AI and the 20+ machine learning services it offers.
We don't believe that one model is going to rule the world. We want our customers to have the state of the art models from multiple providers because they are going to pick the right tool for the right job. One of Amazon's new AI offerings is AWS HealthScribe, a service unveiled in July to help doctors automatically draft patient visit summaries and more. Another big tool in the AWS AI stack is CodeWhisperer. CodeWhisperer generates code recommendations from natural language prompts based on contextual information.
Participants who use CodeWhisperer are 27% more likely to complete their task successfully, and they did it 57% faster on average. Last year, Microsoft also reported productivity boosts from its coding companion GitHub copilot. And then there's SageMaker, Amazon's machine learning hub that offers algorithms, models and more. Autodesk, for instance, they were able to leverage generative foundational models to design a bulkhead of an aircraft.
In one example, it was 45% lighter for a particular carrier. In June, AWS also announced a $100 million generative AI innovation center. We have so many customers who are saying, I want to do generative AI, but they don't all necessarily know what that means for them in the context of their own businesses. And so we're going to bring in solutions, architects, and engineers, and strategists, and data scientists, to work with them one-on-one. When companies are choosing between Amazon, Google and Microsoft for their generative AI needs, some may choose Bedrock because they're already familiar with AWS, where they run other applications and store a ton of data.
If you took the data that we have in Amazon S3, It's stored on devices like this, and you stack them up one of one on top of another, it would take you all the way to the International Space Station and almost all the way back. And that is a lot of data. At the end of the day, Amazon does not need to win headlines. Amazon already has a really strong cloud install
base. All they need to do is to figure out how to enable their existing customers to expand into value creation motions using generative AI. So how many AWS customers are actually using it for machine learning? We have over 100,000 customers today that are using machine learning on AWS, many of which have standardized on our machine learning service, which is called SageMaker, to build, train and deploy their own custom models. But in reality, that's not a big percentage of AWS's millions of customers.
Although most aren't tapping into it for AI yet, that could change. What we are not seeing is enterprises saying, 'Oh, wait a minute, Microsoft is so ahead in generative AI, let's just go out and let's switch our infrastructure strategies, migrate everything to Microsoft'. That is not happening because at the end of the day, even if you're trying to create a chatbot, if you're already an Amazon customer, chances are you're likely going to explore Amazon ecosystems quite extensively. How quickly can these companies move to develop these generative AI applications is driven by starting first on the data they have in AWS, and using compute and machine learning tools that we provide. Imagine you're cooking dinner and you're using a new recipe. It is a lot faster to start with ingredients that you already know, that have been cut and prepared to go ahead and put together the recipe, than if you have to research all the ingredients, get familiar with them, and then learn how you're going to put them together and cook with them. Right? That's what AWS customers are doing.
They have all the different ingredients that they're familiar with and they know how to use, and whether that's storage or a compute or it's a machine learning tools like Amazon SageMaker and Amazon Bedrock, and they're putting it together that much faster. And as generative AI continues to accelerate, all the big players are scrambling to establish how to use these tools responsibly and securely. I can't tell you how many Fortune 500 companies I've talked to who have banned ChatGPT. So with our approach to generative AI and our Bedrock service, anything you do, any model you use through Bedrock will be in your own isolated virtual private cloud environment. It'll be encrypted, it'll have the same
AWS access controls. Selipsky joined six other AI players at the White House in July to sign pledges to ensure that AI tools are secure. There are open problems that still need to be solved, especially when you're trying to deal with highly regulated industries in financial services, healthcare and beyond. We still do not have any well thought out regulatory guardrails around data protection, private information protection and responsible AI capabilities in the space. There's also national security concerns. The Biden administration has proposed new rules that would require U.S. cloud providers like Amazon and Microsoft to
seek government permission before providing China with cloud computing services using AI chips. But for now, there's no slowdown in sight for the development of new generative AI applications or the chips needed to power them. And that race is just beginning. So let's say that we're three steps into a race and we start asking, 'Well, who's ahead? Who's behind? How do the runners look?' But then you look up and you realize that it's a 10k race, and then you realize it's the wrong question to ask, who's where three steps into the race? The real question is what's going to happen at the end of the 10k race? In this case, we're just at the very dawn of generative AI.
Past a small lobby through a plain set of double doors. Welcome to one of our chip labs. There's a simple room filled with a couple hundred machines, blinking lights, a handful of engineers in lab coats, and a bunch of postage stamp-sized chips being put through rigorous testing.
Our goal is to be able to find bugs, manufacturing issues, design issues. We want to find them so that we can fix them and address them before we ship our chips into our systems. Apple has enjoyed soaring valuation for years thanks to its forward facing consumer products MacBook, iPhone, Apple Watch, AirPods. But under the hood, it's also designing its own custom silicon that powers them all. Mac transition is a proud moment.
When we started scaling our chips to iPad and watches proud moment. Building chips for the AirPods that you couldn't chip otherwise, proud moments. Apple first debuted its own semiconductors in iPhones in 2010. As of June this year, all new Mac computers are powered by Apple's own silicon too, ending the company's 15-plus years of reliance on Intel.
I think one of the most, if not the most profound change at Apple, certainly in our products over the last 20 years, is how we now do so many of those technologies in-house. But Apple isn't immune to industry risks. All its most advanced silicon is manufactured by one player, Taiwan Semiconductor Manufacturing Company. Smartphones are just recovering from a deep sales slump, and competitors like Microsoft are making big leaps in generative AI. It's doable. On Apple's last year chip even more capable on
this year's chip with M3, but the software has got to catch up with that, so the developers take advantage and write again tomorrow's AI software on Apple Silicon. CNBC went to the company's California headquarters, where we were the first journalists allowed to film inside an Apple chip lab and got a rare chance to talk with the head of silicon, about how it broke into the incredibly expensive, complex business of processors, kicking off a trend of non-chip companies like Amazon, Google, Microsoft and Tesla now rushing to do the same. Apple launched the first iPhone in June 2007 with a 90 nanometer processor inside, made by Samsung. In 2008, Johny Srouji came on board after stints at IBM and Intel. I came to Apple with the purpose of building our own silicon for the iPhone. It was a very small team at the time, about 40 to 50 engineers, and since then we have grown the team immensely.
A month after Srouji joined, Apple bought 150-person chip startup PA Semiconductor for $278 million. They're going to start doing their own chips. That was the immediate takeaway when they bought PA Semi. Just I think knowing the culture of Apple, their inherent design focus to control as much of the stack. Indeed, two years later, Apple launched its first custom chip, the A4, in the original iPad and the iPhone four.
And we built what we call the unified memory architecture that is scalable across products. We built an architecture that you start with the iPhone, but then we scaled it to the iPad and then to the watch and eventually to the Mac. Now, Apple's silicon team has scaled to thousands of engineers working across chip labs in places like Israel, Germany, Austria, the U.K., Japan and the U.S. in Austin, San Diego and Silicon Valley, where we were the first journalists to ever film inside one. On either side of me, there's about 70 machines testing chips, and these are the M3 series that are going in the new MacBooks, as well as the A-series chips that end up in iPhone 15s. And these machines in particular are testing them for extreme temperatures, high heat, low temperatures, things like that.
Our board holders, we've kind of color-coded them. And what that does is that makes it easier for engineers and technicians to just be able to spot, oh, I'm looking for an M3 validation board. The primary type of chips Apple is developing here are known as systems on a chip or SoCs. It is the silicon and all of the blocks that go onto that silicon. So there's CPU, there's GPU, there's DSP,
there's accelerators. And in Apple's case there's also an NPU that runs the neural engines. Apple's first SoC was the A-series, which has advanced from the A4 in 2010 to the A17 Pro, announced in September. It's the central processor in iPhones, as well as some iPads, Apple TVs and HomePod. Its other major SoC is the M series, now powering all new Macs and more advanced iPads.
First released in 2020, it's now up to the M3 Macs. The S-series is a smaller system in package or SIP, for Apple Watch, first launched in 2015. H and W are even smaller chips used in AirPods. U-chips allow communication between Apple devices and the newest chip, the R1, is for Apple's Vision Pro headset, processing input and streaming images to the display within 12 milliseconds.
We get to design the chips ahead of time, working with our partners from John's team on software and OS to exactly and precisely build chips that are going to be targeted for those products and only for those products. The H2 inside the second-gen AirPods Pro, for instance, enables better noise cancellation. Inside the new series 9 Apple Watch, the S9 enables unusual capabilities like double tap. On the iPhone side, the A11 Bionic in 2017 was a major milestone because it had the first Apple Neural Engine, a dedicated part of the SoC for performing AI tasks totally on device. That was when we first started looking at wow, how do we bring this advanced intelligence into things like our camera pipeline? So things in iOS 17 today, like being able to lift the subject from a photo.
Now leading marketing for the iPhone, Kaiann Drance has been with Apple for more than 15 years. She says the latest A17 Pro is another major leap forward, enabling big changes to features like computational photography. You can take seven different shots all from the same location. This is a macro shot, and then you've got the
ultra wide lens at 13mm. With iPhone 15 Pro Max, you can go all the way up to 5x optical zoom. The A17 Pro's new GPU architecture also enables advanced rendering for gaming on the iPhone 15 Pro. It was actually the biggest redesign in GPU architecture in Apple and Apple silicon history.
So we have hardware accelerated ray tracing for the first time, and we have mesh shading acceleration, which allows game developers to create some really stunning visual effects. Now, for the first time, some big games are coming out with iPhone native versions. Ubisoft's Assassin's Creed Mirage, The Division Resurgence, and Capcom's Resident Evil 4. Apple says the A17 Pro is the first 3 nanometer chip to ship at high volume. The reason we use three nanometer is it gives us the ability to pack more transistors in a given dimension. That is important for the product and much
better power efficiency. We're leading even though we're not a chip company, we are leading the industry for a reason. Apple's leap to 3 nanometer continued with the M3 chips for Macs announced in October. These are the main
chips CNBC saw being tested in Cupertino. We've got one of those M3&#
2025-02-13 05:57
