2024 Taiwan Edge AI Day Forum

Thank you for your patience It's now time to start today's seminar We have a large audience here today Thank you very much I have an announcement to make Today's speaker will be giving the speech in English We have simultaneous interpretation receivers available for those who need them Please pick up the simultaneous interpretation receiver at the reception Yes, we are gathered here today for Taiwan Edge AI Day I would like to start today's program right away Today, we have two speakers from Taiwan and Mr. Kenji Tsuda, the author of an NVIDIA book First, here is the agenda, and we are the hosts Representing TCA, I will give a greeting, and then the next program will be I will briefly introduce the IC Taiwan Grand Challenge and the Taiwan AI Alliance After that, I would like to move on to the speeches from three speakers There are three speakers The first speaker specializes in edge AI They are from a design house for AI chips, on the far left Next, we will have a presentation from Albert Liu of Kneron Kneron is a company aiming to be Taiwan's first unicorn It is a company that is currently gaining a lot of attention in Taiwan The second speaker is Mr. KS from PHISON Electronics He will talk about the theme of personal use of generative AI And the third speaker will present case studies on edge AI Mr. Kenji Tsuda, an international technology journalist and author at NVIDIA

First, on behalf of the organizers, TCA's Deputy Secretary-General which is equivalent to the position of Deputy Executive Director in Japan I would like to give a greeting from Sakura Hello everyone Good afternoon I'm Sakura Yang from Taipei Computer Association I'm the Deputy Secretary General and I'm one of the organizers of today's event So I'll be using English to give my welcome address As a Gen AI from the cloud to the edge AI users can benefit from the faster and more personalized experience which also creates new changes to the next generation of AI I understand that Japan is actively revitalizing their semiconductor industry and strongly promoting innovative AI applications Taiwan and Japan have a strong mutual trust The topics of today's forum focus on edge AI and semiconductors We hope that through this exchange we can create more cooperation opportunities for Taiwan and Japan companies and expand to a new market together next, I would like to introduce today's co-organizers The Taiwanese government has been actively promoting the development of AI and semiconductor industries They are implementing various policies through close ministerial cooperation For example, the Asia Silicon Valley Development Agency SBDA, was jointly promoted by the National Development Council NDC, the Ministry of Economic Affairs MOEA, National Science and Technology Council, NSDC, and other ministries Implemented since September 2016 this year marks the new era of SBDA 3.0 which aims to accelerate digital and net zero transformation of Taiwan's industries In March this year we started promoting the Taiwan chip-based industrial innovation program Taiwan's CBI, the Taiwan government not only invested huge amounts of money but also mobilized many government agencies to promote Taiwan's CBI A vital part of the program is the IC Taiwan Grand Challenge that is sponsored by the NSDC The competition aims to leverage Taiwan's strengths as a Silicon Island to attract global top startups and VCs to Taiwan Registration for the second batch of IC Taiwan Grand Challenge is open and startups from Japan are welcome to join In addition, the Taiwan AI Alliance managed by TCA actively connects manufacturers in various fields such as computing software, hardware, cybersecurity etc., together to promote the development of AI as a technology and applications in Taiwan The Alliance currently has 45 member companies next, I will introduce Taipei Computer Association, or TCA for short So it's the 50th anniversary for Taipei Computer Association this year so it's a historical association in Taiwan with TCA is Taiwan's largest ICT industry association which is equivalent to Japan's incorporated association Our member companies cover a wide range of industries including computer manufacturers such as Acer and Asus as well as semiconductor companies such as AMD, NVIDIA, MediaTek, and PCMC The total output of TCA member companies contributes to over 80% of Taiwan's ICT industry We also manage multiple associations to promote industry developments such as GLORA in the LPWAN field TWIOTA in the IoT field, TADA in the automotive field the Taiwan-Japan Tech Networking Alliance to promote business opportunities in Taiwan and Japan and the RISC-V Taiwan Alliance For your reference, the chairperson and member companies of these associations can be seen on this slide Next, we will introduce COMPUTEX which is organized by TCA and reached new heights this year The theme of 2025 will be AI Next Many semiconductor manufacturers have confirmed their participation in COMPUTEX 2025 making it the world's best B2B ICT platform Held currently with COMPUTEX InnoVEX was established in 2016 and has become the startup hub of Asia InnoVEX has also achieved great results this year gathering the most diverse lineup of startups yet Japanese startups who are interested in InnoVEX are welcome to register next, I would like to introduce the AI and Semiconductor Forum which will be held in Tokyo on December 13 We will invite great speakers to share their insights in the forum which will be one of the official forums of Semicon Japan Everyone is welcome to join the forum and event Finally, I hope today's forum can be beneficial to everyone Thank you very much Please wait a moment while we switch the screen However, it's December 13th that I just introduced We are preparing another seminar on the theme of semiconductors We will have four speakers, including Professor Kuroda from the University of Tokyo, and many from Taiwan We plan to hold it at Big Sight on December 13th with invited speakers We hope you will join us again Next, I would like to talk about what Sakura just mentioned I will discuss the IC Taiwan Grand Challenge, and also I would like to briefly introduce the Taiwan Alliance Federation IC Taiwan Grand Challenge This may be a keyword that you are hearing for the first time Currently, this program is receiving a lot of attention both domestically and internationally That's right The Taiwan Chip-Based Industrial Innovation Program We refer to it as CBI for short However This CBI is a core project of the Taiwan business ecosystem Today, regarding this project I don't have much time to discuss the details So I would like to briefly touch on the key points However There are three key points of the grand challenge For Taiwanese companies, the market is global from the start The second point is that they have a strong team of experienced mentors And the third point is this grand challenge Overseas accelerators, especially from Europe, are paying close attention The market is global from the start, with a robust team of mentors Accelerators focused on Europe are taking notice I hope you will take note of these points When it comes to business in Taiwan, many may think about developing the Taiwanese market For Taiwanese companies, the market is not domestic; it is overseas from the beginning That market has also changed quite a bit We have been producing high-spec, affordable computers in large quantities and selling them overseas However, times have changed to the era of IoT, or AIoT In Taiwan, we refer to this as smartification For example, in the area of smart factories and smart medical We are starting to deploy various solutions tailored to those fields overseas In this kind of expansion, the strength of Taiwanese startups is essential Taiwanese startups aim for overseas markets by collaborating with major companies Major vendors also incorporate startup technologies to target overseas markets Thus, the interests of both parties align And the second point is strong mentors This is a slide I would like you to take a look at In the center, at the top, we have TSMC, PSMC, and UMC These are the foundries of Mycote These companies include MediaTek and design houses On the upper right, we have Wistron, Pegatron, Compal, and Quanta These are major EMS companies from Taiwan Representatives from these companies are listed as mentors and partners To materialize excellent technologies, solutions, and business models Recent business focuses on commercializing startup technologies We provide strong advice to monetize However, these mentors do more than just give advice They gather technologies and solutions from startups both domestically and internationally They quickly assess whether these can be commercialized You could say they are scouting for opportunities They incorporate these into their own business and expand rapidly This is a significant difference from Japanese mentors and partners In a way, they are people looking for ideas for their own business Accelerators from around the world are paying attention to this movement Using Taiwan as a base camp, they first bring startups from various regions to Taiwan We aim to match with major IT vendors and hand-held device vendors in Taiwan By aiming for the global market, this represents Taiwan's unique business ecosystem Unfortunately, Japanese companies seem to be a bit slow to respond to these movements There are very few entries from Japan I hope more Japanese startups and small to medium-sized enterprises with excellent technology will take notice I encourage you to pay attention to these movements and collaborate with Taiwan I hope you will aim for the global market, and I encourage all of you Yes, I hope you remember the term 'Grand Challenge' starting today Next, I would like to briefly explain the AI Alliance Today, I will skip the detailed explanation Regarding the earlier Grand Challenge and the AI Alliance If anyone would like detailed materials, I can provide them later in PDF format So, after the seminar please exchange business cards with me, or write 'request for materials' on your card If you write 'request for materials' on your business card and leave it at the reception I will provide the materials to everyone later in PDF format As for the AI Alliance it was established in June 2023 This is an organization that leads AI-related technology and solution businesses in Taiwan I apologize for repeating myself today, but I would like you to take a quick look at what activities we are involved in If anyone would like more detailed materials Please exchange business cards or write 'request for materials' on your card and leave it at the reception We are collaborating with companies like Foxconn, MediaTek, and AMD Our partnerships with these companies are progressing rapidly Regarding LLMs, or large language models And also, the digital transformation in traditional industries is rapidly evolving Additionally, regarding B2G collaborations Talent development and talent alliances Regarding the talent supply chain About inter-company cooperation and international collaboration And about the various events that are being held For those interested, we will provide materials later Here are the core members, regarding the core members And this is the AI Taiwan event that was held this year In fact, such events are also being held This is the Taiwan-style business ecosystem This is a key point I want everyone to know Taiwan-style business ecosystem Clockwise circulation, counterclockwise circulation In this way, the industry in Taiwan's infinite loop goes from entrance to exit The ecosystem is starting to function quite well If you would like the materials, I would love to exchange business cards Alternatively, as a reminder, please write 'request for materials' on your business card You can leave it at the reception when you return Yes, that concludes my explanation Now, let's quickly move on to the speaker's presentation. First up is?

It's Albert Liu from Kneron, and Kneron is A design house for AI chips with strengths in edge AI As I mentioned earlier, this is a company aiming to be Taiwan's first unicorn From here, I would like to invite Mr. Liu and Mr. Albert to give their presentations Thank you Hi everyone, I'm Albert, founder and CEO of Kneron So Kneron is the very first company on the market to commercialize NPU Neural Network Process Unit So we are the pioneer and also the global leader in edge AI industry And we truly believe that the future of AI should be on the edge And we all know that the career of AI send me dramatic growth And based on Bloomberg's intelligence saying that in 2032 right now it's only up to 2037 for those two figures But in 2032, they expect the edge AI market 3.37 times bigger than cloud AI today And we all know the cloud AI leader is the media, right? So that means in the future, if there's a global leader stand out in edge AI that means no matter the valuation or stock price or even the market share is bigger three times bigger than today's MBS value and we all know that CPU, GPU and NPU, right? Right now NPU pretty much has become a general term but NPU stands for Neural Network Process Unit This term and also the name were the core patterns owned by Kneron actually And if you want to learn NPU most likely you will read a textbook, which I wrote It's probably generally used by Stanford or Princeton And that's also the reason we got the IEEE Dining Terminal and IEEE CTSOC award And actually, this is very similar like technology migration We all know the VCD, DVD and MP3, right? One MP3 can, like the song or the video stored in the MP3 probably is 1000 times better than VCD and like 500 times better than DVD The same thing, CPU is already in the industry for more than five decades And GPU is already in the industry for more than three decades And right now it's a new era, which is AI era, right? So that means we really need a new hardware which is Neural Network Process Unit We shouldn't use the older technology no matter it's logic computing purpose CPU 50 years old and GPU, 30 years old architecture to run today's AI and Kneron has been found in San Diego back to 2015 And right now we already have a hundred to thousands of different customer stand from AIoT, 7N, CGI and Oto And each vertical, we all have the global leader who already commercialize their products or chip inside For example, in AIoT, we have kind of Sony, yeah, it's a Japan brand, right? And also Philips, Garmin, et cetera And Oto, we have Toyota JVC, Kenwoo, and even Foxconn closely working with us for a long time And Surfer, we have never the Korean's largest search engine, like Korean Google and our vision is using the game changer technology as I just previously mentioned, for VCD, DVD to MVC We are also trying to push the migration, technology migration in AI So using this game changer, powerful NPU to enable the intelligent in device to do the game changer or technology revolution close to the different industries so today, AI is more in the cloud It's because we are using the older architecture, which is GPU, right? And then in the cloud, they will cause lots of different problems For example, because every your information will centralize in the big data center And because it's centralized in the big data center, then, for example Japanese government, if they want to have their conference and depends on GPD to do your summarize or your summary meeting summary, then that will leak your confidential information and even national security labels information, right? That's the key issue to cause today's geographic political tension And not only the security label stuff but also for some application, for example, drone or auto If you want to detect stuff and transfer the image or surrounding information to the cloud and make the judgment even it costs 0.2 second delay that will hire the possibility for your auto to hit the pedestrian or another vehicle Another thing is we all know the GPU power consumption is extremely high Today, we all know winter is getting hotter and hotter Even this winter, there is a typhoon Most of the reason is because we are using the GPU that will generate thousands of carbon dioxide Right now, even one surge of the GPD, the power span can cook one chicken That means if one day has 100 million people searching through the GPD that means we are cooking 100 million chicken That's quite dangerous to our environment So the HDI means depends on the NPU enable intelligent on the local device Then that will solve significant power consumption and also no latency because it doesn't need to connect to the cloud and also no security label with quite strong security label We always call that HDI more like Doraemon-like AI The cloud AI is more like Metri or Skynet Terminator AI We believe that Doraemon-like AI is more human-like AI and it should be the future of AI so as I mentioned, the cloud AI is pretty much everything to the cloud and you don't know who is trying to teach the cloud AI the supercomputer Let's say if some crazy scientist keep sending the message to the cloud AI saying that if one day you control the nuclear power weapon you should destroy the human society, which would really happen And so that's higher a lot of uncertainty and also lots of the tension So the barrier of the cloud AI no matter its cost or speed of processing or privacy, really cause the issue And the local AI, the edge AI, powered by NPU can provide low energy efficient, and also high security and also flexibility and security So we believe the future AI must move from the cloud to the edge and this is the number of today's ChatGPT costs If you want to serve 100 million people the annual electricity cost is up to $5 billion And that will generate 170 milliwatt and also 50 milliwatt tons of carbon dioxide Pretty much one Taiwan's nuclear power plant can only support half of the GPT server powered by GPU And the total cost of the ChatGPT server powered by GPU pretty much is the same as Singapore's annual cost So that's the reason many of the tech giants no matter it's Microsoft or Google claim that they want to build a nuclear power plant to support the future of the power-hungry machine powered by GPU which is non-sustainable and also ridiculous and then on trying to use the next generation or future hardware what we call the NPU, New Network Processing Unit means the power consumption efficiency is 1,000 times better than GPU The same idea is just like using USB dongle can repress 1,000 of the DVD Then we already have a bunch of different NPUs on the market more than 6 of them And we commercialized our first NPU back to 2017 Ingrid, the largest air-conditioned company in the world And so far that IP already shipping more than 140 million units And we are also the very first company to come out the name or come out the structure for NPU back to 2015 and we also have a software platform what we call KNEO You can notice that the name of the KNEO is because it's the main character in Metri because we believe that we should be the right side And this platform is very user-friendly And even the elementary school students can program and leverage that platform to play and use our AI we also use our architecture to come out the different type we call the scalable GPT machine One is up to the pretty big one Right now, recently we are helping Saudi Arabian government to build their national software AI And also the KNEO 330, I think we have been helped by some big finance and also medical centers and some manufacturing semi-industry leader in Taiwan to build a local GPT And the KNEO 300 is more like a small group local GPT which can provide quite low power consumptions and also quite high security and high privacy local GPT So those two videos, the next two videos, are all generated by our AI engine And the power doesn't need to consume up to the nuclear power plant's levels and won't cook the chicken's level Let's see how the video plays welcome to the future of AI with Kneron's Edge server You cannot tell it's generated by a human being where AI actually stole from AI even the voice and the image and everything With Kneron's Edge server experience faster processing enhanced security, and unmatched versatility across industries It's more than just technology It's the next step in transforming how you work, connect, and thrive You can also not tell it's powered by GPU of our machine neuron integrates into every part of your business from administration to marketing research and development to legal, human resources to finance The Kneron Edge server handles the heavy lifting providing real-time insights automating routine tasks and enhancing decision-making across your organization With this all-in-one solution your enterprise becomes more efficient, secure, and future-ready Kneron, where cutting-edge AI meets the needs of your entire business Meet the Kneron Edge server, your private GPT machine designed to process and protect your company's information all while keeping it secure on your premises With the Kneron Edge server your company's data is uploaded stored, and processed right on-site, ensuring full control and privacy It leverages powerful AI to analyze your data and generate insights all in real-time Every industry faces unique challenges that require specialized AI solutions Vertical LLM applications are the answer tailored to meet the specific needs of each domain Kneron's full-stack approach encompasses everything from research and data collection to model training edge computing, and application deployment This allows us to create seamless scalable AI solutions that are secure and efficient Kneron's products are making a difference in real-world applications across retail healthcare, smart cities, manufacturing financial services and academia, enabling smarter, safer, and more efficient operations The Kneron Edge server is a versatile powerhouse capable of handling a wide range of tasks that keep your business running smoothly From managing PDFs, Word documents Excel sheets, and PowerPoint presentations to summarizing meetings and converting speech to text, it's all within reach Need answers? The Edge server can provide multiple solutions to a single question deploy different LLM models, and even fine-tune them to your specific needs With the Kneron Edge server, everything you need is right at your fingertips bringing efficiency and intelligence to every corner of your enterprise The possibilities with the Kneron Edge server are endless It's energy efficient, cost-saving private, and incredibly versatile designed to meet the diverse needs of your business This is more than just technology It's the future of AI, empowering your enterprise to achieve more with less Step into the future with Kneron, where innovation knows no bounds Visit us at www.neuron.com to learn more Follow us on social media to stay updated with the latest innovations Previously, before the generative AI technology became mature we spent like $2 million Taiwan dollars to come up with a commercial video And actually, the commercial video is not that good as the one we just generated and we showed you guys And it also takes like months to make up this video But after we have this technology we pretty much only spent one hour and one engineer And the machine power consumption is around 20 watts And we all know like NVIDIA's H100 pretty much the power consumption is up to thousands of watts That means the power cost or electricity cost is like a few hundred times bigger And the money you need to spend is also a few hundred bigger And carbon dioxide emission you generate is also a few hundred times and also not that good for our Earth This is another video we asked our engine to create a commercial video to introduce our auto solution Let's take a look welcome to the future of automotive technology where cutting edge innovation meets unparalleled performance Introducing Kneron's advanced Edge AI technology for automotives Edge AI is transforming the automotive industry at every level You can see the drive, the auto should be on the right hand side, right? But it's on the left And also, so that's really generated by AI It's not, it's not by With ultra low power consumption ensuring your vehicle is always one step ahead Today, we are living in an ecozystem of technology and transportation Our Edge AI technology processes data in real time directly within the vehicle This means faster decision making enhanced safety and improved efficiency all while keeping your data secure and private Kneron's different AI system on chips are designed to enhance the visual and processing capabilities of devices particularly for automotive applications The KL630 series excels in real time tracking capable of monitoring multiple targets around a vehicle with very low latency This feature is crucial for applications such as driver monitoring systems and advanced driver assistance systems Where in today's rapidly evolving technological landscape DMS and ADAS and DMS are the future of fleet safety solutions The most advanced in the lineup of these SOCs is the KL730 series It's support up to 8 million pixels at 60 frames per second They serve as powerful vision and intelligence organs for vehicles providing high resolution imaging and advanced processing capabilities to enhance the vehicle's ability to perceive and interact with its environment neuron's Edge AI supports a wide dynamic range stellar level illuminance as well as panoramic fish eye camera hardware correction Our solutions operate effectively on moving vehicles under complex lighting conditions including overexposed and low light scenarios Kneron supports safe driving through blind spot detection collision alert, as well as driver behavior monitoring Kneron empowers L0, L2 intelligent driving scenarios with its unique AI SOC with high performance and low power consumption Equipped with the Kneron AI algorithm it can efficiently identify people vehicles, signs, and obstacles et cetera, and offer users with a safe and extraordinary driving experience Kneron has revolutionized the automotive industry with its state-of-the-art face recognition technology enabling a seamless and secure way to access vehicles Kneron's face recognition technology offers a high level of security by ensuring that only authorized users can access the vehicle This reduces the risk of theft and unauthorized entry providing peace of mind for car owners Kneron's driving monitor system utilizes AI to enhance driver safety by monitoring and analyzing real-time driver behavior It detects signs of drowsiness distraction, and other risky behaviors providing instant alerts to prevent accidents This innovation is part of Kneron's broader vision to integrate advanced AI solutions into everyday life enhancing both safety and user experience Leveraging their expertise in AI and commitment to pushing technological boundaries Kneron is shaping the future of smart secure, and user-friendly automotive solutions Join the movement towards smarter, safer, and more efficient driving With Kneron's edge AI technology the future of automotive innovation is here, today neuron, empowering the future of mobility Visit us at www.neuron.com to learn more

Follow us on social media to stay up to date with the latest innovations actually our our chip can be scalable that means you can using one chip to put in the pc to build your personal gpt and you can cascade the chip up to like a to build the company's own gpt and with more to support different market need and different computation party so for example like Stanford university there are few professors they don't really want to share their years experience of material to church ubd because they will feel that their know how so they purchase our local machine and just dump his material into the machine so it becomes twenty four hours non stop ta and actually we are also not trying to significant repress the GPU this is a very famous GPU global leader we believe the future of the CPU GPU and NPU should be seamless like in the CPU plus GPU era like right now our PC has a cpu and independent GPU right so we are also helping some GPU player to improve their total performance and their the empower consumption as well so uh this is uh i think top leader global GPU player they invaded Kneron 520 and improved their power consumption and performance up to 30 percent to 25 percent and we also come out with a new hardware which is a dongle with our chip inside so that can plug and play to enable the local AI capability in your end device no matter if it's your PC or a robot or surveillance IP camera And if you feel one dongle is not good enough you can buy two or even three to multiple the AI capability for any device So here I'm just saying that our solution is really good And it's already been applied in many different markets like AIoT, surveillance, smart city, auto, and Azure server And those are the customizations in four of the major verticals for example, in auto with this Toyota And actually, Foxconn's solution also has our chip inside And they also use ours to request some competitors who are also the global giants In IoT, we have Panasonic, Garmin Ofilm is the supplier for Apple as well And surveillance, we have Hanwha Hanwha is the fourth-largest surveillance company, number one in Korea The top three we cannot touch because the top three surveillance companies are mainly in China We are more on the western side But non-China global leader use I think they have more than 50% of their AI camera using their own solution VivoTech, it belongs to Delta Group, is number one in Taiwan And server, we have Chunghwa Telecom and Naver Naver pretty much is the largest one in Korea, pretty much like Korean Google And Quanta Quanta is the largest ODM in server Quanta even have an open price saying that using their own solution improve the AI capability up to four times and significantly reduce the operation cost up to 75% And even Qualcomm partnered with us last year The RB1, RB2 partnered with their own solution So we already have a bunch of different global leader partners like Foxconn Chunghwa Telecom, Delta, etc And we also got the well-known global institution for many honor and recognition For example, EE Times named us the top ten AI chip with Intel and NVIDIA IOT Times named us as the top three AI chip globally with Intel and NVIDIA as well And our paper have been published in Nature And I got two IEEE awards, Darlington Award and IEEE CTE Association Award And even I wrote a bunch of different textbooks The most popular one is why did you use Princeton and most of the top universities globally And our team, a group of us send me back to have been working at global top universities and also top like Qualcomm or Samsung before Yeah So that's pretty much Thank you Before we move on to the next presentation, I would like to take a moment to switch the PowerPoint It's Albert Liu He is the founder I apologize for the repetition, but we aim to be Taiwan's first unicorn company His younger brother is here Could you please raise your hand? He can speak Japanese He is very good at Japanese, so I think communication will be easy Please, after the seminar, feel free to talk to Mr. Albert and his brother They are both Mr. Liu However I hope we can exchange business cards. Now, I would like to invite our second speaker Phison Electronics, Mr. KS Pua AI home computing, generative AI, personal use of AI With that, we will hear from him. Thank you

Good afternoon, I try to speak slowly Today I'm going to bring you about the generative AI which can bring to your office for on-prem use So before that, I'm introducing myself I'm a founder CEO of Phison Phison is a company in Taiwan we are the biggest storage controller and solution provider in the world 24 years old, I started the company by year 2000 So we are around 2 billion revenue So before that, I want to introduce you This is a Phison SSD already tested at the year 2032 in the International Space Station So around 10 months, proven So if everything goes smooth by next year They will have a data center on the moon And the SSD most likely will come in from Phison So if Phison SSD good to use on the lunar, then don't worry on the earth So please buy Phison solutions By year 2020, Phison already on the Mars By the NASA perseverance Months ago, I tried to check from our channel This machine still working on the Mars So means quality proven And the flash itself is coming from Kioxia So good quality So Phison actually we are the biggest storage supplier to the worldwide automotive market We have over 40% share So we cover the top 20 brands of the worldwide automotive So we are good not only in the SSD, but also good in enterprise So we are now working with many American storage box makers Developed yesterday in the Atlanta Super Compute 24 Phison announced 128 terabyte SSD And we partner with few of our American partner to build a high-density drive for the AI applications So Phison, we promote every kind of storage solution From the consumer to embedded to industry, to military, to AI, now to out of space I'm not coming here today to sell SSD but I just want to share you our new invention We Phison deliver the new invention to the market, we named Adaptive So what is this? I think we all agree in our life from today We need to use generative AI We have nowhere to escape So how to use generative AI in your office? As individual, I think like my kid They use a ChatGPT I'm not happy because I invent these things, right? But they still use ChatGPT But in my office, I don't allow my people to use ChatGPT because of privacy So if you want to use in your office, either you go to the cloud Or you build the server in your company, in your server room If you go to the cloud, for example, you put your data in the Google Drive By law, Google cannot read your data So this data still belongs to you But if you open your data for cloud to train to learn, then who owns the knowledge? You or the cloud? So this is debatable So basically, most likely a lot of corporate, they do allow to share the data to the cloud And if you use the cloud for AI, the subscribe fee is unlimited You have to pay forever And the payment is no ceiling Okay, so this just happened in Taiwan two days ago It's the news from the Digital Times KPNG in Taiwan made a study In this period, Taiwan, somebody found Taiwan government people leakage the government information to ChatGPT The same story happened last year In this period, Samsung engineers tried to use ChatGPT to debug their design The source code, they shared the source code to ChatGPT Then they leaked the confidential to the ChatGPT So in Phison, we broke all the ChatGPT We did not allow engineers to share anything to ChatGPT because of privacy So then you have to build your own AI server in your server room And this is expensive H100, H200, A100 is all high cost So if you're not allowed to use cloud, you have to buy this expensive equipment You don't have budget, means you cannot use generative AI But Phison, we invent adaptive We follow this concept Imagine 25 years ago, your email address In Japan, most likely maybe use Yahoo, Rakuten, or Tokomo 25 years ago But today, every company, small company, you can easily buy a small server Buy a software, then you can have your own address in your office You keep data in the office This like today's cloud AI This will be tomorrow's edge AI But one thing is important We need to develop very low price machine Without low price, forget it, impossible So what is adaptive? This is a server or workstation with one piece of GPU card can be NVIDIA, Intel, AMD One piece, two piece, four piece, or eight piece of a GPU card With Phison proprietary adaptive solution This machine can fine-tune the LLM model up to 180 billion Imagine it's 180 billion size of LLM can be fine-tuned Without this machine 180 billion of the LLM fine-tune, you need at least $6 million But with this machine, up to $100K, you can make it So from 100K to 6 million, you can imagine how much price we reduce It's only few percent, 2% to 3% of the original cost So with this machine, we can easily put the AI server into your office Phison will not only provide the hardware platform We also develop a lot of software We have a user interface, we have a training model solution Then we design many microservices such as We use the mail system, the Microsoft Exchange to use email to ask questions We now use a chatbot like LINE, WhatsApp, We Chat Can go through this to ask to our Edge machine So we also develop a lot of application software by ourselves Good to use in our office, today Phison's legal team My firmware team using our Edge AI solution to develop our firmware with our co-pilot Legal use our machine to write the legal document And we use this to manage our internal knowledge So, this we name ProSuite ProSuite is our adaptive user interface It's a window-based, easy to use Any new beginner can learn how to use it within 30 minutes This we name as a guru If we're talking about fine-tuning Everyone, you have a lot of documents How to convert the document to the training set When CHPT start to doing their design They use a human in Kenya and India According to the media, they pay $2 per hour To the people, help them to convert the data But if I provide this machine to my customer You use a human to convert the data Then forget it, this won't happen So, Phison, we develop a model we call aiDAPTIVGuru You input PDF word as a document Then this model will help you automatically translate, convert to training set To reduce your loading in a human So, this is our design and our training And already commercial We also use LINE, it's popular in Taiwan and Japan We use LINE to take, to use our edge machine Develop the PPT to convert the voice to the text To the summarize, to build the e-mail, to translate So, let me make a demo here This PPT generated by a server with an edge GPU card The cost is $100,000 All you can use, you don't need to pay anything to the cloud Your cost is $200,000 plus power electricity fee That's all, okay, and this is powerful for the office use Second, readable e-mail Just answer, you ask a question through the LINE, answer you within 30 seconds We visit to the banker The banker in Taiwan, they want to use the edge AI What they have a pain point, pain point is Every morning 5 a.m., the analyst has to wake up Start to listen YouTube, CNN, CNBC, BCC, all the news Then make a summarize 7.30 a.m. morning meeting, present to the boss But now with this AI solution, he can wake up 6.30

Just pull in the link to edge machine, 2 minutes The machine will summarize everything to him Then he can make his life much better So, if, I think a lot of, if you want to read somebody talk 30 minutes You need to watch 30 minutes video, right With this machine, easily, you don't need to do anything, just plug in If you feel something interest, you can go find the video You don't need to spend that longer time Believe most of you already experience ChatGPT You may say, oh, this ChatGPT can make it Yes, ChatGPT is very powerful But ChatGPT lost 5 billion dollars every year We compare that, okay This machine, again, is a one server with only 8 GPU card We run the 70B Lama 3.1 And we can doing fine tune for our internal own use Same machine, not only for fine tuning, but also doing inference So, imagine your office, you have a few hundred account You take one machine, this machine can serve the whole office use To help you improve the document, to create a document To solve the knowledge management So, this is the solution to help you to bring the alternative AI From cloud to the edge with very limited cost Then you can keep your data in your hand You don't need to share to the cloud So, we did more than 250 POC in Taiwan, Malaysia, America and India So, we already start to using this in the Taiwan's hospital And also in the Taiwan two police stations To helping them to easy to build their own document We keep developing a lot of in-house software Now, we are here in Japan We try to talk to a lot of Japanese system integrator software company Student, if you interest to build your own business on generative AI This is your only solution to bring this to the industry So, this solution, what we can use here We also have Taiwan's patent lawyer Patent lawyer use our equipment to generate the patent To read the patent, to write the patent report And we keep asking the Taiwan government start to use Because government data most likely cannot go to the cloud It's confidential But they don't know how to generate the edge AI in their office So, we are helping them to build the AI solution in their office Helping them to solve the problem So, if you're running your company Definitely your company has an ERP system, right? The ERP can be from SAP, can be from Oracle, can be from local Japanese supplier When you start to use an ERP system You need to customize your solutions How to bring the generative AI into your organization Similar as an ERP So, Phison start to build our own AI ERP by ourselves I established a software team, more than 50 accounts Start to develop this kind of software to solve my internal use Once this ready, we are going to bring this license to other parties To help them improve their efficiency in their office So, we also been request by a lot of factory How to use a generative AI to improve the quality The material management, the production control, blah, blah, blah So, this is what we have this year And we start to develop software This is a big market And this is very new So, this design, we need at least 10 years to digest To make the design mature So, this is very big potential So, conclusion is Use cloud, again, two issues Your subscription fee will be unlimited Here, you pay your tax to Japanese government Then you use cloud, you pay your second tax to the cloud forever But, one more is your data privacy With our edge AI, cost affordable, 100K dollars of the equipment You can keep your data in-house You can customize the solution by yourself And you can create your new business model One more thing When Phison start to promote adaptive to the market Every organization, the chairman, CEO, president They're happy, they want to use But, they tell me they got no engineer know how to use So, when they take Phison's equipment Phison need to send our engineer to stay two weeks to four weeks To help them, to teach them But, I don't have that many people So, I'm develop that new solution Imagine, year 2010 A lot of Asian good students go to US Taking computer science Remember, right? 2010, because of Google Google running so good business Asian students thinking computer science, big data is a trend Today, if your children in the university If they are in the engineering Most likely, they want to go to AI But, unfortunately, in the world Every university no money to buy NVIDIA GPU So expensive So, when professor teach about the AI algorithm by theory Student homework by paper, by writing Because GPU so expensive Even in US, the good GPU Occupied by few senior professor for research For education, the campus no budget, no money So, Phison, we invent We call AI training PC This is a desktop PC With a one GPU card Name is a gaming PC Maybe you buy a one gaming PC for your son, right? Okay, this gaming PC plus Phison adaptive We convert this machine to AI fine-tuning machine Up to 13B LLM model Then the student can use this Very cost-effective machine to practice How to do fine-tuning How to find the best solution How to learn the practical way So, university can put this in the class for student use When student, they like it, they go home You can buy one for them Help them to improve their AI practical knowledge For example, if you want your children go to learn piano You have to buy one piano in your home Without that, I don't believe she or he can play piano But from today, every parent will need their son Wish their son, their daughter go to learn AI But AI GPU is so expensive So, this affordable machine can help in education Then, after they've been educated, they go to the market Either they can create their own startup Build their own solution, their new business Or they can go to the corporate as AI engineers To help the office to solve the AI problem So, Phison provides two solutions One solution is for enterprise use to improve efficiency One is for education to train the student This AI training PC, very popular, not popular The Indian government is crazy because price is so low In Malaysia, government start to approve this Now Taiwan, I donate this to my university I donate 40 sets The class will start by December And this class by daytime Professor, actually the first class, 15 hours Will lead my engineers My engineers teach students at the same time tech professor After that professor can teach students in daytime And teach engineers in evening time To public This can easily help the society to gain more AI practical talent So, this is, I want to bring this to Japan To help university improve AI practical solution And also bring the edge AI to the corporate So, if you're interested about Phison solution You can come to us, we can help to serve you Yesterday, I visit to one GPU expert in Japan They already start to do the POC with our adaptive And they believe Phison adaptive is only solution Go to the edge corporate use for generative LLM And Phison encourage a lot of software company Students to build their own startup To develop their application software Go to this business to help the human life better So, if you're interested, you can contact us through this barcode So, I have five minutes because I've been asked If you have any question, I can answer you, if you have Anyone got question? Yes? Okay Okay, I got still five minutes Advertisement Okay, let me share with you one story How Phison start this solution Last year, April, April, remember? 2023, April CGPT just start popular, right? Remember, right? Phison, we have many engineers, very young Very good, very smart But they never focus to Phison's business When the data mining good, they go to learn data mining I pay, I pay salary They learn data mining When CGPT going popular, they start to play ChatGPT So, one day by April last year, one of team leader came to me He's a Lin, Lin San, came to me He said, boss, Phison is very powerful company in the world We are very good in storage We need to use AI for future development I said, okay, what can I do? He asked $2 million from me $2 million to buy three DGX I said, why? Then he start explain to me Why it's a llama, why it's a 70B, blah, blah, blah, blah $2 million My answer is, no money Last year, business was bad, I said, no budget So, no budget So he went back, he start to think GPU, HBM When you train the model, you need a lot of HBM But HBM plus GPU is so expensive So he think, how about he use just only one GPU One video DRAM is a 4090 RTX With SSD, we build SSD SSD is a memory Can we use SSD to replace HBM? Answer, of course, no Then they start to study, study After two months, they come to me Boss, we put Phison SSD, customized SSD with middleware It's work, it's working It can run 70B But very slow, very slow When he report to me, I said, good Now tell me how much you want I give you money, I give equipment, I give you engineers So, then I start build a big team After six months, adaptive solution proven Now going to the market Why we create this? Because we need to use If I need to use, means everyone need to use So, this solution, again, let me repeat for the generative AI Cloud is a good solution But you have to pay forever privacy edge AI, generative AI, adaptive is the only solution No other solution So, we are welcome Japanese market Just come to build ecozystem together Not only good to your business, but also good to your country Thank you Yes, thank you very much I apologize for the delay, but I will be your host today My name is Yoshimura from the Taipei Computer Association, Tokyo office We haven't set aside a special Q&A time today If you have any questions, comments, or feedback I have my business cards at the reception, so please take one I have my email address, so I would appreciate it if you could reach out via email I understand that it can be difficult to raise your hand and ask questions in situations like this Feel free to send any questions or comments after the seminar I wonder if everything is okay, just a little It seems like there's a delay, has anyone from Faison arrived? Could you please raise your hand and stand up? You're from Japan, right? If there are any representatives here, feel free to ask any questions later Today, I want to take plenty of time for business card exchanges, so please do Mr. KS Also, there are representatives from Japan, so I hope you can exchange business cards Well then, it seems we are ready, so this will be the last presentation of the day The theme is application cases of edge AI, presented by an international technology journalist Just the other day, I published a book about NVIDIA in August Yesterday, I heard that there has been a reprint I believe some of you may already have it in your hands However, I think it's probably the most detailed book about NVIDIA in Japan Now, I would like to invite Mr. Kenji Tsuda to give a lecture

Please hold on for a moment Thank you for your patience Now, let's discuss the application examples of edge AI He will talk about the theme of NVIDIA's strategy Alright, thank you in advance As introduced, my name is Tsuda I recently published a book last month It's about NVIDIA This covers topics like AI and semiconductors It's aimed at investors who don't know much about AI or semiconductors They hear that NVIDIA is an amazing investment There's a lot of buzz around it People are asking what exactly NVIDIA is I've received many requests about this The publisher asked me to promote the book I wrote it As for today's agenda, it's in this format It's about how AI neural networks are growing Earlier, it felt like AI is just the norm I was introduced to that, but I want to discuss something more basic For example, NVIDIA is actually a very different company When you look into it, it's honestly the opposite of Japanese companies It's a company that feels completely opposite That company has a very flat organization, while in Japan, it's completely vertical There are regular employees, section chiefs, department heads, and deputy directors There are department heads, executives, and even the president It's that kind of organization, but this is just the president There's also a flat structure, which I'll introduce later I find it very interesting Also, Japan is somewhat lagging behind in AI Japan is behind in AI software, but this actually opens up opportunities The reason it's expanding is that AI is just getting started In short, as the two of you mentioned it's just begun, so Japan can still catch up In fact, it's a field where there are opportunities to become a leader So, I would like to talk about that To put it simply, AI will continue to grow, and that's all there is to it AI will grow It will definitely grow This year, the Nobel Prize in Physics was awarded to these two They are Dr. Hopfield and Dr. Hinton, a Canadian and a professor from Toronto University in the U.S As for what each of them did, Dr. Hinton worked on this Think of a neural network as individual neurons Knerons, which are nerve cells There are these nerve cells, and they are all interconnected The output from these neurons is sent to the next layer Send from this layer to the next layer Then send to this layer, and finally output This is the basic concept of a neural network And since it's basic, it was modeled by Dr. Hopfield And this is what earned him the Nobel Prize in Physics Basically, each of these neurons is more like multi-input, single-output It's a kind of transparent circuit, and if you write it as a transparent circuit, it looks like this What this is calculating is data times weight, data times weight, data times weight In other words, this is performing the Akahane operation Mathematically speaking, we are performing Akabane operations. In other words We are doing calculations like x1×y1×x2×y2×x3×y3×… And that results in one answer Now, if we assume this is either 1 or 0, one of those will come out So, moving on to the next layer of neurons When we go to the next layer of neurons, this data is either 1 or 0 When calculated as 1 or 0, we clearly know which one it will be, so we actually apply weights here We will explain by applying weights We perform multiplication with the weights Data multiplied by weights This will always result in 0 So, multiplying anything by zero will always result in zero That's why modern GPUs are actually calculating those operations seriously But that's a bit too power-hungry, so let's skip those We know that if the data is zero, it will definitely result in zero It's about how to perform operations knowing that the input here is definitely zero By processing those kinds of things, we can create something that consumes less power That's the gist of it Now, why did Professor Hinton receive the Nobel Prize? It's because he enabled the recognition of images like cats and dogs And when a different image is input, it truly This is about the data that was used for training It can determine whether it's really a cat or a dog That's image recognition, but the recognition algorithms have actually been This is actually from Professor Hinton, known as AlexNet Other models, for example, if it's a cat, have whiskers here and eyes there They have ears, and they model things like the distance and spacing between the eyes Even if you work hard with a model-based approach, the misrecognition rate the rate of incorrect recognition is about 25% to 30% That was considered normal However, Dr. Hinton didn't care about that Anyway, with the neural network model I mentioned earlier I actually tried it with this neural network, and the error rate dropped by 10% That's amazing! After that, companies like Baidu Google and Microsoft got involved as well Now, the error rate is in single digits This error rate has significantly improved the recognition rate And actually, what I used at that time was NVIDIA's GPU and CUDA, software for parallel processing I created the neural network using the combination of CUDA and NVIDIA's GPU So when I tried it, for some reason, the error rate dropped significantly This is the award reel And this AI kept spreading more and more Ten years later, in 2022, generative AI emerged In other words, what Dr. Hinton did was in 2012

So, ten years after 2012, generative AI came out AI is suddenly breaking through again And that's the current state of things And NVIDIA, as a frontrunner, honestly has a tremendous amount of power Their power consumption is extremely high It's high, but they're also working hard on their software, not just the GPUs For example, this NVIDIA Clara is a medical AI There are many different types of medical AI What they ultimately aim for is personalized medicine In current medicine, the discussion about whether a drug works or not is based on overall averages Some people respond to the drug while others do not, but it's generally based on average values Drugs are developed based on this, but each person's body structure is different Genetic structures differ, so we should first analyze that Then, by analyzing the genes, we can also develop new drugs for these specific genes AI for drug development, this one is for genetics and analysis And this is an AI for natural language processing, but there are various AIs like this Even when we say medical AI, there are many different types This is an AI for image analysis, which distinguishes whether it's cancer or not And this is an AI for medical devices And this is an AI for surgeries, but various AIs like this are needed The reason genetic analysis is necessary is that in personalized medicine, your genes are like this Then, it will be possible in the future to say, this medication will work for you, while this one won't So, instead of taking the medication prescribed by our doctors First, we do genetic testing to clarify whether this medication will work or not before taking it This will definitely lead to effective treatments in the future That's why NVIDIA is developing medical AI They are also developing software and libraries for AI Now, this is When you attend an NVIDIA event, you will definitely see this It's a video that is quite interesting Actually I am a visionary, I am a helper, I am a transformer, and so on These are the kinds of things they talk about For example, saying I am a visionary means the invisible universe, like black holes It visualizes various things, like incomprehensible galaxies Being a visionary means to visualize It helps visualize, or even weather The clouds of a typhoon, for instance, over the Japanese archipelago or Taiwan It creates simulations of the whirlpool when a typhoon approaches It simulates And it visualizes that properly This is, oh, excuse me This is what being a visionary is all about That's why it's visionary; it makes things visible Another point is that this is a completely different AI for each Now, when I say I am a helper this is really for people with disabilities, for the visually impaired, who wear goggles It tells them, 'There’s something to your right where you are walking now.' It also says, 'There’s a rock to your left,' and similar things In other words, the AI sees and tells visually impaired people what it is This way, it helps that person, so I say I am a helper It means I am a supporter, someone who helps Then there's another point When I say I am a transformer, it actually refers to nuclear fusion We are trying to achieve nuclear fusion Nuclear fusion is the energy of dreams In places like Japan, where there is no oil or anything else, all energy is imported Almost everything is imported Except for renewable energy, everything has to be imported In contrast, we can create it from water Basically, it comes from water, but we decompose water molecules and apply plasma to break them down We create it at incredibly high temperatures, in the thousands or millions of degrees, using plasma This process is similar to how the sun works If we can create a state similar to that of the sun Nuclear fusion exists because it could generate infinite energy If we have nuclear fusion, humans could use energy infinitely To create this, we need to achieve temperatures of millions of degrees And when comparing the gravity of the sun and Earth, oh, not Japan the Earth's gravity is about 6 to 7 times weaker The sun i

2025-01-05 18:06

Other news