Webinar “How is AI-powered technology changing the smart city?” , November 12, 2024
[Music] Em-bed has never been so exciting. It's life-saving [Music] [Music] It's life changing. [Music] It's dream making. And it all starts with this chip. A small and mighty computer, the size of a credit card.
With embedded chips and processors, enabling our passion and curiosity to solve anticipation of a problem, mitigate risks and create solutions that improve business and life. The potential that lies behind executing dreams is so tremendous , f we fail to realize them the responsibility lies solely on us. So dream away. Be great. Enclustra. Em-bed with us. Good afternoon and welcome to Today's Vision Systems Design webinar
titled "How is AI-Powered Technology Changing the Smart City", sponsored by Enclustra. I'm Linda Wilson, Editor-in-Chief of Vision Systems Design, which is part of Endeavor Business Media. To begin let me explain how you can participate in today's presentation. First if you have any technical difficulties during today's session, please submit it to the questions window and our technical experts will help you out. We recommend disabling any pop-up blocking software or extensions in your browser as these can cause issues with the webinar player. Additionally, we welcome your questions during today's event. We will answer as many questions as possible during the Q&A session that
will follow the main presentation. But please feel free to send in your questions at any time. To do so, simply type your question into the question window on the side of your screen and hit the submit button. Also, please be aware that today's session is being recorded and will be available on the Vision Systems Design website within the next 24 hours. You'll be notified by email when the
archive is available. Now I'd like to introduce our speakers for today's discussion. First we have Gael Paul who is VP of innovation and Enclustra, leader in FPGA based Systems-on-Module technology. He leads a multinational team focused on developing cutting-edge hardware, software, RF, mechanical and thermal solutions. With 25 years of experience in semiconductors electronic design,
automation and cloud technologies, Gael is a recognized expert in product architecture chip design and field experience. Next we have David Wyatt with 30 years of experience in Automation. David is a certified AIA Advanced Vision professional and a charter member of the Machine Vision Association of the Society of Manufacturing Engineers. He also is founder of Automation Doctor. Welcome, gentlemen. I'll start with our first question. Smart cities have
been discussed since the 1990s, taking shape in the 2010s and evolving rapidly ever since to help ground us and our viewers. Can you walk us through how a smart city operates today with AI and what key components are driving this Evolution? Gael, let's start with you. Yes thank you, well welcome everyone to this webinar, thank you for joining. I'm very happy to be with you Linda and David for this session. So what is a smart city? Well, it's a number of functions and I will give you just a few of them: surveillance systems for real-time video processing and rapid threat detection it can be traffic management to optimize the flow of vehicles and which use congestion, environmental monitoring, gas or noise for example, smart lightning, adaptive lighting to adapt to the conditions on the city and the day, energy, waste management is also very interesting how you can optimize waste collection; water management, smart parking public transport, health care and even all the way down to citizen engagement. Interestingly, Linda, a smart city pretty much
operates like a high-end industrial facility with a multitude of sensors, multiple systems, many of them are critical in terms of flows of people or information and as well as security and safety. David, how about you? Well, thank you, Linda, Vision System Design and Enclustra and Gael for inviting me along and it's a privilege to be here. Gael really hit all the high points there so I think that he's exactly right on when he talks about what the smart city is envisioned to be. You know years ago we used to think that it was flying cars and and robots in every in your apartments helping you clean and that's really not what what it's all about. Smart cities want to enhance safety, they want to enhance sustainability the quality of life and the residential experience the experience of the actual residents of their city and they have to find a way to avoid some risks such as data privacy. There's critical infrastructure security that they have to be aware of and that adds challenge to the governments and public bodies that that try to govern and contain those problems over the next five years. I'm expecting
AI generative AI to impact cities through integration of digital government services, smart transportation and interactive digital twins where you have a city that you have model as a digital twin and things that you want to do can be tried in in the Cyber sphere as as opposed to maybe making some mistakes and building some things that don't belong in certain places. Without quality in mind, digital government could also risk have risk replicating or aerating some existing inequalities so we must be ensuring that our Technologies and our Innovations are deployed ethically and inclusively okay well you've touched on some of this already David but let me ask both of you what do you see as the biggest Tech challenges in Urban Development like handling massive data insuring cyber security and scaling up for growing cities. Geal, do you want to touch on this? Sure, well the what we've we've described a number of use cases right and what is interesting in those cases is they actually involve sensing a lot of different stuff comprehending the world in a way that wasn't accessible to us and by the way then processing that understanding especially with AI so the first challenge is the the diversity of sensing for comprehension of the world obviously vision camera based Vision enhance with AI is the basis of most of the systems we now have the ability to make a lot of sense from Vision based cameras thanks to to AI but that's not it there's multitude of other sensors that are very useful I could site image sensors even all the way to Thermal and infrared spacial sensors, lighters, radars, GPS, physical sensors you know pressure sensors initial sensors light sensors audio sensors I will actually talk about that or even chemical sensors. And let me give you an example. I think it's the city of Copenhagen that actually enhanced the water distribution system to with a leakage deduction and that's quite hard if you have buried pipes in the ground how do you detect leakage and one of the solution found was audio processing. It turns out that if you listen to the sound of water turbulent sound within a pipe it can be detected when there's a leakage because that turbulences will be different. I don't believe my brain can figure that out but AI can make difference between a leaked and a non-leaked pipe thanks to audio processing. So a key challenge here is having the
ability to aggregate tons of data coming from a multitude of sensors in real time. Most of the systems need to respond in real time so that's the very first Challenge and maybe David you want to tell us a little more about your perspective on that very first challenge. Thank you, Gael, yes, you're exactly right, the biggest challenge is going to be to put in the data infrastructure because as you point out there are lots of sensors, there's lots of data, you have to have infrastructure to connect all those things and they're not all going to connect with Wi-Fi, because although 5G is barreling in and and we have a lot of bandwidth, it's going to get eaten pretty quickly so we want to be careful not to pollute the airwaves, because Wi-Fi is just radio, you know, so there's going to be a lot of infrastructure that's going to have to have, fiber you're going to have to have some some hard wiring, and you know FPGAs are perfectly suited to the job, to reduce the data down to schemes that AI and generative AI can actually work with FPGA have been doing it since the 1980s and and I think they're kind of like the the sister behind the behind the other sister you know you don't see her a lot but she's there and she's she's she's providing the support that is needed for the CPUs and the GPUs. Many cities are focused on transportation and they're looking for ways to reduce barriers to Civic PR precipitate I'm sorry participation promote transparency of that data and improve access. We are going to see a lot of digital government services and smart Transportation, then we're going to find that through our interactive digital twins and and those digital twins are not going to be able to function unless we have the data stream coming in and as Gael points out, there are lidar, there's radar there are so many different things that we have to look at, that that that is going to be our technical challenge and what I talked about before was more of an ethical challenge. This is the technical challenge side is getting that data reduced and getting that that infrastructure for the data stream okay. And in fact, Linda, let me let me you know augment the challenges. So we
talked about the very first challenge which is the this aggregation of sensing data usually called Sensor Fusion but there's actually five additional challenges right the second one would be that the fast space of changing models, AI models, tend to change very quickly or security threats tend to change very quickly so the ability to quickly adapt at the hardware and the software level to the fast pace of changes. The third challenge is real time which really goes for very low latency systems in many of situations you know if you do traffic management or many of those you need very very low latency response time to efficiently manage the system the force challenge will be low power as you are you know putting down thousands, tens of thousands, hundreds of thousands of sensors and devices they have to be low power otherwise we would you know we would consume more energy that we would save to the smartness of our cities. The fifth challenge will be security, especially at the device level those are exposed, so we need build-in hardware based security. And last is the product life cycles as you deploy the systems throughout the city, you won't be able to change them every six months or every year so you need products which have a very long life cycles typically seven to 10 years so these are the six challenges that I could identify in the Smart City development. Okay thanks, well let's dive into how Tech Solutions like AI powered FPGAs are helping cities tackle major urban challenges? How are they making a difference? David, do you want to start us off with this one? Sure, so maybe echoing this but many cities are focused on the transportation related use cases and as Gael points out those are often times camera related, although there are a lot of other sensors involved there also they're looking for optimization, they're looking for intelligent intersections and multimodal transportation Network improvements. Some cities recently and some of the reports I have read have taken a step back from large scale deployments in favor of building their connectivity infrastructure first which as Gael points out, all of the sensors have to be, the data has to be collected and reduced so that they can actually use the FPGAs and the generative AI to make decisions pointing out though that the FPGAs are often at the device level which I think is an outstanding place to put them. But they want
to ensure the longevity of their initiatives and so a city will focus on topics like sustainability and enhancements and then work backwards to determine those technologies that can help them achieve those outcomes. Many deployments begin small scale in a block to block area and then they determine that that works and and like anything in the industrial world you pilot it first and I think that's where we are we're in the pilot stage in a lot of cases. I think we made some big steps early and now we're piloting a lot of the new things and and this can grow and this is sustainable. The FPGA can be rewired with software. It is replicating an ASIC and so it is basically programmable hardware. Thanks, David and yes in fact, Linda, let me dive a little more in that but first let us back up a little bit right there are essentially three classes of compute devices that can be used. CPUs, GPUs and FPGAs. And let me go through them because FPGAs aren't
the answer to every problem. In fact the the the the simplest and most efficient well easiest ways to do things would be a CPU. It's a very versatile device, it does sequential processing and it's the simplest to program and the ubiquitous and if the job can be done with a simple CPU this is what everybody should do. However, CPUs tend to have high latency, they're very inefficient machines so the power consumption for the task is not that good and so they usually fall short on the number of activities. So the next level up would be GPUs. Right now you're into Data parallelism. You know single instruction multiple data model of compute highly performance moderate latency very very high power extremely powerful but high power consumption and there you need a next level of programming typically Cuda for NVIDIA , GPUs and they deliver great performance but not usually at the device level. And then come FPGAs. FPGAs are massively parallel devices millions of operations simultaneously, concurrent hardware. This is low level devices extremely high performance. This is
spatial compute at its best and they provide the best latency as well the lowest latency as well as the lowest power consumption per test. They're extremely customizable and you can do very very complex processing however there's no free lunch they're much more difficult to program than GPUS, which are themselves much more difficult to program than CPUs. They require expert level engineering skills to do that so they provide a very very high value than at the cost of a higher development cost initially. So FPGAs shine when you need low latency, low power and combining in
fact the sensor fusion which is the sensing part of the the task with the thinking part which is going to be the AI inferencing and the reacting part which is going to be - hey, this is the control action on actors and and others that I will do to respond to what I've measured. The second element to mention is that smart cities like everything else is a layered architecture and there are really three levels in which this is being resolved. The first level is the device level. Think about cameras sensors distributed throughout the city and here we're talking about hundreds of thousands of devices throughout scattered and that's going to give you a space where you have millisecond time. reaction time where you need low power obviously you have such
a high volumes you need low cost of devices. The next level up is going to be the edge. Think about a control room that's going to be aggregating video fits from tens of cameras let's say no more radius of maybe a mile and there you have a bigger power budget, a lower volume and moderate cost budget. And then finally you've got the cloud level and by the way the edge is going to give you second time reaction and then the cloud is you know maybe tens or hundreds of miles away from the city it has minutes time reaction extremely high compute power and cost is less of an issue here and so it's this overall combination. Now FPGAs can actually play a role in all those three. There are very low cost efficient FPGAs for devices, a medium range FPGAs for edge and high-end FPGAs for the client. They will sit together with high end CPUs and high-end GPUs from Nvidia and the like.
Nice, hey, Gael, quick question. So historically I mean back years ago one of the kiles heels of the FPGAs was is typically when you put it into a system, you're going to be increasing the number of power supplies in your system. You have one for the memory, you have one for the CPA CPU I'm sorry, one for the GPU and then you have one for the FPGA. Next thing you know you got five or six different power supplies in your system. What have you guys done to address
that? I'm sure it's not still the case. Well you're right on the money David, you know as I said FPGAs are no free lunch. They're they're very complex to put down and there's an old ecosystem. You mentioned power but you can mention clocking and memory and connectivity. And what Enclustra, you know the game of Enclustra is actually to resolve that for customers by designing a manufacturing system on modules. This is the single board computer approach it's a complete subsystem delivered as a very very small PCB the smallest of our PCB is 30 x 30 millimeter and it can be up to 60 by 80 millimeters where the complete solution around the FPGAs is already designed manufactured ready to go and it really makes FPGAs is like the software defined platform which is exactly what you get from Nvidia or the CPUs today so it puts them on par relatively speaking to that by solving all this plumbing electrical issues, power clocking storage memory, connectivity once and for all so that you know people listening to us today can immediately get to their unique differentiating value, your secret sauce, add the stuff that you're really good at, that is going to make your product better and different than competition. Don't worry about power clocking and all we've got it covered for you sounds like you did a great job we're proud of that you know it's it's maybe less glamorous than doing you know the processing intelligence into it but everyone has a purpose right and we're very proud that we've been serving over 2,000 customers and dealing great value for the last 20 years allowing people to reduce risk in providing faster time to market. It's interesting looking to the Future how do you think AI-powered FPGAs
could help cities manage the massive amounts of data they're dealing with, especially when keeping things efficient and secure? David. Well luckily technology is keeping up with AI Solutions. The market's driving it and it's a$ 31 billion dollar market. The next few years it's amazing but you know just as an example right now I can buy a 4 terabyte memory stick for under a 100 bucks and there was a day when we couldn't get more than 256k in the same box you know so to see what we are doing now with the technology advancements and to see what what companies such as Enclustra is doing to see what what Nvidia is doing with their SoC modules and the price points that are that are happening and FPGA can in fact live with the Nvidia Jetson platform and there there's several examples of that and I think Enclustra is one of them but the point is that we are able to bring forth the technology changes that we need to keep up so part I'm not puning into the future but I'm confident the future will keep up as it always has that's never going to be the problem and I think we need to be relying on our youngest scientists to continue the our the trend of data reduction those guys are the ones who are going to be coming up with the next set of algorithms if you look at just what H265 has done to compression of video everybody uses it on their Netflix but nobody really appreciates it and if you look at that compression and the people who came up with that algorithm and it was based on the old run length encoding but they did that in a color in a color space and I'm just really amazed when I read the data on how that came about from you know H264 for instance and then back to the MPI carriers and then of course there's there's new connection technology like GMSL, the gigabit media serial link which doesn't go as far as CoaXPress and it certainly is not going to go as far as fiber but you know in a local situation you can have you can have one piece of CoaX that carries both audio and video into into an EG type carrier in software and and and these are the things that are coming forward right now we're at GMSL 3 now and and the bandwidths are getting bigger and the speeds are getting faster which means we can have either more cameras or we can have bigger cameras to get that detail that I know Gael is really looking for. So you know we need the kids to pick up the Raspberry Pi all right the Raspberry Pi is not a FPGA. It's a it's it's an arm computer but people need to start learning those languages they need to start learning Linux they need to start learning TensorFlow they need to start learning PyTorch, they need to start learning Python and not to mention C++ and visual studio and and until they get that experience they're going to be handicapped and so I'm really looking in the future for for people that are going to come along and be able to do that with less and less hardware and hardware like the SoCs that that Gael points out and hardware that's a lot cheaper. yeah in fact David that's so great you know I've been around the FPGA industry for 20 years and the evolution has been amazing um I would call it maybe you know the fpg 3.0 and we we keep calling
them FPGAs but they're much more than that they're actually complete platforms so they started being with the DNA of FPGAs was you know lots of iOS and and and Hardware processing and today you have terabytes per second of iio capacity in FPGAs then next wave the second wave was s so so now for many years now in those platforms you have multicore arm subsystem with real time um um Processing Unit application Processing Unit multiple arm cores I think the latest and greatest have up to 18 different arm cores in this complete memory subsystem and very Advanced connectivity and the third wave is where you add AI processing into the mix so that's now that the priority of the three all together into one chip and there you've got um AMD with the versal Technologies who actually really have three pieces of silicon glued together for the so for the AI and for the programmable logic and IO capabilities and you also have a friends from Altera who have infused AI they actually completely fused the AI capacity with the programmable logic and both approaches are extremely good and and high performance so view them as platforms then can do it all for you and they've been designed to be software definable um all these guys from MD Altera and other they give you tool flow where you can compile train your train model that you probably train on GPU somewhere in the cloud and you can compile and optimize them for the Target silicon and have them running at high performance so they natively going to read your your high torch model and others tensor flow models and deliver great performance likewise you have all the compile chain and all the software tools to do your s SOC and obviously the the the tool for for doing the programmable logic so those platforms are phenomenal because they combine all three parts into one chip uh with lots of um software environment and by the way an ecosystem of solution providers around it uh to help you not reinvent the wheel and get on quickly with your design so you're running a multinational team and it's mindboggling the the the knowledge that you have to assimilate into one spot to to be able to play in all those Arenas at the same time indeed David it's very right the second aspect to your question Linda was security which is really really important I we briefly touched on it previously if you think especially the device level those devices are scattered throughout the city in your example they're not physically producted in many cases they can be accessed relatively easily a camera as you can imagine or even if you have straight cabinets they're not that hard these are not vs or you know for Vols so you need the highest security elements and FPGAs provide Hardware based security they're really what soal the root of trust the hardare root of trust and the programmable logic part is essentially a blackbox there's there's nothing you can do or virtual you can do to figure out what's going on can you hike or spy on the PG part it's virtually extremely hard to do that so they provide best-in-class security uh which is essential especially at the the device level combined with the flexibility of software for AI and S so okay um next question as cities push for better Energy Efficiency stronger cyber security which we've touched on and resilience against electral magnetic interference how could FPGAs make that happen dle do you want to start us off yes well thank yeah absolutely so I already touched on security but let me reace that so Hardware root of trust and Hardware security is best in class and this is what FPGAs natively provide and they provide all sorts of different protection mechanisms against side attacks and cloning and and whatnot there's been a a tremendous evolution of security FPGAs we also touch on safety and that allows me to bounce on functional safety so functional safety is really defined as how do I ensure that my system keeps on running in presence of errors or even attacks you know and that is best implemented when you essentially replicate the logic and use either a voting system or majority system so you have multiple instan of the same stuff running concurrently and only if there's agreement or majority then you can continue and this is what fpgs can lo being massively parallel you can duplicate triplicate the processing that you need and ensure that it keeps on running in regardless what's happening electromagnetic um events um malicious attacks from hackers and whatnot functional safety is best in class with FPGAs and in fact in cluster we have uh quite unique expertise in functional safety and we will announce a functional safety module um in the subsequent the near future and these are unique capabilities that it's much harder to obtain with software or CPU or GPU based compute then the the energy element is that the performance for what because this is the metric that you're interested this is the performance per what is again Best in Class for FPGAs there's there's it's a very efficient machine and if you look at frames per second per watt for example for video inferencing uh or tops for what FPGAs again provide Best in Class GPUs from Nvidia will give you the absolute best performance but at a cost of a lot of power they can be tremendously useful in the cloud much less in in the you know Urban scale development at the device level David I'm sure you are you you you you agree with that and you have more to say yes sir yes sir so so I started my my industry experience in the power industry in Texas and we used to run 138 KB across across the fields and the Plains and and um we thought that was always going to be enough uh and and it's not it's still not so the energy is so important you know the FPGAs live at the device level in some cases many cases and they're more energy efficient there and oftentimes they can they can live on solar powerered uh Banks and I actually have a solar powered camera system doing Intruder detection at my mother's house 500 miles away and um I I I I connect with it with G4 and um writing algorithms to determine whether or not the garage door is is just opened or open later or trying to figure out she's she's at the age where we need to be careful uh with what's going on there um in in in terms of um FPGA they also live of course at the edge which is not necessarily at the device level but that's where Energy Efficiency is so important the the Nvidia uh CPU GPU um that whole AI generative thing is is based upon you know redundant uh statistical analysis and so they're going to run over and over and over and over and over and over and over again until you get the right engine that you can deploy that's not going to take that much and a mind-boggled again I don't use that word too much is how much energy is being used by our government in terms of just doing AI work and why do they need that much and what are they doing I'd love to find out but they won't tell me I haven't really asked but I don't think they will um so talking about protection against uh BMI so being a ham radio operator for many years and having worked in the communications industry um it's important to have a local enclosure an IP67 type enclosure uh that that is going to Shield Emi not only from outside world but from the from the inside getting out so we really don't want all of these devices out there oscillating in 452 megahertz whatever they might be doing and causing other Emi problems so it's important to find a a way to get them into uh containers and and one of the ways is to you know pop them of course but that's not always practical but tamper proof enclosures are important so there's an entire another Market there entire another Revenue stream that someone's going to take advantage of the FPGA is the DNS level uh I think they they provide some of the best firewalls you can have um I've seen a lot of the the DNS traffic uh stopped um because of Bad actors and and uh denial of service attacks and FPGAs are involved and I think it's a great thing okay kle was there anything else you wanted to add on this question well yes but briefly we touched on this the the reconfigurability obviously we're all used to software flexibility right we can always you know fix or improve software and download it over there in a secure manner right what FPGAs is bringing on top of that is Hardware reconfigurability the ability to expand the hardware perimeter maybe to add new types of sensors without changing the processing logic right maybe add think especially when you have so many propretary formats in sensors technology this is not a well standardized industry so the ability to reconfigure the hardware is extremely important and I would say even more so for security um elements right because it really allows you to potentially significantly upgrade an example would be um moving from traditional encryption technology to past Quantum encryption Technologies can be done as easily as a software upgrade what looks like a software upgrade but how upgrading FPGAs and yet maintaining super high speed in real-time processing a CPU typically would not be able to evolve into postquantum era uh without changing the hardware FPGAs can do that which is really awesome so you can always man maintain and stay up above the level that you need you know barely up what you need to be in order to provide the the best rade of between safety security and cost to the mix so i' say so wonderful devices they don't let you down right they remain your best friend for many years very true okay okay um so let's uh let's now move on to um let me ask you this what are some practical First Steps for someone looking to use FPGAs and AI David thank you well um stakeholders the very first thing you always have to do in any Endeavor is to talk to your stakeholders and and those aren't necessarily just the people who are writing the checks those are the people who are going to receive the services and you need to make sure that those Services can be delivered in a in a way that makes a lot of sense um through automation doctor and some of the work that we've done we we soled hundreds of Machine Vision systems and actually we're we're pleased to find out we're going to do a VC webinar on that uh somewhere in January I think but I I prefer to start with the interfaces you need to find out what the customer requirements are you make a traceability matrix and you take those those requirements uh whether it be a resolution or a frame rate or uh you know the the distances that you have to to to telephoto and and then once you have the interfaces once you have the requirements then you figure out what your interface has to be to the device itself right because obviously to get get data into the FPGA or the microcontroller or whatever you're using you're going to have to have an interface that meets the standards that you're trying to achieve um you know with the new CSI ports that are on all these cameras now um you know CSI can only go 18 inches it's not going to do anything for you I mean it's great for getting that data into that module but at that point you know um you're going to have to talk to the to the other side of the edge you're going to have to talk to the to to the mid edge with with something else and so start with the camera interfaces and and there's almost no room for multicolor cables I'm sorry multiconductor cables this means you're going to have to look at fiber coax Express copper ethernet uh GNA have to find ways to to to get the data to where it needs to go and nothing happens if you can't do that so that's the first spaghetti thread I like to pull uh Wi-Fi is always possible but traffic can a problem plus it may not be well known but medical facilities they do not want your Wi-Fi devices in there sucking down their bandwidth they don't want all those images in the medical area going around uh and and you know when you have nine or 10 cameras that are running in a medical device they would they really want the other devices to run that's more important um like the you know the the uh the the blood filter machine it's more important so I think the best way to start is to sum it all up is I like to find a moderate proven living design and I know and cluster has them uh with a repository a software repository that has all the project files that are required and that is ready to be built today in a specific integrated development environment I I like Visual Studio code uh people can use whatever they want uh and that's not even touching on the software side of the FPGAs but at the end of the day you really have to have something that's ready to go you don't want to spend six to eight months uh dealing with uh libraries that don't work or trying to get this thing to to to uh interface with that thing everything needs to work when when you first grab grab a hold of it now there there is a room for that R&D and you should put certainly 30 to 40% more time into your project than you think you're going to need because there are going to be there are going to be things that need to be done uh what say you Mr G yes well I'm with you there you know first step would be decide if you what you need you do I need a CPU or GPU and FPGA and and and and and David you really nail it is the amount of data the connectivity requirements that you need is going to be your number one decision criteria uh what's the poity of of data sensing data I need and how fast do I need to process it the second is going to be what's my power budget if if you are in a low power budget again if FPGAs are probably going to be your best friends the thir is going to be in know latency of mentioned that if I really need low latency again FPGAs are going to be your best friends um if if none of that apply as I said go for CPU that's the easiest and the simplest or a GPU if you need more compute once you you've defined that you need an FPGA the second layer of decision would be what classes of FPGAs do I need and I see really three big you know clusters we have the high-end guys right the the AMDs and Alteras of the world with those completely integrated platforms that will deliver the best performance and Leading Edge technology we have a friends from microchip who will deliver you the absolute lowest performance FPGA right so if you're in the lowest power budget then go look for Microchip Solutions they're really Best in Class and if you are in the realm of ultra low cost then we have a friends from Latis semiconductors and epinex just to name these two which they were extremely cost efficient FPGAs so you're well served with this industry um depending on your needs the next step would be def gits all FPGA vendors you know offer cheap def GS which allow you to get started in minutes for just a few hundred bucks you can get a def G lots of reference design lots of examples and get started once you get to the next level well system and module sorry for the Shameless plug but I would certainly recommend that you look at that and and clusterize the world leader and we have those modules and we have development kits as well where those those modules will plug into it they they're going to be a little more expensive than the the very affordable def kits from the FPGA vendors but they will give you higher performance you can actually develop a complete application of these right and lastly the ecosystem there's a ton of ecosystem Partners to help you with AI models with software tool flows um libraries for Hardware functions or software functions uh very rich ecosystem so do not reinvent the wheel right get you know just focus on what you uniquely do different from your competition Ju Just as a added added point on your dev kit and that's a very great point I meant to touch on but on your dev kit make sure that your that your dev kit touches all the points of your traceability Matrix I have uh some horror stories that I can tell I'm not naming names but if you get something that runs in a particular language like Python and and you want something else that runs in C++ but you and you want to put them together they're not going to work in the same dev kit uh in this particular case it does but then again you have to figure out all of the translations yourself it could be a mess just make sure your dev kit does touches make sure it can touch all the points in your Trans in in your traceability Matrix that's all I'm saying that's a good one and Linda maybe I I should also add because you know if FPGAs have a repetition to be much harder than they are but I would say don't be afraid if you you know if you get the guts to to to get on with that challenge what your product will be some so much better than competition right anybody can can program CPUs that's not that hard but much fewer people can do it at the hardware level with those FP platforms so go for it because you're going to be rewarded immensely by being able to provide much better much more efficient Solutions than your competitors and we there's a bunch of people like us to help you along the way we even have Design Services to help you getting started if you so desire okay um any anything else to add David oh um just as a as a final question I you know I I I I'm really excited I've seen this industry since the early 80s and I'm just so excited I think by uh embracing the forward-looking approaches uh to to our City's problems of sustainability and equity and resilience and I I think was really important is the expandability that that g talked about and being able to to reprogram the hard RK that you have uh I think the the AI cities could usher in a new golden era of prosperity and happiness yes I could not agree more you know to me Linda it's it's really immen immensely be rewarding to see that FPGAs can actually contribute to a better world this is all what we're trying to do and I'm not just talking about you know getting faster to my parking lot or getting faster to the office let me give you an example with Waste Management right in smart cities um a number of cities like Copenhagen and Madrid studed to put sensors on the the buried containers or even the TOs the wheed beans and what the figured with that is collection was wrong in 75% of cases in 50% of cases collection was too late and we've all been there overflowing garbage in the streets which is really not good and in 25% of the cases collection was too early half full containers once they started to put down sensors which can be weight sensors b or visual sensors for a cardbo container then they were able to reduce by two by 50% The Collection um of those garbages and that turns out to be more cost efficient but also it turns out it reduces pollution those big trucks that collect waste they actually are quite polluting so all of a sudden smart technology like FPGAs can make it a better world right we have a better environment in our cities without the rats and the hygiene problems of overflowing garbage and in the same time you know we've reduced our footprint on the planet and for me this is very exciting to be part of this I've been around FPS for so long and it's part you know my kids are very much into that into being environmental friendly and respectful to a planet and and and FPGAs can greatly help towards that so I just wanted to share my excitement um in in those uh emerging use cases for FPGAs well thank you I definitely hear um excitement from both of you um I'd like to move on now to some of the questions that we have from our audience um a few of you as I just mentioned have already submitted questions so we're going to jump right in but as a reminder if you would like to submit a question type your question into the question window on the side of your screen and then hit the submit button okay let's see um okay here's a question how common are applications where you use infer inferencing of small llns in s so's like versal or zinc and then um it says speaking of high energy consumption for modern AI this would be a low power chatbot so G you want to start with this one yes well the so ai's been now ubiquitous in FPGA for quite some time already and it it starts with um convolutional neural network CNN and FPGA is extremely efficient at that um llm so gen is also showing up in FPGA right although this is an emerging use case es Genna is predominantly used in data centers Ty application more for chatbot and I'm not sure that FPGAs are necessarily the best platform for that um in in smart cities there's less today a use case for gener I think it's coming up and in fact both versal gen two for example support um llm typ Transformers um as well as CNN on their devices likewise the infused AI capability from Altera allows um developers to actually build the AI processing that is you know tuned fine-tuned to what the task that you want to do whether you want to do a CNN or whether you want to do Transformer base you can do it all in an fptas David I'm in agreement so the large language models and he's trying to do a chat box it looks like so he can have a box that will allow him to do translations in a many Lang languages into a large city where somebody can come up and put in their token or wave their hand or their phone or whatever and and they could allow them to to converse with somebody else right now I walk around with the the Google phone in my hand you know my iPhone in my hand with a Google translator I try to go back and forth but I love the idea of of a chat box but the large language models uh I'm going to defer uh to gu because I'm more in the convolutional neural network side right now and and haven't done a whole lot with those although they are high on my list but they're not Vision so much and so uh being the vision guys you know we we haven't really scratched that surface as deeply as we should but we are working on a a project um with a medical device company and we want to have uh the languages uh we want to have the conversation um interpreted and it's going to require different languages so I think I'm on my way in that direction maybe G can help me out what I might add is what we're seeing as the trend is multimodal AI combining CNN and llm that is Transformers um FPGAs allow that I'm also thinking of our friends at Hyo and SEMA which also have dedicated silicon for that which can be greatly combined with FPGAs to deliver those those unique AI capabilities so it's really happening and the hardware configurability of FPGAs made them uniquely suited for for AI the the the the pace of changes is extremely high right there's a a new small Revolution every other week or every other months right I think between two YOLO versions it was as short as a month or two um so the ability to evolve my my my AI compute engine in Hardware is unique if I use dedicated silicon from Nvidia what I have is what I have and I can't change it with FPGAs I can change my number format my quantization I can change my pruning the size of my layers how many layers I have it's so adjustable well I love it okay um let's see here's another question have you got examples of smart City projects running and do you see an Roi we've talked about a few projects I have that I have an examp multiple example um Oslo in in Europe has installed um smart lightning system I think it's over 65,000 LEDs what they've seen is 70% reduction in energy consumption 70 70 which is enormous um um and adaptive here will means that the lightning will be depending on what's needed the time of the day the lightning the natural lightning conditions and even I believe only lighting up when there is something that needs to if I'm working down the street it will only create a hollow of light around me and it's not going to light up the part of the street where there is absolutely nobody so here this is seven % reduction in the um Waste Management I think it's University of Idaho in the US who um was able to reduce by 50% so cut in half the cost to do the the waste collection at the University and there's many other examples where there is indeed a very very tangible significant error in cost of apparitions in the environmental impact energy consumption it all goes in the same direction together so there's an investment in the tech which results simultaneously in better operating conditions safer operating conditions lower energy and lower or improved impact on our planet so not just money by the way all you know indirect forms of uh of improvement it's interesting David do you have anything to add the only thing I would add to what G said is that and maybe embellish a little bit is that that return on investment is going to be so hard to measure because how do you measure you know satisfaction of your citizenry how do you measure uh what you didn't do all right so if if you are looking at traffic patterns and that traffic pattern made a shift uh on a particular day because of an event or on a particular month because of the weather and and now we don't have as many wrecks now we don't have as many things we have things that didn't happen and it's so difficult then to put a finger on on on what that saved us but I'm very very glad to see that it's happening that way you're so right David traffic jams I think have an enormous cost uh to to S Society so reducing that alone is going to improve productivity which is a very tangible R probably in in billions per year throughout the planet and and you've spoken before about about the climate um enhancement that you can do by rerouting people so they don't sit and traffic and burn fuel although a lot of our cars aren't doing that now uh you still if it's a if it's an electric car you still got charge it and that still has a fossil footprint somewhere so it's it's just important to make sure that we don't waste energy yeah absolutely okay let's see here I think we've got time for one more um is the versal FPGA the device of choice for AI based applications I'm going to take that one so this is one of them as I mentioned today on the AI side if you look at Leading Edge solution you've got AMD Verso as well as Altera Agilex both of those pandos give you exceptional AI capabilities with some differences both of them exceptional both of them as a complete platform that can do wonders to you with the iOS the AI inferencing and the S SOC control system all integrated so for high-end AI I would look for that now you also can do extremely cost efficient AI in microchip devices and Fenix and latis devices um um lower performance but U extremely low power so depending on what you need a great example that ltis has demonstrated is the ability for a systems to infer if the human operator is actually watching the device and only then interact so for example the screens will be turned off until you actually look at the screen and then it will turn on and that saves energy but it's also great security features for industrial applications so there's more than one device of choice here um and that's what's great with FPGA is a lot of offerings from different vendors very cool yes yeah had a question um for GU about the the software package that that is used on on their systems and how that's leveraged into um uh into the common user well thank you and I touched already on it but I can recap that so it's essentially you have three tool flow when you're dealing with FPGA remember there's three compute engines right one is going to be your programmable logic tool flow to do all the Sensor Fusion and the parallel data processing and that's the traditional tool flow second is going to be your s so tool flow which is really just a CPU to FL because that's what it is it's an embedded server in a sense into ufp platform where you can compile on those multicore uh subsystem the third is going to be your AI tolow and again all these vendors they give you great AI to flow the starting point is everybody wants to be able to inest trained model that were not designed for FPGA so the starting point is you take any train mod that you have or you can buy on the shelf or get on the shelf and you know hugging face for example and then the tool flow from those vendors fpg vendors will allow you to very quickly optim imize down that Network those layers in order to get great performance out of the Silicon they're very easy to use and in a few number of iterations you'll be able to quantify use the number format that works out choose the layer tabs configure your inferencing Eng so that quickly you get really high performance and power efficiency you know remember the key metric is performance per what so there's a complete tool flow available to all of you guys here to get started today with AI on FPGA platform how does vhdl and verog uh interface into that scenario well David the HDL verog this is the first part I mentioned this is the Sensor Fusion part this is traditionally how you would do that part um AI does not need any htl coding at all today when you're doing you want to do AI for you know Altera Agilex or AMD versal you do not need any hardware knowledge you know it's all ready to go for you and you really just need a model and and a few optimizations of that model through the to flow to get to get started I should also mention by the way that even for HDL there are hls high level synthesis tools today that exist that allow you to design your Sensor Fusion part in C++ and that is a great way to actually get a working system really quickly and potentially down the road you can further optimize down cost reduce that Sensor Fusion part using HDL or know vhl very log so it's a very very much soft defined you know experience that you have today thank you for clearing that up for me I'm all about the fast start you know no need to a sh expert okay well on that note I'm afraid we've run out of time for those questions we didn't have time to answer we'll get back to you via email um this concludes today's presentation on behalf of vision systems design I'd like to thank our speakers for their insightful discussions today and cluster for sponsoring today's webinar and of course all of you for joining have a great rest of your day thank you thank you everyone have a great day goodbye friends [Music] embed has never been so exciting it's life saving [Music] [Applause] [Music] it's life [Music] changing it's dream making [Music] and it all starts with this chip a small and mighty computer the size of a credit card with embedded chips and processors enabling our passion and curiosity to solve anticipation of a problem mitigate risks and create solutions that improve business and life the potential that lies behind executing dreams is so tremendous if we fail to realize them the responsibility lies solely on us so dream away. Be great. Enclustra. Em-bed with us.
2024-12-09 05:03
