[Music] thank you for joining us today as we discuss the evolution of sensors new sense in technology and testing and certification challenges based in manufacturers that want to leverage this new technology now as with most areas of technology sensors have evolved over time the earliest forms of sensors were able to turn a physical event into a mechanical movement with the use of electricity electrical sensors start to appear on the scene and these sensors had the ability to turn a physical event into an electrical current and these sensors paved the way some basic remote control and automation then we start to see miniaturization which highly increased the accuracy and reliability of sensors and today we're seeing a new step in sensing technology we now see the communication within the senses and the internet of things promises fully automated processes based on a complete digitization of physical assets but what does this mean for for companies and manufacturers and businesses that want to leverage this technology well from a process perspective filling up a factory plant with sensors to optimize production is a great idea but the implementation has big practical issues for example the number of sensors needed to to digitize an entire factory could easily be in the thousands and the cost to power up and maintain all of those sensors can be restrictive for many companies and they're also technology issues for example most sensors can't work reliably or can't be installed in harsh environments like the ones present in some industrial processes for example high or cryogenic temperatures aggressive chemicals or salt water or high pressure or high radiation and also these sensors can be too expensive they can be unreliable or they may not even exist at all this is where an emerging technology called virtual sensors can help but what are virtual senses and how can they solve some of the issues present with physical senses well joining us today to discuss these topics are dr ed gesota senior director of new product technology at csa group anna zalisco technology consultant at csa group rajinder jakku technical oversight specialist of signal sensing and trevor pereira product manager for home and commercial products at csa group edgar why don't you start by by telling us a little bit more about the historical background of virtual sensors and how they're used yes so virtual sensors or soft sensors or some people know them are inferential models that take input from sources like physical sensors process information or historical data to estimate an unknown variable just to give you an example of what this dm soft sensor virtual sensor is um we have the weight scale recently a few months ago i bought these smart weight skill that i use in my bathroom i put my my feet on and giving my weight not too happy what i see every day but well it is what it is so uh this weight scale besides my my weight it gives me my fat content my muscle content the bone content of my body and also even my hydration levels at that point now if you see the scale it does not have one different sensor for each of those values or parameters instead it only has two electrodes where i put my feet on and those electrodes connect to a bioelectric impedance sensor that the scale combines with my weight and my height to estimates all those other values so this example shows that virtual sensors usually reside in embedded systems that most of the time also have physical sensors the the example also shows a really good characteristic of this uh technology is that it can reduce the number of physical sensors that you need to find new parameters this is key for the evolution of industry 4.0 or in the industrial internet of things just to give you an example to explain this concept through an example you mentioned in the introduction about the cost of digitizing a whole factory plant so let's go to that example and imagine that we have to do that job for a 5 000 square meter plant and let's say that and we do our math and our estimation and we come to the conclusion that we need around five sensors per square meter uh to do the whole work the whole project if you do the calculations five sensors per square meter in a 5 000 uh square meter plant you will need around 25 000 sensors those are a lot of sensors now you you were right on mentioning the cost because uh just to change the batteries of those devices when they and the battery dries out or uh to replace the broken sensors will require a full-time employee for that there is also another point let's assume that uh each of those devices will require around three watts of power which is normal given that those devices may connect via wireless to a computer if we do the calculations three watts per sensor um five sensors a square meter and uh 5000 square meters you will need around 75 kilowatts of power for those sensors to work that is a lot of power you will need to get a hole a high power transformer just to power those sensors which will add to the cost of the project so now we see why just using physical sensors it's not a good idea from an application standpoint now let's bring uh virtual sensors to the to this example so from the five physical sensors we're using let's assume that we can estimate two of those parameters using the other three sensors so instead of using five per square meter we are going to need square meter that will reduce the number of sensors from 25 000 to 15 000. that is still a high number still a lot of sensors but is less than 25 000.
now let's assume that we can take we can model the warehouse or where the factory is we can create a model of that and put it inside the virtual sensor which means that instead of needing to put three sensors every square meter we can now do it every 10 square meter that will reduce the number of sensors from 15 000 to 1500 which is way more mana manageable for for a company so you see how by introducing virtual sensors we were able to reduce the number of physical sensors from uh 25 000 at the start of the project to 1500 this is the power of this technology now this is not a new technology if you do if we do a literature research we can find papers back from the 1990s even before of that but there is a reason why and now this technology is getting a lot of traction there was a survey done in 2009 by the journal of chemical engineering of japan it was actually it was a paper released by that journal and in in that paper um in the survey they mentioned two major reasons why companies were not adopting this technology that fast one was the cost of acquiring the data from the physical devices from the physical sensors to the computer that was analyzing the data and the other roadblock was at the cost of creating the inferential models from scratch it was too expensive to do that those two issues have been solved with all the communication technologies that have iot brought that the internet of things have brought and the inferential models are now easy to make thanks to all those machine learning tools that are available for developers today and we can see how um this is creating a lot of research and development including virtual sensors i did a quick search in in a database recently and find and found several papers that were really interesting that included virtual sensors it's going to mention two of them one was the use of a virtual sensor to estimate the state of the charge of the battery of an electric car so instead of using a current sensor which is a the standard uh way to measure the charge um what they did was to estimate the state of the battery by um using the voltage uh using the amount of time the driver uh step up on the gas pedal well in this case it's not the gas because it's an electric car but it would be like they speed up a pedal or something like that so they use those two values plus another couple of more values to estimate the charge and it enough to be really really close to what the physical sensors was was providing another example is a collision sensor for rods so instead of using proximity uh sensors to detect when the robot hits an obstacle uh in this paper they were showing how um they could use the current that the engines of the robot pull when the robot hit he hit an obstacle so when that happened just the force that the robot needs to make pull more current for the motors and they could correlate that to the presence of an object so just those two examples show how um people how are now using this technology to uh find ways an application uh to like provide redundancy to physical sensors or in some cases even replace physical sensors thank you edgar sensors have certainly been around for a long long time and there have certainly been some major breakthroughs and because sensors have been around for a long time they've been able to collect a lot of data anna can you tell us about how different mathematical models can use this data in sensors and how these models are built sure tara so you know as both you and edgar mentioned sensors have been with us since the industrial revolution so we've had a lot of time to collect data associated to physical sensors so that would be the environment in which that physical sensor existed the attributes of that specific sensor and finally kind of the output that the sensor provided us over time now having this information we can study the patterns of those physical sensors or the different traits that provided us with the outputs that they gave us and once we understand those trends and behaviors to model to recognize those in the real world as well so that's where we kind of get into building a mathematical model really in our environment there's two forms or two groupings of mathematical models the first is regression models which really focus on producing in numeric output and classification models which focus on creating kind of smaller groupings to really answer your question and now you typically will pick one or the other depending on really what problem you're trying to solve or what the sensor is trying to get at so and as edgar mentioned kind of in his example we saw that there was we were trying to have a virtual sensor that measured the electrical charge of a battery when we think about that that's the direct question we're asking is what charge is are we currently at for the battery so that would provide us with a percentage or a number of you know this is how much we have remaining so to get that we could put in attributes such as what kind of car are we looking at how far has the car traveled and so forth and so forth and we would like to get kind of a final number with which we could evaluate what to do so a regression model would work really well in this case and it you could kind of test a variety of the regression models that are out there to see which one really fits your data set the best now secondly in the classification example another piece that edgar mentioned was having a wide number of set physical sensors around your factory even though you can kind of replace them with one small virtual one so in this case what a few of the papers i have read have done has actually they took in all the data from those sensors and actually grouped them together to say you know which ones actually are telling us similar information about that area or that region so it allows us to then group that information and train virtual sensors and classification models on specific data and if a switch is very simple so with the charge example really what we might be going to is do i need to stop for gas right now the answer there is yes or no so that's more of a grouping classification mechanism as well which the virtual sensor kind of informs us of a simple choice is to say yes or no answer or something even broader as well so in either of those cases it's really the problem that drives the decision between the two models but i hope that gets to answering your question as to how mathematical models can help in assisting virtual sensors become more powerful in our times thank you anna for discussing you know how those key models can help support virtual sensors given the versatility of sensors regender what certification considerations should be taken into account when using a sensor as an input in the control system of a device oh yes i'll i'll go through with the the existing certification model we have and how the same model we can use it for the virtual senses so i just go over like the safety control that we commonly use in the household appliances so where the sensors are monitoring the temperature humidity and the outputs are driving like motors alarms and light so in the normal mode the controller will operate within the functional requirement of the product but under the normal situation it will act to mitigate hazards to provide safety to persons and property by disabling the outputs so sometimes the safety is achieved by safety devices in safety critical circuits such as thermal fuel bi-metal thermostats so for example if the thermal fuse is installed in the motor winding and in case of the winding temperature exceeding the limit the thermal fuse will open to protect the motor winding against any fire hazards so the reliability of the safety devices are verified by reviewing the construction of the product performing the task some of the tests are listed in the product standard and we do the test under the extreme operating conditions so in case of bi-metal thermostat it is checked if the thermostat will open when the temperature will exceed the set point and also we do the endurance cycles on the thermostat and the full load and the number of cycles are calculated based on the product cycle of the product life cycle of the product so with with the advancement in electronics and embedded software the the sensing is performed by discrete sensors and the controller will process the input signal to control the output devices under both normal and abnormal situations so in case of motor the the winding temperature can be monitored by like such as npc type of thermo sensors as an input or controller the the controller will open the motor relay when it detects the temperature is exceeding the maximum limit of the winding temperature so sensor will act as input signal to the controller and by itself it will not open the motor winding circuit and the circuit control is done by the controller so when the sensors are used with electronics and embedded software to achieve safety then the reliability of the product to act safely in abnormal situation become difficult to achieve there are a number of factors for that so such as detecting the input signal correctly so the reliability of electronics component due to aging and environmental factor so hazards and risk associated with the software implementation also electronic circuits are sensitive to main bond returns and radiated disturbances so in this case during the certification we verify the safety of the electron circuit by performing electrical testing software evaluation and emc testing so in in the in the software evaluation the systematic fault in the control uh which are using uh safety software and electronics are provided by implementing remodel of the software life cycle and the random faults are dealt with the component fmea techniques so therefore the safety system can be designed in such a way the systematic errors are avoided and end of faults shall be dealt with by proper system configuration so now the virtual sensors are are being implemented the pro in the product safety as integra introduced by edgar and anna and csm able to evaluate virtual sensor implements the safety critical application under attestation program on the case-by-case basis so a test station is a non-certification service and is intended to provide manufacturer with evidence that product has been evaluated by cst csa to the requirements indicated on the attestation marking and the in the testation report i'm back to you tara thank you agenda trevor i would love to hear your thoughts on this too is there anything that you'd like to add i would echo the same criteria that rajinder mentioned when we are certifying pay attention to the safety software analysis functional safety emc because the virtual sensors are heavy on electronics you know getting input from different models and trying to process that or try to simulate a normal sensor having said that i would like to add that we have we are inputting data from known sensors into the mathematical model and expect the virtual sensors to operate close to the real sensors which definitely has a lifetime degradation and interference so uh we will have to subject the virtual sensor uh to to the same environmental conditions when we are testing but that alone is not enough because it's more heavily on the electronic side it's because the reason i'm saying that is the virtual sensors do would should not degrade like electrochemical sensor or a metal oxide sensor and so forth so their life is much much greater uh from that aspect but from their interference electronics you have to do due diligence to look at those and as project said attestation uh would be the the the obvious path to start testing evaluating these sensors and the next step obviously would be a national standard like a can standard or in the us and ansi standard that lead to more recognition throughout the world you know so uh as i wanted to point out that census that we have been testing this so far they have been for detection of carbon monoxide flammable vapor sensors and combustible gas sensors uh those are the ones that are most widely used right now and there are established standards and test methods to evaluate that so the virtual sensors are looking to those types of standards and requirements either to meet them or exceed them in their performance because you are after all from those the lifetime data the behavior and environmental conditions inputting all that like you said in the regression model or classification model and then coming up with something that will closely resemble the real sensor performance specifications so uh that's what i had uh additional to what reggie rajinder said yeah thank you regina and trevor for for providing insights into the critical importance of sensors and also you know the challenges of testing and certifying products with control systems that rely on the use of virtual sensors edgar in light of these challenges it does seem particularly important for manufacturers to find a solution to assess the accuracy and the reliability of virtual sensors can you tell us a little bit more about that yes para so um this technology virtual sensors is getting a lot of traction lately uh manufacturers are not only using this technology to reduce cost by reducing the amount of physical sensors they need in their product or systems they are also using them to expand their product roadmap to design uh new products or systems to put them in areas where a physical sensors cannot be used like areas with pretty high temperature or low temperature as you mentioned in the introduction or areas where there is a high amount of radiation that could destroy any physical sensor uh so that's the reason why these manufacturers are using it more and more and more often um trevor mentioned a good point too these sensors don't degrade as physical sensors can like if you have electrochemical sensors they could get poisoned or they could degrade with time virtual sensors don't have that issue and you can see when we do a research online to find patents we can find several patents that are related to virtual sensors in recent years just mentioned two of them one was for a combustion um system a combustion control system that uses virtual sensors to assess the temperature in the chamber in the furnace as well as gases inside the chamber i also found another patent from other company uh very closely related to this one that was the use of a virtual sensor to retrofit a border control system so companies are already uh thinking on using this technology not just to improve the performance of their systems but also to provide redundancy to sensors used for the safety critical parts of their systems and this is where accuracy and reliability is key it starts to to gain more attention because as a retinder and trevor mentioned and any sensing technology that is uh that is used in these systems have to uh prove that it meets the same level of performance that exist in sensors so a manufacturers have to come to to a way to assess uh the accuracy and reliability of these uh virtual sensors now this is not an easy task uh because even though the virtual sensors don't degrade with time as physical sensors do there is a challenge here because these um pieces of software these inferential models are expected to improve as time goes because it's suspected that they will use the same data they are acquiring to improve the model that means that the test that we do today might not provide the same output of the test that we can do a month from now which can be challenging if we start thinking on ways to certify this technology so but to start we have to first think on a way to assess the accuracy and reliability of virtual sensors thank you edgar i think that it's clear that you know it's very important that we need a a robust solution that can test for accuracy and reliability um anna can you tell us about processes data sets and models that can be used to assess the accuracy and reliability of virtual sensors sure tara um so an important thing here is to really go back to the model building process right as you build your model there are a few key stages in which you want to really take a step back and say is this an accurate step that we're doing is how is this actually going to influence my model in the end of the day because ultimately as trevor mentioned you know we want to build a virtual sensor that is just as accurate as a physical sensor if not more accurate and the first step of that is back to the data right we want data that has been you know cleaned it doesn't have many blanks or those blanks have been filled in we want data that is an accurate representation of that realistic situation so if there are some you know weird off occurrences you know do we want to include those occurrences what is it about these specific occurrences that and are they accurate of how a physical sensor may actually perform in the real world those are questions you kind of want to ask and finally you want to make sure that your data sets pretty diverse so if you're building a one specific model to answer a key problem you know do we have a way to diversify that set to maybe use two or three physical sensors that are trying to calculate the same thing just to see the data across physical sensors not only across different time frames as well so that's kind of the first question looking at a diverse set with good data that's clean and hopefully you have a lot of it and now the second piece is actually where we're starting to integrate all that data into a model and so typically this would work with evaluating the types of information you're putting in some information might be you know duplicate information of another piece you've already put in some might not actually even benefit the model at all so like in the energy and battery situation you know if i included what i had for lunch probably not gonna help the model so you know i got that data but is it helpful so you know you gotta you gotta give that to the model and let it actually or even other algorithms and they'll actually inform you of whether this is a valuable feature and you should put it in the model or whether this is a feature that maybe you should drop and leave for another occasion so to make sure that that process is consistent and that you haven't even exposed some of your testing data to your model is very important because once you've done that you know your model already knows what you're about to show it and it's going to give you a 90 accuracy and you're like great model but actually you biased it so that's an important part of this process as well now finally if you've actually you know got the data put it in the model we know we've got the great features the final piece is to actually evaluate your model and most models the regression of classification have their own form of metrics that are already built into the system and that a lot of data scientists and machine learning engineers out there use and kind of the community has confirmed are good metrics to use so that can be anything that's an aoc curve an f1 score sometimes an r2 score so each of these is dependent on the type of category that you pick along with even the type of model you're using and sometimes it's also highly encouraged to train some different models in the same category to see you know which one of these three models is the best model to actually represent this data set so you can easily cross compare within categories the challenge you want to cross compare across categories that can be a little trickier but still possible so there's a lot of opportunity to kind of step in and say to help with the standardized piece and say you know this is what your data should cover or should look like this is kind of what we were thinking your process in terms of building your model should look like and finally okay let's actually evaluate what your model score is so that's in the full picture kind of how one can test the accuracy and really be uh validate whether a model is a reliable representation of a physical sensor or physical world even thank you for your questions it does indeed thank you anna for that very detailed explanation edgar there are clearly challenges and opportunities for using virtual sensors can you explain what csa group can offer to manufacturers using virtual sensors in critical systems [Music] well tara there is not one quest one answer for that question as reginder mentioned just a few minutes ago we will have to go on a case-by-case basis here in the current standards that apply to uh products that are using sensors for control systems like the csa 0.8 1998 and the csa ul and iec 6730-1 and xh they don't consider the use of um systems like virtual sensors these are too complex for what these standards are considering it will be hard for us to provide a certification to these standards however there are options that can be helpful for a manufacturer like evaluation or sort level of attestation like a radiant dimension but we will have to go on a case-by-case basis it will depend on the specific product and the way it is using uh the virtual sensor now one thing to keep in mind to remember is that this is not the first time that csa is dealing with new technologies with emerging technologies we have done it in the past several times and we have found solutions uh for it and so we will do it again this panel is just an example of how we are proactively looking for ways to approach this emerging technology virtual sensors thank you edgar i would like to also thank all of our panelists edgar anna rajinder and trevor for such a robust and interesting conversation today it's so exciting to see how much knowledge of this topic we have in-house at csa group as we're looking into possible solutions for new technology and to our audience please do keep an eye out on our digital channels for more information about you know emerging technologies and new solutions and if you're interested in learning a little bit more about this particular solution you can reach out to us via the the contact us form this concludes today's session once again thank you so much to our panelists and thank you for joining [Music]
2021-02-22