Leveraging Adaptive Learning & Emerging Technologies to Train the Hands-On Learner

Show video

as Jay mentioned Eric joined our team just about  a year ago as his anniversary is coming up and uh   excited about that but more excited for him to  share with you some of the really cool things   that he's been working on uh before we get to  the fun stuff though just a little bit about   who we are at uh repairify and Aztec um Aztec  is the leader in electronic vehicle diagnostics   and calibrations and you'll see a couple  of our devices up on the screen there uh   electronic vehicle diagnostics are if your car is  involved in a collision there's so many cameras   and Radars and lidars and other elements  in that vehicle that require work when a   vehicle is being repaired our folks do that  work so we connect virtually into tools our   the the Gen 2 device that one sitting on the blue  cabinet in the upper right hand corner is actually   just a communication tool and it allows us to  connect to any car anywhere in the world that   has internet connection and attach an oem original  equipment manufacturer scan tool to that vehicle   um but Aztec is just a part of the puzzle  the bigger part of the puzzle is repairify   and repairify is the umbrella organization  if you will that Aztec is under but you'll   see there are also many other uh Brands so we  focus on diagnostics calibrations we also do   remote services and uh we're starting to get  into the the world of Automotive intelligence   um we conduct a large number of scans each and  every day and so we have data sets that lots and   lots of people are very interested in so we within  repairify and Aztec uh again you can see some of   the other brands under there special shout out  to atg the automotive training group they deliver   seminars and webinars focused primarily on the  mechanical side to technicians around the country   but for repairify itself uh and Aztec we have over  400 technicians that are certified and Eric will   talk more about it what ASE and icar are um over  22 000 customers we have eight patents that uh   that keep our lights on so to speak and in 2020  we repaired over 1.2 million vehicles and so far   we have uh in 2021 over 6 million uh separate  events in repairing cars that were conducted we   scan about 6 000 cars a day and then we calibrate  about a additional thousand cars a day so uh lots   of folks doing lots of work to keep all this  running um about two years ago we decided as   an organization to really enable our learning side  and start leading as uh as education as one of our   our leading indicators to our customers and to  do so we the organization created the position   of Vice President of Learning and Development  and we hired a gentleman by the name of Chris   Chesney into that role so I report to Chris uh  Chris is uh extremely well known particularly   in the mechanical Automotive space with over 30  years experience and he's done some remarkable   things in those 30 years so he was brought on  intentionally to help us up our game in the   world of learning and leveraging Technology to  achieve that so much of what you're about to see   was brought to the table by Mr Chesney and as a  part of the growth of the training department we   added several people to it including Mr McClellan  as the senior training manager uh as Jay mentioned   a really unique fit because Eric used to be a  car's got a car guy has literally written the   book on the GM Ellis engine written three of  them actually uh published in Hot Rod Magazine   so he he's been a card nerd for many many years  and actually when he first started and my boss   met him my boss said weren't you the guy who did  that that blog of a build out on a car and and so   it turned out that my boss had been a follower  of Eric's many years ago so very much the right   guy for the job so with that I will turn it over  to the legendary or inFAMOUS Mr Eric McClellan thanks Mike I appreciate that um uh if you're not  familiar with the name McClellan it is a Scottish   name and Scottish tend to be storytellers so  uh you're probably going to hear some stories   and you're going to see some kind of threads go  through the presentation but I highly encourage   folks uh to chat with me in the chat box I will  follow it as we go to me the worst question is the   question that never gets asked so please do ask  as we go um even even if I'm going to answer it   later I've already answered it please just ask  the question anyway I want people to walk away   really feeling like they got something out of this  so please uh do ask your question and I'll answer   them uh as soon as possible I might call Harvey  I'm in and out like a ninja you won't even know I   was there let's talk about the agenda for a little  bit um the we're going to do actually an in theme   with this session we like to think of things  a little bit backwards uh here when we do our   trainings and make our materials so we're going  to start actually with the breakout session first   and then we're going to talk about some ducks and  then I'm gonna talk to you about adaptive learning   and then I'm gonna tell you uh basically our  what we've come up with as far as a solution so   the premise of our breakout session started about  six seven months ago I went to a meeting and I I   met a person who did a you probably can see this  in the in the photos of Lego foundation in the   upper left uh presentation by Lego and they are  positioning themselves as a learning company using   the Michael brand and the presentation started  out like this it says uh two plus two everybody   knows the answer is four okay he says that's  memorization that's not learning that's not you   know applying what you've learned that's just pure  memorization he goes but what if we gave you the   answer for first and then we asked the audience  the learning audience Give me the give me your   solution how did you get there so if we give you  the solution first then we get to see your process   we get to see how you think we can see how you  solve Solutions and there's infinite number of   solutions to get to the answer of four so start  with the answer first and then use the audience   and their process and their Journey Through the  to get to the answer of four to teach you not   only about the audience but how you can teach them  differently and then they will actually learn more   in the process so here's my premise and here's  what I'm going to do we're gonna do our breakout   session this is going to be a little different  I know a little uncomfortable and that's okay uh here's here's the premise the premise is how do  you plan on training a group of Hands-On learners   who are traditionally Hands-On  people who learn tactilely   virtually now I'm looking for Creative  answers I love Creative answers   um the most creative answers will get a a whole  bag of Internet Stars uh I I uh I farmed a whole   bunch this morning so don't worry I won't run  out of any but I'm curious to know given the   directive and this is all you've got to  work with you've got Hands-On learners   This Is How They how they operate this is how they  learn how are you going to train them virtually   and I'm gonna let uh Luke and Jay and Jill do  their internet magic to break folks out into their   uh respective groups and again I love Creative  answers the more fun the more wacky uh sometimes   they can be uh you know out there totally cool  there's no wrong answers uh because I'm a big   believer and work should be as fun as humanly  possible because hey work is hard you should   you should have fun so I'll let Luke Jay and  Jill do their stuff and I guess we'll come back alrighty thank you so I got to listen to uh  some of the folks who had some ideas some of   them are really fun some are very practical  some were uh not as practical but uh still   very very fun so I'm hoping somebody may you  know get a couple uh folks I remember I saw   this bag of Internet stars I could give away for  today so I'm wondering uh who had some fun ideas   and uh I heard some people have actually done  this before so I'm curious to hear from others so I can jump in because I had we had a an  interesting discussion so I I started with   it if you're gonna do if we use the example  if it's a vehicle that everybody gets sent a   vehicle so that they can uh be Hands-On right  with the the actual thing that they need to   work with if not that maybe there are centrally  located vehicles that are easy for people to get   to to to do Hands-On if this if that's going to  work for them but my as I was talking we got a   lot of different things around this but the value  of getting feedback on the work that you're doing   um as Hands-On Learners there's often these people  are often good at doing problem solving um trial   and error kinds of of Concepts and so I I also  made the suggestion of let's put a GoPro on their   head so that they can be demonstrating back to the  group or to the instructor to show what they're   doing and get instant feedback on their approach  and their quality of work and just to to ensure   that it's it's happening not only correctly but  in the most efficient manner so how's that Eric I I liked it so much I wrote it down [Laughter]   because I could share it's  it's Molly day here apparently   hi Molly so hi Molly so we talked about that  same thing the GoPro um having like a pre-study   and then sending out some sort of a kit so that  the instructor would be doing you know that the   learner could see what the instructor's doing and  then the instructor can also see what the learner   is doing at the same time um we also had uh just  some kind of dreamy discussions about drones   flying out to the location of this of the learner  to be able to view and provide feedback somehow   from that we talked about VR which we think is  you know seems a little on the dreamy side right   now but it's probably going to be a lot you know a  lot more mainstream within a short period of time and if my group has anything  else to add to that please   do but that was kind of those were  kind of the main things that we the all right anyone else yeah we we discuss similar  things in the one different one we thought about   would be like um you know a library of uh videos  with different aspects of the technician or the   person who's the Hands-On learner with need to  be able to do on the job and that they could be   searched out of a you know a library of videos  by the title or the step maybe you know like a   micro step that they need to do uh to do the job  correctly and then they could use those on the job I have something I'd like to share too one  of the ideas that came out of our group was kind of inspired from the recent movie on a wing  on a Prayer having a hard time hearing you Bridget oh sorry I want to try a different microphone source can  you hear me better now yeah a little bit yeah   a little bit I'll try and talk  louder basically we talked about being able to have maybe a cardboard kit sent to  the home or maybe even have the folks gather some   household items from their house and build some  and uh it kind of was inspired a little bit the   idea from the movie that recently came out on  prime it's called a wing on a wing and a prayer   where they build from Soup cans and cardboard  and scissors they put together something to   simulate you know the land and airport and they  actually called somebody accomplished so anyway   I thought it was a great idea maybe do it low cost  if you have you know some something that you think   and then the group built on like 3D  printers are big out there too right   so maybe maybe that's right but if  you're trying to do something global   peace all right 3D printers uh cool ideas so yeah  I I don't think I've seen that movie before so uh   thank you I appreciate everybody's uh ideas I  actually wrote down some of them because maybe   I'll talk with Mike later about some of these  uh I did kind of give away what we ended up   doing because this was our directive uh well  Mike and I were sent to the directive of hey   we've got thousands and thousands of people  who need training and you you can't really   reach them because they're scattered throughout  the United States and there's only a couple of   us so what do we what do we do and along the  same lines I want to show uh this which is   I know this is a bag of Legos there's uh Lego  excuse me it's not it's singular not plural   uh there's six bricks and we were given the idea  uh you have to build a duck you're only going to   give you six of these bricks and everybody's got  the same thing and you have to build a duck and   we had a room full of people and we basically  came up everybody had different ducks because   we all came to the same solution we all know what  a buck looks like right I say the word duck you   can picture the a duck and how we go about doing  it is different but we all came up with something   that looks like a duck as a matter of fact I still  have my duck it is right here and it reminds me   that just because my duck does not look like those  Ducks does not mean I'm wrong doesn't mean they're   wrong it just means we came to the same solution  in a different way okay so uh I I keep this on my   desk as a reminder to myself that there's often  times different ways of doing it to get to the   exact same solution now I will share with you the  problem that we are faced with is that there are   two yes that is right one two two locations for  automotive collision repair facilities to go to   ones in Racine Wisconsin the other one's in  Chicago that's it so that means that all of   the repair facilities the United States must come  into these two areas take time away from their   busy schedules time away from building cars and  making money to go to this training and often the   trainings are not short they are many days long  and in the Collision space Mondays and Fridays are   the very the busiest because Monday's work when  cars show up and Fridays are typically when cars   go out so uh there are thousands and thousands of  technicians across the United States and Mexico   and Canada that need this sort of training and  there's only two locations so uh to get there   they have to take time off they have to have a  very expensive travel to get there and uh often   these uh these trainings are fairly repetitive  so many technicians who have been in the field   for a very long time probably been doing the same  stuff they're getting trained on 5 10 sometimes   25 or 30 years and they're coming back to train  on something that they do pretty much every day   so that was another kind of wrench haha uh in our  problem so what did we come up with this was uh   handed down uh through uh collaborative work with  Mr Chesney and Mr Willis uh that was uh Mr Chesney   was mentioned it before and what we kind of came  up with was what we call the purify Institute has   these six academies repair If I complete Academy  the adaf academy now we use a lot of acronyms so   if there's an acronym that doesn't make sense feel  free put it in the chat I will respond as best I   can I don't instill in all of them because  there are so many but adafs stands for um adaptive driver assist systems Advanced  sorry it's Advanced Driver assist systems   so these are things like your uh rear  view camera your TPMS sensors uh 360   cameras if you have them so that these  are the things that we're training on   uh we're talking about Diagnostics academies  The icar Alliance Academy which I'll talk   about in just a second some sales and Leadership  Academy and then vocational support so uh we were   tasked with uh the the the amount of material  that you see on the right at the very long uh   set of criteria but basically all these are  all the things that we have to train on and   we have to train uh for people who come through  our facilities on these type of uh materials   so we have a lot of Partners and we also have  to partner with the sanctioning bodies uh which   is uh icar and ASE icar being the Collision  sphere uh kind of training non-profit that uh   basically tries to uh Train everybody as much as  possible and also gives out accreditation so um if   you ever need for for whatever reason you need to  get your car repaired and you see the little icar   sticker sitting on the window or on their website  you know that they have certain credentials that   lend them to being a uh you know either a gold  class or platinum staff status repair facility   ASE looks more like the mechanical side of things  so if you need something fixed with your engine or   you need your brakes worked on generally these are  going to be the people who are trained uh through   ASE so a number of partners that we work with CCC  one CMS map on et cetera atg being the automotive   training group that we recently acquired goes  out and does the trainings and locations that   are much more closer to the technicians themselves  so part of the uh that the directive that we were   given was we need to come up with some locations  for all these people to go to so we have to set   up these facilities that are within a couple  hours drive of virtually any uh mechanic OR   technician in the United States so that was the  other kind of big uh directive that we were given   so through some audience analysis we learned  that the overwhelming majority of our Learners   are coming from the mechanical field so these  people are very Hands-On type of folks they   tend to be mostly male they tend to be older  with what they say as years of experience so   in the upper right you see one of our trainers  from atg showing an electrical board with a with   a real live training class and you see uh Alexa  here on the right using a Target that looks like   a front camera for a vehicle I can't tell you if  that every vehicle is a little bit different I   think I want to say that's the forward Target for  the front facing cameras and then on the bottom   photo you've got some of our typical Learners  that you would see in one of most of our classes   so these are the stereotypical Hands-On type  they look learn best when they have something   in their hands don't necessarily learn great by  just reading things some people are really good   at that I unfortunately am not so uh these are  the types that really like to get their you know   kind of hands dirty and also like to get a sense  of kind of what things look like and feel like   and they can kind of manipulate them and that's  how they do it so they're very experiential type   Learners these people also learn by acquired or  lived knowledge so that's passed on from person   to person uh and uh they you know well you know  I've been doing it this way for 30 years uh you   know this is this is the way to do it I know the  handbook says this this is a lot of something we   get of the handbook of this but this is the real  way to do it so we do get a fair amount of that   too this is also the group that will say things  like time is money okay uh the faster I can do   something the more accurate I can do something I  can do four things at once and make a lot of money   then uh you know that's that's the value that I  bring time is money and this is also the group   that during trainings what they will tell us is  just give me the facts and nothing more right just   just the facts that's all I need so when we do  these trainings we try to keep them as succinct as   possible to the point and not give any extra added  pieces that the person uh may may or may not need   the other problem we run into is many of  you are probably familiar with this the   dunning-kruger effect which is people  who say they know something actually   tend to be very confident that they know  something and then when pressed or tested   turns out they don't actually know as much as  they thought they did so confidence on the left   uh and the the y-axis and then the x-axis is  the top performer so bottom performers on the   left and top performers on the right and their  confidence level so people who tend to do better   actually have a less confident uh attitude and  people who tend to be pretty confident tend to   also not necessarily know what they're doing and  one of the things that we get a lot through our   trainings is uh Eric I've been doing this training  I've been doing this particular you know event   for 20 years I know exactly what I'm doing if you  do it in my sleep I said okay so what we want to   do is we want to be able to test that we also  want to figure out who is legitimately telling   us that they know what they're doing and maybe  weed out those don't so that's uh kind of where   we're getting into this adaptive learning and  I'll explain it more in depth in just a second   so as most people are aware the most effective  type of training is one-on-one mentorship and this   is Bloom's two Sigma problems uh that basically if  we're doing some for some sort of Mastery learning   the fastest and quickest way is one-on-one  mentorship and those people who get that type   of mentorship tend to do much much better and  score much much higher than even those who are   doing a standard Mastery learning course or even  better than the conventional classroom where you   have a teacher up front and then everybody kind of  just listening very one-way type of communication so let's move on to what adaptive learning  is and how it kind of addresses that problem   there's a lot going on in this screen so I'm  gonna I'm gonna direct your attention to a   couple pieces on the screen that I think are  important for for the actual concept itself so adaptive learning is basically the combination  of testing knowledge immediate remediation and   then testing the person's confidence whether they  think they know that information or not so the   very big screen in the front is an example of a  a question or a probe that you may ask and then   you have to rate how confident you are in your  answer if you get it correct and you you say   I knew it or I know it and I know this with 100  certainty you move through the course much faster   if you are less confident and still get it right  that's okay you'll still move on you just won't   move on as quickly and if you get it wrong and  high confidence you're going to get remediated   immediately so many people that I noticed  during our exercise said this exact same thing   immediate feedback and when you have immediate  feedback you tend to learn more effectively   now uh the when we talked about the two  Sigma problem this is the this is a way to   electronically give training to people and  help mimic that one-on-one type mentorship   to a large group of people that may not be  available when for every single trainer I I   certainly can't do one-on-one training with  every Tech that we work with so uh that's   where that remediation comes in a lot so on the  bottom left to see something that self-assessment   um and it got a little bit cut off on this slide  so I apologize but there's a little slider and   it asks you hey uh how confident in this material  you are so if you say I'm super confident you put   yourself on the far right of that graph what it  will do is it will send you racing through this   material and if you get it wrong what it'll  do is it'll remediate you and it'll actually   send you back and and keep doing basically a  similar type activity until you get it right   with a decent level of confidence so we're all  getting through the same solution everybody who   gets through this is getting the same same amount  of information they're just going through it at   different rates because we are showing who is  confident and uh consciously competent and who   may not be so confident and may be overconfident  and might be unconsciously incompetent so we have   to kind of find those people okay and uh with our  trainings that people come in and say Eric I've   done this one procedure for 20 years I said okay  cool we can test you out of this very quickly uh   and as a matter of fact we can get you through  this if you if you can pass this in 10 minutes   or less with a high degree of accuracy we'll  move you on you don't have to keep doing this uh   so those who have struggle with this this takes  longer those who are easier goes much much faster   but a little bit about the hot and the how and  the why of adaptive learning so uh the the why   of it is online delivery method that automatically  adjusts to the each learner so many of you already   kind of got to my answer too uh which is the  ability to remediate somebody right away when   there's a learner who has a bit of a struggle  in a particular area right so uh Joyce great   great points you know brilliant example of uh ID  without adding uh without adding technological   features to The Learning Experience correct  uh so the the technology is is pretty smart   and we'll figure it out as the person goes along  and I've got some graphs to kind of explain this   here in a couple slides coming up so the idea  is that this will mimic and recreate the scale   of that optimal teaching Approach at a one-to-one  mentor level so that's that's the idea behind this   and again got some graphs to kind of back some of  this up so uh you can we have some data analytics   uh that will help us with some of this they use  a little bit of an AI technology it's not fully   AI like chat GPT which I heard some examples uh in  in our breakout session which I thought were great   uh I love out of the box thinking it's fantastic  so uh similar idea maybe not full autonomous AI   just yet in learning uh technology but will help  optimize The Experience based on this uh the it   will help pull up personalized learning and then  it will help save time and effort so if you've got   the technician who comes in and says Eric I know  all this stuff and they go through it and they   they click through and they have a high degree  of confidence and they said man I don't like took   that only took me like 10 minutes exactly you knew  exactly what you were doing we know that you we   you know what you're doing and then we can move  you through this uh track much much faster I'll   tell you a short little story and this will kind  of help things this is what helped conceptualize   it for me I went to pick up my son from school  one day I said hey bud how's it going he's uh   I think he's 11 at the time and he goes oh we had  to take our state test today Dad I said oh okay uh   how did it go he goes I don't know I said what do  you mean you don't know he goes well the test was   really weird he said in the beginning it was super  easy and then he goes I add you know I answered   about 10 questions and then it got super hard like  I didn't know what they were talking about oh okay   what what happened he goes well about 10 questions  later the question's got a little bit easier   and then a little bit harder and then a  little bit easier and then the test was over   and that was it and I said well how did you do it  I don't know I have no clue I said Okay cool so we   did find out he did fairly well uh and that was  an example that the schools are using to figure   out where somebody is so they start really easy  they ramp up the difficulty go oh that's way too   much because they were a little bit difficult less  difficult oh that's too easy and then I start kind   of honing in on that Baseline where that student  is at and the goal of this is not completion   we're not looking for somebody to complete this  that although that is the the end uh you know   effort here the idea is we want them to get  to the same level of competence at the end you are probably very familiar with this  unconscious incompetence from the bottom   left that's the red that's the bad no that's  the we don't want that that's bad uh on the   upper left we've got conscious incompetence so I  know that I don't really know what's going on like   I know I'm I'm I need some education here bottom  right you've got unconscious competence uh which   is you're doing the right things but you don't  really know that you're doing the right thing   and then in the upper right you've got conscious  confidence meaning I know what I'm doing and I   know I'm doing the right thing the goal is to  identify the people who are in the bottom left   so if you look at the black graph in the upper  right you will see we've got somebody who's in   a lot of unconscious incompetence and then move  them through this adaptive learning process to   conscious competence and we can get them there by  doing a number of different things you heard me   say the word probes and I can tell you that this  process is uh once it's done and it's working it's   fantastic it works uh right out of the gate it  is a it is a bit of a challenge to get things   going and it is intentionally difficult so when I  went through this process and I became a learning   engineer for adaptive learning one of the big  pieces that they taught us was you have to make   the questions and answers very very specific more  specific than you're already thinking and you have   to make the answers very very hard because we have  to figure out who really does know their stuff and   who is just really good at test taking so there  are some people who are just really good at that   and we need to weed those folks out so that we  can test the actual material that we're giving earlier I kind of talked about the the uh how you  know how does somebody go through the process and   you have three examples here in order one two and  three notice that they all got to 100 proficient   because they were able to go through it get some  information and then remediate things that weren't   uh working so well so the easiest way to read  this is you've got you know your green dots   and some red dots the Green Dot is the start the  Red Dot is the end and every time you see a loop   that goes around that's when somebody had to go  back and go that and that wasn't that wasn't so   right so let's go read back and do this and then  you will also notice the size of the circles   the size of the circle indicates how much time was  spent on that one particular item if you notice   the person on this has been these are kind of the  data points that you'll get from the uh you know   adaptive learning program that we went through uh  through area nine the uh organization that we work   with so you'll notice the person on the left 88  correct uh very low uh unconscious incompetent and   a little bit cautiously incompetent but that's  okay that's pretty normal you'll notice the   circles are very small which means they send spend  very little time on those items and then we're   able to get through the material very quickly and  notice not a lot of loots not a lot of Loops there   second person took longer about double a little  bit more than uh double the time so got to 100   efficiency only about 50 correct had a lot of  unconscious incompetence and a lot bigger circles   meaning they had to spend a lot more time on that  particular item and then they had to go through   those Loops a few more times meaning they went  through it and it didn't get it right okay gonna   go through it again yeah still not 100 right  and then they went through it a third time got   it right okay great we can move on third person  still 100 efficiency now we're up into 30 minutes   and how can that person get there well they had  to spend a lot of time on a couple items okay   um when uh Molly your question about  proficiency versus efficiency is a great   question that is coming up right now so  traditional efficiency okay so if we're   looking at something and and I love the questions  that like lead me right into the next thing that   because that tells me okay we're going on this  journey together this is great so uh if you're   taking General uh traditional e-learning course  which takes between 45 minutes and an hour and   you you turn that into an Adaptive learning uh  process what you find is that time generally tends   to go in half kind of your average half because  not every bit of information is needed or needs   to be remediated or taught on something because  some people know some of that material so if they   already know it great prove it to me and let's  move on so what we have found is that the time per   learner is almost halved so we're talking between  a 45 minute and an hour session now that person's   down to between 20 and 35 minutes and they  only really focus on the stuff that they don't   know or they have a high degree of unconscious  incompetence really trying to Target those folks a lot of data that you're going to get from this  this is a very heavily data driven organization   that we work with and that's something that  we really like is that we can look back and   go okay so how is this working how are things you  know now that we've deployed this and we've got a   bunch of Learners taking this information is it  working and is it working effectively so one of   the pieces that we use is called the heat map so  we can kind of find uh our few people struggling   on one particular area is one area way too easy  and we need to really hone in on this and make it   a little bit more difficult and is there a good  a good mixture because if everything was green   that's not good if everything is red also not good  we want something somewhere in the middle because   we do want it to be difficult enough to where the  person who is kind of middle of the road gets gets   stuck and has to remediate and somebody who does  know it can you know relatively get through it   fairly easily so uh how does this uh you know kind  of work at the end you know if you've got you know   people who have not taken it and then they take it  one of what and uh essentially happens is you've   got an improvement in conscious confidence a  reduction in unconscious confidence another   reduction in conscious incompetence and then you  know the big reduction in unconscious incontinence   that's kind of a big thing that we're we're  targeting so it's kind of lower left to upper   right that is the trajectory that we're looking  for uh to your to your point yet again Molly   time to metacognition so learner one that same  example that we used from earlier those same   folks learner one if this graph will show you  time to uh cognition essentially did they get   to uh the stuff that we were trying to teach  them so the first learner you'll notice very   low amount of time actually I think this took  him about 12 13 minutes right so somebody who is   you know middle of the road about an hour give  or take and then somebody who doesn't know it   is going to take a lot longer because they have to  remediate over and over and over using different   materials and different methods for doing that uh  to to get him to that so uh depending on their uh   conscious competence as you can see that's  kind of the big factor here uh we'll move if   they are accurate and highly confident they  will move through material much more quickly   and then finally uh as far as statistics there's  a lot of Statistics I didn't want to bore you with   all the all the numbers because we could nerd  out literally all day on this stuff I think   it's really fascinating but uh suffice to say  lots of stuff we can do and a lot of stuff we   can do uh look at and we can also look at  what are the most uh difficult objectives   and find out where people are struggling so  uh when they're done with this do we need to   have a different learning opportunity for these  particular things so we're looking at you know   maybe pick the top three and then when they move  to a different modality of training maybe we can   hit these topics a little bit harder so uh that  is that's another another way we use this data   so there is a uh as you heard me say uh I  did get the learning engineer certificate   certification uh through area nine and what  I was told when I went through this is that   uh of the people that go through it who  have an MBA master's degree or higher   who are instructional designers do not complete  or cannot complete and earn their certificate at   the end of it it is very difficult and they said  it's difficult for a number of reasons so I have   moved on to their next level which they call a  curator which uses a bunch of these different   electives that you see here on the right but  basically there is a certification path that   you can use through area nine uh the folks  have been very nice to us helping us along in   this process and understanding kind of how the AI  system works so it is difficult and even you know   people who are very intelligent uh sometimes  uh don't always make it all the way through   so that was the big bulk of it if you have  questions about adaptive learning please follow   them in the chat uh because I'm going to move  on to some of the other things that we're going   to be doing uh to address the Hands-On learner  but from Super far away so feel free ask your   questions this is a great time to do it I'll I'll  get to it if it comes through the chat but I'll   move on a little bit here the other thing that  we're doing and I heard a lot of people say this   um that they wanted to do some sort of virtual  reality thing and we've looked at that too and   we think it's a great idea it is a is a bit of a  lift in many places so uh unfortunately at this   time you know the the resources just aren't there  to create a bunch of virtual reality but I did   really like the the back and forth live training  uh type of learning I thought that was cool so I   think that is super fun so uh I actually I wrote  that that idea down and maybe we can do some   virtual sessions we have some virtual training now  but it tends to be more one way so if we can get   it two-way man that would be the sweet spot we've  also looked at augmented re-augmented reality um   because of the uh system constraints and resources  for all of our Learners not everybody's going to   have a VR system and a place to do it and time to  do it and all sorts of things but what else what   does everybody have including technicians they  have a phone right I've got mine right here and   if you want and you want to play along with this  you totally can what you'll need to do is you'll   need to go to the App Store on your any device  and download bundler bundler bundle AR the app   and if you would like you can download our kind of  pre-test there are kind of tests of this called a   i o FEMA test s-e-m-a test you can download  that and basically what will happen is you'll   get a 3D image of our device this is one of  our devices I've got one sitting right here   and what you need to do you need to kind of  point it you see that picture on the left you   kind of need to point at a flat surface any  flat surface will do a table floor doesn't   matter and then you'll hit the lower left  button the kind of the icon of the the image   and then what you can do is it'll actually display  it on that flat surface and you can walk around   it three full 360 if you want to and you can view  the image and what I'm trying to work with the our   uh our AR people to try and actually manipulate  the device be able to pushing buttons and that   sort of thing I think that's super fun so this  there's a lot of applications for this and one   of the things that I'm looking forward to doing  is working with uh possibly Google Glass where   our technicians are wearing safety glasses as  they all should be and they have the augmented   reality kind of up in front of them kind of like  if you're playing a video game and you've got you   know your HUD showing you all the information of  what's going on and then potentially bringing up   the service information if they need it so they  can look at it and they can flip through the   pages while they're looking at the car they  don't have to get up they don't have to go   to the computer they have to do anything they  can just kind of flip through pages on their   glass while they're actually touching the car so  that's something that I'm looking forward to uh   question was was the area 9 adaptive platform the  preliminary assessment step or does it train your   Hands-On learner sufficiently so that is a great  question and a little bit of both so to answer   your question it's both uh it is step one in our  process as Mike pointed out uh but it's a little   bit about a little bit of weeding out who uh who  kind of knows their stuff and who doesn't and One   Track goes this way another track kind of goes  this way and then also at the same time it does   uh get them to proficiency as kind of the the data  shows it does get everybody to the same exact spot   it just kind of does it sometimes in a different  way where you kind of have different tracks   I like to talk about our centers of excellence a  little bit these have been uh feng shui feng shui   a little bit uh to address all the different types  of learning that we have this particular building   uh was originally slated to be our building but  unfortunately didn't work out but uh we're still   looking at different things but you can kind  of see the the layout that we have we've got a   Hands-On experiences we've got some uh classrooms  that you can see in the photos here we also do   have a retail space too to kind of help keep the  lights on and then we've also got a virtual space   which I think is one of those blank rooms that  has nothing in it that is going to be our virtual   training space for uh one-on-one interaction  between a an educator and the Learners virtually so to answer your question that came in uh kind of  what are the steps so here's our our basic steps   uh this is uh brought to you by Mr Chesney and the  the way that they train uh Aviation mechanics this   is also very similar to the way they they train  uh mental health people as well uh myself being   the mental health field for a number of years in  my licensure process this looked very similar so   we've got some core competency learnings you've  got our e-learning modules that have adaptive   learning some do and some don't uh most of them do  we'll have also live online sessions that's kind   of that one-way communication that'd be great  if it'd be two two-way and then also in-person   classroom as well so any combination thereof to  get them to the core competency and then on top   of that we're going to have them uh hook up with  a mentor either virtually or in person depending   on the type of education that they need and we're  going to have them go through what we call just   reps or repetitions and the mentor then can say  okay you've done this five times but I see you do   it with a great uh High degree of accuracy every  single time we don't need to do the full 20 but   if somebody's got let's say uh you know a low  uh degree of accuracy low degree of confidence   and uh has you know has some deficits in what  they're doing and they're probably going to have   to do more reps until the mentor is sufficient uh  sufficiently satisfied that the person is doing   what they're supposed to be doing beyond that is  an oral exam where folks will have to uh you know   basically be fired questions and they'll have to  answer uh those uh random questions that we have   and uh yes thank you Mike you must demonstrate  proficiency a certain number of times before   moving on to the next step so this is  very linear one step one two three four   now in a lot of these certifications that you  see there are between four and six competencies   and what we'll do is uh to save time and effort  is we're not going to test on every single uh   efficiency what we're going to do is  we are just going to take two or three   of those and then we're going to randomly  select them and then they must demonstrate   those proficiencies and so they basically  have to be ready for all of those things   uh question uh see everybody is with me this means  that my presentation went fantastic because people   the questions people ask or comments that come in  are right in line with what is happening next and   the question was Mentor mentee how are you keeping  standard and consistent evaluation fantastic that   was my next slide here it is Mentor mentee no way  yes with an H Hoy very true so yes we're using uh   pyramid fee so yes Molly you were 100 accurate  and that's the way we're going to track uh these   particular things so yes uh you have to train  a lot of people very far away who are very   scattered with very limited time ability  very limited resources and uh oftentimes   don't necessarily uh always want to be there  because they're taken away from their job they're   losing money they I you're already training me  though on this Eric I already know what I'm doing   Etc et cetera so a few challenges that we are  looking to tackle uh in in large scale uh coming   up uh shortly now that is the end of my pain  presentation Eric if you would please go back one   step I just want to uh reinforce the point so on  the um left hand side down at the bottom you can   see my repetitions so we actually in this Mentor  mentee application you can build a a spreadsheet   if you will of exactly what it is that the mentee  needs to perform reps of so you can customize this   to whatever it is and that way you can help to  make sure that it is being applied consistently   so that any one of my mentors they're all  operating off of the same checklist if you will   and area 9 is a subscription model so  um don't know about the minimum number   to to make it affordable I know um what  we're paying for it and and uh but we have   some growth plans associated with that that  would make our situation uh unique in that   and Eric how long did your first level  of certification take you to complete I think it took me because I was doing this  during during work hours you know you're gonna   have to juggle a fair number of things you have  to grab you have to get a you have to get a a   module essentially first and then they will walk  you through it and there's a number of different   uh their own modules that you have to take I  would say between 8 to 12 weeks depending on   how quickly you can go through it I think uh  just our time constraints we were I think I   did a six but they say to block out you know eight  to twelve weeks per to get to that first learning   engineer stage now they they do have learning  Engineers of their own so if you wanted you   could hire them to create your content for you  the other really interesting thing about this   is is we actually can take our training content  and put it into our internal learning management   system and all of the data flows into the area  9 LMS so our Learners are internal Learners have   never even seen the area 9 LMS but that's where  Eric and I go to to gather all our data foreign yeah it's uh ukg or ulti Pros um LMS and  I would not recommend it to anyone but   It came it came free with  UltiPro get what you paid for yeah we do have other options that we're exploring so with that said thank you Mike for that  information for the the updated information   if you have questions you can you can direct  them this way if they need to get to Mike   I'll make sure that they get the mic but my  email is uh Eric dot McClellan app repairify   dot com and uh as a short a little anecdote  when Jay said to me hey Eric uh you like cars   right yeah I like cars would you like to work  for repairify I said I've never even heard of   it that sounds like a made-up meme he goes no  no it is very real so uh here here we are so my first conversation with Eric was so let me tell  you about your dream job and he uh immediately you   know red flags went off and uh and he was uh very  suspicious to say the least but fortunately I was   able to convince him that it was yeah and Mike  there's a question uh about Mentor mentee uh is   it an application that we purchased yes and uh  if anyone is interested I will um if I can find   it quickly enough here I will put the uh email  address of our contact uh out in the chat I've uh   provided to some other people in the um Collision  Repair space because this is a really big need in   our industry but uh yeah I'll uh I'll share the  contact info because it's a really cool technology give me one moment to see if I can find it all right so Luke J Jill I that is the end  of my material for today I'm not sure what   else you would like um thank you uh what learner  feedback are we getting that's a good question   uh some of it is uh very good which is hey I got  through this really quickly Daniel thanks for the   thanks for the time there are some organizations  who are using adaptive learning for their legal   requirements their yearly uh requirements such  as HIPAA or compliance training or something   to that effect so they are finding that uh they  can get through compliance training much faster   our organization not using it for that because we  already have something in place that's working so   instead of spending that you know five six hours  every year they're finding that they're spending   two or three hours every year doing compliance  training so um you uh I I will share one funny   anecdote though uh which is uh one learner  who we immediately identified as unconsciously   incompetent uh got got uh caught pretty quickly uh  by the the learning uh saying that oh this is this   this is this is baloney there's you know this is  this is not true only come to find out the person   was a little too confident in that person's uh  Supervisor was then able to kind of go to that   person and do some one-on-one remediation uh with  that so we were able to kind of catch that person   so um it was a little funny um but it was it it  worked out uh pretty good at the end so uh yeah   actually uh pretty good people like the ability to  move through material quickly that they know that   they know that they know so they they appreciate  that it takes them about 10 minutes just to prove   that they know what they're talking about we can  do it at scale thousands of people all at the   same time can prove to us yeah they really do not  know what they're saying so yeah it's uh it's good   how do you pair mentor mentees and does the  pairing relationship last beyond reaching   proficiency Mike is uh you're the kind of in  charge of the mentor mentee part of things   we find oftentimes that it does um however what  you are being mentored on will also have an impact   upon who the best Mentor is for that so we have  different mentors that are in different areas   so an individual might stay with one Mentor  for a while um in other cases though if I'm   going off to learn uh the Volkswagen tool and my  previous Mentor didn't know the Volkswagen tool   then of course you'd have to have a new mentor  and um choosing people to be mentors is always   difficult we have found that the extremely highly  productive technicians are not usually the right   people to be mentors they don't want to slow down  nope to show people how to do it so uh definitely   do some interviewing and some assessing of uh  people before they step into the mentor role   and actually uh Mentor mentee has a really robust  training program for mentors uh built into it so   I would encourage people to Avail themselves of  that so they you don't have to reinvent the wheel and Mike to your point uh I've sorry go ahead  oh I was just going to sort of add to that so   once you've identified who can be mentors how how  does the matching occur is that a manual process   yes in our world today [Music]   yeah I was going to say to to your point and  also to to your point Jill uh same thing which   is uh some people who are just fantastic at their  job who are top performers um don't make great uh   teachers and uh and and also what we've found  is that uh they in the beginning there was some   initial resistance to getting these people to  be mentors to trade train others well can't we   have some other process because they're worried  about the high performers or the ones who are   really good slowing down in reality what we  found is that there's a small brief window of   slowdown and then a huge impact when you've got  this person training for three or four people   at a time now you're doing the work of uh four  people at at once in a very short period of time absolutely so in a situation where we have a  part-time or or temporary Mentor we   typically would bonus them for the weeks  that they spent on the mentoring program   um I do have uh six full-time  mentors on the team right now so they're totally compensated 100 yes Jill agreed yeah they those people  are selected for their people skills and   so that so they actually have to have a dual  competency they have to be people people and   they also have to be very proficient at their  jobs so the selective process is well selected okay well we are getting close to time but we do   have a little more room are there any  more questions before we wrap up today all right well let's give a big thank you once  again to Eric and Mike for presenting for us today   I know I found it very interesting we really  appreciate it oh our pleasure our virtual Round   of Applause yep go ahead thanks for the invitation  yes thank you for the invitation thank you for   hosting and and again thank you Frederickson for  all you do helping to connect us learning leaders thanks again everyone yeah the  mollies took the Day pastor and   yes moving forward today we'll always  be National Molly day excellent foreign

2023-04-25

Show video