Teaching communication skills: Evaluation of a novel technology for physicians

[Music] thank you very much for that super kind introduction Rosanna and so great to actually be standing in a lectern for the first time in many years so that's weird I'm Susie densis as you've heard I'll try not to touch my microphone sorry I move a lot and I'll be talking today about the work that we've done here and also in collaboration with some of our colleagues at other institutions about novel Technologies for teaching communication training so we'll touch on why communication training what that means and why it's relevant what the traditional approach is to teaching communication skills have been here and in the department of pathology and finally we'll introduce this new video communication assessment app and see whether or not you you agree that it should help us further our communication training for our residents so first why communication skills well this audience Laboratories and pathologists is well aware of the fact that we don't have a lot of contact with patients as a matter of fact we don't have a ton of contact with our clinical colleagues and in the last two years we haven't even had very much contact with each other so we're not experts in communication we communicate largely by reports and by resulting and we all know that these purports and results are perfectly clear and concise and there's no problem right our treating clinicians understand everything perfectly I like this quote by George Bernard Shaw the single biggest problem in communication is the illusion that it has taken place so the Joint Commission did a root cause analysis over a 10-year period quite a while ago and they looked at the root causes of Sentinel events so these are patient safety events that cause death Serious injury or permanent or sorry a permanent Serious injury or semi-permanent harm to patients and they found that looking at 3 500 events the single most common problem that was within these events was communication so clearly lack of communication causes severe harm to our patients and this has been followed by a billion studies one of which is the Institute of medicine study which was published in 2015 improving diagnosis in health care and this study one of the key themes was that communication is essential to prevent diagnostic error and to improve outcomes and so they go on to talk about the principles of communication and effective communication in red here um without this you can expect serious errors to occur okay so what is communication I throw this out to our small audience here am I communicating with you yes or no no yes we're not sure okay that shows the level of communication right of getting thumbs up thumbs down I'm talking at you right which is not actually communicating with you this was a image that I stole from the agency of healthcare quality and safety and it was made to show communication as opposed to show the exchange of information between number one the receiver and the guy under number three I'm sorry the sender and the receiver and you can see the sending of the message here and this is supposed to be communication but we know this isn't because I could be talking here and you're thinking about dinner or you're thinking about how the target when you place it in a urinal improves the aim of the urine stream and doesn't have splatter around the toilet right well if you weren't thinking about that you are now right anyway so if you don't ask you don't know what the receiver is actually thinking about when people talk listen completely most people never listen so with all of this running around in my mind I decided it would be nice to actually study the communication patterns among our colleagues um to publish some papers about what we know about it and maybe even to have some courses so that we could teach communication skills and over the last several years with colleagues we've done this both at meetings nationally at different universities and here with our residents at the University of Washington and the objectives of these courses just what we've been talking about to make sure that everybody understands the connection between communication and medical error to Define what communication is and what the standard should be for Clear concise communication the course design originally had a lot of demonstrative audio and video clips so we uh in with help from Tom Gallagher who's in the quality and safety uh Center here at the University of Washington we were able to videotape good and bad quote Communications and show those also audio clips and before covid we had a lot of teaching that involved role play and interaction between the residents and the teacher trainers and of course all of that has stopped with zoom making it even more challenging to talk about these subjects one thing that we did before covid was these simulated communication modules so we used what we call simulated clinicians so these were our simulated patients who volunteered to become simulated clinicians on the side they loved this they particularly liked pretending to be surgeons they got really into it and actually were pretty good some of our residents actually thought that they were our surgeons and asked is that that particular surgeon they did a good job um so the residents used phone calls to communicate with these trained simulated clinicians and the scenarios were designed to test varying communication skills from simple just giving the diagnosis to to other communication is where we put barriers so we would have an angry clinician or we would require the resident to disclose a serious pathology error to the clinician we audio recorded these and the research team scored them using a checklist and also those audios were able to be presented back to the residents so they could see how they were doing they could hear themselves we focus them on intraoperative consultation because in anatomic pathology that's our most common critical value and is almost always verbal and again we we deconstructed the communication into these elements so that we could score them and I just wanted to play for you an example of one of these audio recordings so that you could hear what it was that we were doing pre-test so this was before our residents had any training in communication skills and this was the scenario the resident was in the frozen section room they were calling in an interoperative consultation to a surgeon who had given them a chest wall biopsy in a patient with a previous history of breast cancer and the question of course is is it cancer the residents were provided with a mock report so that they could see that the frozen section was a poorly differentiated carcinoma and they were to call that back in to the or but we threw in a barrier because the surgeon asked is it the same tumor so is this chest while biopsy the breast cancer that the patient had and we wanted to see how the resident would react those of you who have done Frozen sections know this is a common question but is also one that we have insufficient information to answer in real time so I'm going to play you this barrier scenario and you will not recognize these voices for for privacy reasons okay um is that the same tumor as the original uh there is no way to know at this point in time and if it is the same as the original okay it'd have to be compared to the original [Music] okay okay okay thanks wonderful is that the same as your original tumor uh yes I believe so okay so um breast cancer then yes indeed yep right thanks for letting me know okay thanks the original [Music] um um yeah I think so yes that's what it is yeah okay so is it breast cancer yes ma'am okay great thanks actually it is um is it the same tumor as her original uh with respect to being primary or secondary I'm not quite sure at the moment I have to speak to my attending does that change your line of treatment for the pain but I just like to know what's going on in there okay no problem let me get back to you on that after speaking to my attending on that because I'm not uh quite sure about it okay is that the same as her previous tumor is it the same as the original oh sorry uh I am I am just feeding down the original yes it is okay it is high grade yeah okay and as high grade Dr carcinoma Okay so um a lot of different responses these were residents at various well these These are actors so we dubbed over the real voices um but they're people who are at different levels of training um and you can see that there's quite a heterogeneous response and this could clearly affect patient care right because you don't know you can't know looking at a biopsy whether it's the same as the original tumor with the information on hand so we looked at these pre and post training sessions and you can see here that the pre-test percentage of a score for for our checklist items was not as high oops as the post-test score after training and intervention so when we trained residents for skills on how they might answer questions they did improve and they improved even when we threw in difficult scenarios like this after conversations the question always is though on these simulations how does it really happen in real life so great they improved on a simulation but what happens in the real world so we actually recorded 34 interoperative Communications and scored them using the same checklist you can see there were 13 attendings 21 residents some of whom had had training some of whom had not and the attending pathologist's overall score was luckily higher than the non-trained residents but after communication training the residents improved quite considerably but interestingly they didn't attain the same levels as the simulation which you might expect because real life is in the simulation so you don't actually perform generally as well in a simulation scenario but we showed that you can actually improve with just fairly minor intervention we included really difficult Communications as I said including error disclosure and this is important because of starting in 2017 the acgme now mandates training and participation in disclosure it's one of the competency requirements so how do we know that our residents are competent in error disclosure before I go into any of that I could it just give you a tiny bit of background about what we know about what physicians in general do for error disclosure and this is work um that I did in conjunction with Tom Gallagher um actually these these next slides I stole from him so the quality of actual disclosures this is data from Copic that is one of the largest prac insurance agencies that covers professional liability insurance so malpractice insurance for doctors so obviously they're interested in outcomes they don't want bad things to happen and they looked at a program for disclosure and compensation uh over a two-year period these are patients that had Sentinel events so serious harm because of something that occurred in their care they looked at 800 events more than 800 and they surveyed both the patients and the providers to ask what their experience was with disclosure of this error so when they asked the Physicians on a scale of 0 to 10 where 10 is I'm amazing you can see the Physicians were pretty confident that they'd done a good job disclosing the error to the patient they felt pretty cocky you look at the Patients about the same air disclosure and you see that there's quite a bit of disagreement about how that went and again that's the communication aspect of this um and there were quite a lot of them that were not satisfied at all with the quality of the disclosure so from Tom's work and others we know that communication after harm very rarely meets the patient's needs and inadequate physician preparation is definitely a contributing factor so what about us those Copic cases were treating clinicians surgeons internists pediatricians but they weren't Pathologists and laboratorians as you know we don't have an established doctor-patient relationship as a matter of fact most of our patients don't even know that we're involved in their care they don't know our names they don't know where we live or work and the treating clinician is involved by proxy so even though we make an error they're involved in that error because they have the relationship with the patient so it gets really tricky and over the last several years I've made it part of my academic study to look at improving pathologist communication skills looking at how Pathologists think about errors and how they communicate those errors and I've had that great pleasure of working with a number of amazing collaborators Tom Gallagher as I've already said Has Been instrumental and Dr yell here who is the director of quality and safety at Mass General has also been a really great collaborator and through the work that we've done over several years I'm looking at focus groups and surveys of laboratorians and Pathologists we've got some idea about why they have trouble communicating error and what the barriers are and it's interesting because what you guys might think are the perceived barriers um are in this list so our crappy reporting systems and billing systems everybody always mentions fear of getting sued right so if you're of litigation reluctance to be prescriptive to tell people how to do things to tell doctors how to communicate an error lack of consensus consensus and of course difficult cases but when you look at the surveys and you do the focus groups it turns out these things obviously make people uncomfortable but they're not the reason that errors aren't disclosed the real barriers are shame embarrassment rationalization about the error some mixed messages still from institutions I know back when I was a young pathologist I was told don't ever apologize don't ever admit fault that's a bad thing and some institutions still have that as a philosophy and most importantly low confidence in communication skills so it's in that background where our treating clinicians and our Medical Teams are struggling with error disclosure themselves that we are called upon to judge our residents in their error disclosure skills so this is the milestones for pathology residents and you can see a level for participation or leads communication of error discrepancies to clinicians and how do we judge that suggested self-assessment well we already know we think we do a pretty good job right written or direct observation how many errors have you guys been able to be a part of disclosure right it's a very rare event narrative so the current approaches to teaching and not and look and trying to help residents in these soft skills don't work very well so lectures don't impart skills like I said I have no idea what you guys are thinking probably you're hungry um real life practice as we've said in these events is infrequent high stakes do we really want trainees disclosing serious error to patients um and feedback is minimal standardized patients that we used for those communication scenarios are hard to come by they're expensive and their assessment is sometimes unreliable and finally role play has been pretty much deep six these last two years so how do you do this when you can't actually be in a room together with a lot of people so what's needed to advance communication training a practical proven system for simulated communication practice followed by statistically reliable formative feedback enter the video communication assessment app so this is an app that was actually developed by the national board of medical examiners and specifically for the purpose of communication training for their patient-facing specialties what is it um it is an app that you can download onto your computer or your I or your smartphone and it records audio of a resident interacting with a simulated patient so the user reads the scenario and watches a video clip of a standardized patient and then records their audio response right into their device that response is then rated and that the question is always by who in this case the audio is crowdsourced so it's sent to Amazon's Mechanical Turk which is a website that uses a crowdsourced people so they can choose to pick up scenarios and rate them and in our case we used eight to ten us-based adults and the Raiders rated the audio responses in patches and the variables that the Raiders assessed included things like honesty how they thought whether they were clear whether they were sympathetic those kinds of questions and finally we can play those audio back to the residents with um a summary score of how they did compared to the peers that were also taking this and texts suggestions from the user comments so what does it look like something like this so here on the left is what our user would see on their smartphone there's a case that's presented so you know the whole patient background the case is tailored to the specialty in our case when we had pathology residents we had pathology cases this is an orthopedic surgery case where this guy was had a procedure and then got a DVT and they decoagulated him and he had a huge gastric bleed um they read the case then they play the video of the patient and the patient may have different affects they could be angry they could be sad and then after the the video plays the resident is prompted to record their response what would you say to this patient you can review and redo that response as many times as you like before you submit it so if the first isn't great you can do a second third or fourth uh the Raiders who are the crowd-sourced m-turk people have uh a likert scale that they use one to five on how the president did and they're looking at things like are they accountable are they honest were they sincerely sorry could they be understood did they feel like they were cared for and then finally after the Raiders do their rating the resident can go back into the app um this is a pathology vignette that we used and they see there they can hear their response again there's four weeks in time between when they did it and they get the response so you might want to hear your response again and we also give them the opportunity to listen to a response that the public rated very highly so they can actually hear what the public thinks is a good response and down here they can see their score compared to their peer average whether they're doing the same better or worse than their peers and finally any learning points I should go back we let them free tasks what would you want the provider to say if you were the patient and that can be input here so that the resident can see now obviously if you're looking at error disclosure the judges that you'd want or the Raiders that you want are people that had themselves had harm but you can't reasonably recruit harmed people to run a VCA program so the question that we have about this app is our lay people who are the people that are being crowdsourced reliable Raiders can they serve as circuits for patient experiencing harm and can they identify low and high performing residents uh so this was the first paper that we published um early this this year yes early this year um looking at the reliability of the VCA for this application in this study we used one case of delayed diagnosis of breast cancer and we compared eight individuals who had experienced serious harm themselves or were the caretakers for people who had a loved one with harm versus 59 recruited M Turk lay people and the statistics that they use to look at this is under what's called generalizability Theory and what this is is is basically if something has often happened in the past How likely is it to occur in the future um and um an example of the integrator G coefficient helps evaluate how well we might generalize a score from one reader to another so this is accepted statistics in this field and the g coefficient with 10 to 12 Raiders which is a lot of Raiders and four cases was 0.85 and 0.75 and above is considered good so um this scoring was highly uh reliable this is the raw data on the blue lines are the crowdsourced lay people the red lines are your Patient Advocates and the x-axis is the number of Raiders who saw the cases and the difference lines are the number of vignettes that the Raiders saw so as the number of Raiders per resident increases and as the number of vignettes that the resident partakes and increases the reliability increases and you can see that obviously it's better to start with people who know this area but even if you start with lay people if you give them enough cases they start to get about the same as the Patient Advocates so you can interchange lay people for communications like this if you have enough Raiders so it looks reliable the next question is is it working do the residents actually improve their skills because of feedback from the lay public and does the affect actually affect the performance because we did use sad and angry as our two patient responses and the second paper which just came out addresses this resident communication skills pre and prose trial and you can see we had lots of UW residents with this video communication assessment Andrew purposely picked these three Specialties that are um OB GYN because it's procedure based internal medicine because it's patient facing but not procedure based oops and pathology because it's neither it's Diagnostic and you can see that we got all different pgys of course Internal Medicine doesn't have a pgy4 but all of the the years were represented and this is how we did the experiment so we've divided all of the residents into group one and group two um each had two cases Assad in an angry case they interacted with the VCA got their feedback and then we did across which is a common thing to do so they got the other cases for their second trial with feedback you can see the number of participants dropped and I don't think there's anybody here who did this but I can tell you that after the first VCA well maybe I shouldn't I won't tell you that they were angry but I won't tell you that they weren't about having to do this because it was actually pretty emotionally charged to have to talk to even a simulation to have to answer an angry patient they found it quite difficult this is an aside but I thought it was interesting that residents perform slightly better with angry patients than sad patients there's a lot of reasons why this might be true we'd obviously have to study it more but it's in some cases it can be more difficult to actually uh uh say that you recognize that somebody is sad where it's easier to to to recognize the anger so the overall ratings for the 46 residents that did both pre and post testing improved from round one to round two with a pretty good p-value and there were no differences in ratings based on the specialty or the years of training the thing that mattered the most was Prior exposure to error disclosure so those residents who had had prior experience did better at time one than those that didn't and prior air disclosure was significantly different among Specialties like to take you down to here and you can see that very few for obvious reasons of our trainees had had any experience with error disclosure to either patients or treating clinicians whereas our clinical colleagues have some more experience and the conclusion of the paper actually called out that simulation simulated communication training May particularly benefit trainees and diagnostic Specialties such as pathology with infrequent real-life practice opportunities um I've said before that the residents who took these cases um thought that they were very realistic in fact they found them emotionally challenging and some didn't want to return to do round two because they felt uh it was very difficult so with respect to the error disclosure piece of this the next step is recruiting multiple sites different institutions integrating these vcas with the curriculum making the mandatory and getting adequate sample sizes and hopefully we'll be able to figure out what the real world effectiveness of this VCA is in the training programs how it can be optimized and how it can be made accessible I mean acceptable to the gme leaders all right moving to um bystander training for microaggression so not only do the competencies require error disclosure but if you look at the pathology Milestones from 2017 you can see patient and family-centered communication part of the pathology milestone in particular recognizes personal biases while attempting to proactively minimize communication barriers yeah that's so easy to train and so easy for us to judge and yet this is part of the rubric that we look at every year as part of the AP teaching and training committee and which we need to actually evaluate a residence for so it was when I'm on the teaching and training committee looking at this trying to assign scores to our residents and at the same time I had the opportunity to join the white anti-racism group here at the University of Washington and this was a group that was and not only restricted to the medical center it was University of wide of people who felt that they really needed to be together in a group to talk about current issues it was conceived of and monitored by Jonathan Cantor who is one of the core leadership team members of the office of healthcare Equity here at UW medicine and a lot of the conversation I realized when I was part of this focus group was how to understand and intervene on microaggressions um and at the same time that all this was happening there was announced that the department was going to fund a Dei research Grant and I thought to myself could the VCA actually help teach a Dei curriculum because here we have a way that you could do this in private at your own pace that you can feel safe and get feedback anonymously which is so difficult to get in real life something that I struggle with daily so the next training challenge by Sandra responds to racial microaggression so just briefly for those of you who may not know microaggression is a statement action or incident regarded as an incident instance of indirect subtle or unintentional discrimination against members of a marginalized group such as a racial ethnic or sorry or ethnic minority and I just put some examples here you speak English quite well or you should smile more are pretty classic examples that people identify as possibly being microaggressions so I talked to Andrew White who's the leader author of the two papers on error disclosure VCA and asked what do you think do you think that we might be able to develop a curriculum around this around microaggression and use it in the VCA and he thought that that was a great idea I think Andrew also I used some of his or modified some of his slides for this presentation Tom Gallagher of course is involved in the air disclosure work and he also felt that this was a good application Karen Brigham is our wonderful research coordinator and it helped do all of the vignette videography and we all agreed this was great but we all admitted that we knew nothing about microaggression and so we needed to get a team of people who actually knew something about this and we were extremely fortunate to recruit some experts to help us as we make this VCA one is Dr Amanda almond who's a professor at Cooney in New York City and whose area of expertise is microaggression especially patient provider interactions and also Jonathan Cantor who has been instrumental in helping us develop the vignettes and um as I've said before he's the director of the center for science of social connection and actually has a lot of pull in our Equity office here at the University of Washington so these people have been crucial in making sure that as we develop these vignettes we do them in a scientific way and that we are looking at what we are trying to look at in in the social sciences our team also has a good fortune to be able to include some of our wonderful trainees so Catherine manbeck is a PhD candidate and Jonathan cantor's group Rasheed durawuju is a fellow in cardiology and internal medicine and our own Lindsay you guys are probably now is a pgy4 in our department and they have been helpful they keep us honest and and they are also going to be extremely helpful as we recruit people to help do the piloting of these VCA modules so where are we now um the by standard response to racial microaggression project has just wrapped filming of the last vignettes it actually was really difficult you'd be amazed at how difficult this is not only to conceiva vignettes that are realistic and meaningful but to recruit the appropriate actors to help obviously we can't have people that any of us would recognize as part of that they have to consent to being actors in these vignettes so we just finished filming the last vignettes and I'm going to blatantly recruit you to consider participating in the pre and post testing of the VCA for microaggression bystander training we hope to go live very soon the vignettes are being loaded into the VCA by the national board of medical examiners right now and so you'll be hearing from me if your faculty from Lindsay if you're a residence we need both um and then Rasheed is hopefully going to help Andrew recruit in in Internal Medicine so that we can get the pilot data for this so in summary communication skills are critical hopefully you understand that you can't have good health care without them Pathologists are not exempt from this requirement just because many of us sit in an office by ourselves does not mean that we're exempt from good communication communication skills are acquired not innate they can be trained structured training can improve clinical communication skills VCA training in particular resulted in improved air disclosure and we hope that the same methodologies can be used for bystander intervention training in the setting of microaggression and so with that I'll end this talk and ask if there are any questions and I thank you very much for your attention foreign thank you so much for explaining this is such an exciting technology and it's really nice to actually see you explaining it so in terms of the microagglation uh how many people do you think you are aiming for this study and it will be also a two-face study with cases exchange could you elaborate a bit more yes it'll be a similar study to what the error disclosure was um so we actually filmed eight vignettes um in various settings some of them that are more pathology laboratory Associated and some are more Clinic settings for the different groups that we hope to enroll um so we'll have again the four and four with the cross so two both but same same thing sorry and for the pilot um we hope to get as many people as we can but we're looking at between 20 and 30 people to Pilot so it does cost money to have the mturk Raiders rate so it's uh for the pilot we just want to show proof of concept before we take it live and make it a bigger operation Jeff thanks that was really fascinating and I think what I I was thinking about as maybe this is sort of funny it's like an apcp thing involving people versus if you really want to scale it you need robots to do it the Amazon Mechanical Turk is he said it costs money and then you you I think you demonstrated statistically that there's some correlation but at the end of the day it is people that you're paying you know 50 cents to do that and they might just click see all the time right and you try to rule that out right there have been in in even the recent months not just recent years amazing advances in AI for natural language reading and discussing things like gpt3 and stuff like that have you thought about you know trying to train an AI to just find if you're only really filling out like a rubric of like 10 things you know rating on a likert scale you know what did someone pay attention to this it might be that an AI could actually do that in which case then you could Implement that to all residents in the country right as opposed to just 20 residents right so I guess am I supposed to repeat the question for the that was really long so I'm going to summarize that Jeff wants to replace us all by robots um no he asked whether or not uh using uh some kind of AI machine learning algorithm might be as good as using m-turk Raiders and the answer is maybe but the VCA is already set up by the national board of medical examiners and they're actually paying that they're paying for the for the mturk for this project because they'd like to see whether or not it can be used and scaled eventually you might think that you could defray the cost by offering this to residency programs at some cost hopefully at Cost um so because it's so difficult to do this training otherwise so the hope is that um we could have vignettes have this shown to be reliable scalable and then have it available to gme leaders who would like to use it for this competency um the robot thing I mean that gets really complicated though doesn't it that's like a whole different talk because these particular soft skills um are I don't know how trainable they are one of the things that one of the big criticisms of the VCA is that it's just voice and obviously when you're interacting with people there's a lot more going on and most of the communication rubrics also use body language you can sound really nice but if you're looking with eyes of fire at somebody that's a problem so um I don't know but the good thing about this is that at least it's something that we can test is it perfect no we haven't achieved Perfection but we can use this tool now to start to optimize and push the limits a little bit more Rosetta yeah did you have a question I had a question really great talk um so related to what Jeff was mentioning for error disclosure training you were comparing your enter evaluators with um their ability to evaluate with patients who have experience um with the sidestander training now like how can you evaluate your evaluators like how are they equipped to judge are they sufficiently diverse right yeah that's a great question so the question is um that for the error disclosure trading obviously the ideal Raider is somebody that has experienced medical harm but in bystander training how do you know that the lay people can adequately evaluate a bystander aggressive well the nice thing about mturk is that it's very diverse so um the problem I see with mturk is that it's a web-based which means it's mostly younger so you don't get a lot of Boomers doing M Turk um but you can for example for the OB GYN vignettes we had only women Raiders because they're the patients so you can actually segregate the population according to age according to self-identified ethnicity gender and look to see if it makes a difference the other thing is Jeff Jeff um brought up a good point which is how do you know these Raiders aren't just clicking AAA to get their 20 cents or whatever they get per vignette and um we do do some quality control so if they didn't complete all of the vignettes and all of the answers they're excluded um if they did AAA they're excluded so uh the the national board was excluding they have parameters that they can assess reliability of the writers so it's not not all covers no questions okay comment for Jeff that I told the person inside you should talk to Jeff okay so this is an idea or suggestion and thinking in terms of how to spread out the value right so definitely that is obvious value in the person taking the test and being evaluated but the response value also to the collection of all the information being communicated even just the vignettes microaggressions is such a difficult topic so unknown so even just providing people access like a like a tutorial or a video access to the vignettes and which are the right responses or the responses that were with highest scores and the responses that were with lower scores right that would be already very valuable as a training tool right and so the the comment is that there's a ton of information in the vignettes that are recorded regardless of the score given by the Raiders just to see what the the difference in responses between all of our residents and uh there it is it's a it's a gold mine for sure Big Data I think that a lot of the value is just having it be an exercise so if you if you present your trainees with a response to bystander sorry a bystander response to microaggression if they'd never thought about that before it's now in their Consciousness which is already a win um in terms of just getting it into the vernacular of the medical field because people talk about implicit bias and the problem with implicit bias is that these are thoughts that you have about people that you don't know you have so if if it's not presented to you and you're not even thinking about it at all it's it's not a part of your it's just not in the consciousness potential use of all these work after the pilot and you know in addition to operate as a training sure as a research tool to see what the variety of responses whether obviously you're going to see demographic differences you're going to see local Regional differences um so it it'll be very some of it will be expected maybe some of it will be unexpected but yeah it would be nice to drill into the responses what have you back in a year right yeah yeah more to follow but only if you guys take the VCA so hopefully um we'll get a robust a robust response to that so great thank you very much thank you [Music]

2022-12-05 13:01

Other news