Innovations in Technology Conference 2023: Creating Dynamic and Interactive Online Trainings
All right. Good afternoon. Good afternoon. Welcome to Creative Dynamic-- sorry. start all over again.
Good afternoon. Good afternoon. Good afternoon. Welcome today to Creating Dynamic and Interactive Online Training.
My name is Jane Ribadeneyra with Legal Services Corporation, and I'm very happy today to have with me Nalani Kaina, the executive director of the Legal Services-- the Legal Aid Society of Hawaii. We have Heidi Behnke, who's with Georgia Legal Services Program. We have John Mayer, who's with the Center for Assisted Legal Instruction-- Computer-- Computer-Assisted-- Legal Instruction. I always get that wrong. CALI.
And we have Katie Mayer-- no relation-- with the Legal Aid Society of Hawaii also. So thank you very much for being here today with us. Today's session, we put this together from a couple different proposals because we wanted to talk about online learning systems and the best way to create online learning tools. We've had sessions at prior conferences about learning management systems, about creating good video content, and there's a lot of questions still out there of what are the best tools for what you want to deliver, what are the considerations you need to make.
So our objectives today are to help explain some of the differences between online training platforms, learning management systems, what works best, best tips for how do you find what platforms are out there, how do you develop the content, how do you sustain good content. And also talking a bit about developing the content and user engagement, user testing. All of the good tools that you'll need for creating these online training systems.
So I am going to pass it now over to Heidi and Nalani, who are going to start off with just some definitions and trying to talk a little bit about the differences between a learning management system and online training platform, where there's crossover, where there's a lot of similarities, and what-- to give you some definitions that will explain what we're talking about after. There we go. So actually, there are not a lot of differences is what we found out in part of our conversation. But we wanted to provide you a map for some of the things that we think people are thinking about between a learning management system and a training platform.
So I'm going to-- both of us will kind of just talk a little bit about them. One of the first things that we looked at was really whether or not you have an independent user registration. So one of the things that when you think about systems, there are two different things that we look at. Do these systems have-- can somebody independently register for the system? We found out both of our systems could do that. We use a training platform in Hawaii, which we use a platform that's already been made called Thinkific.
And for Georgia? We have learning.glsb.org, which is a formal learning management system. Independent user registration is definitely a thing that we have. I do want to say, as we're going into this, even though all systems seem to be capable of all things, the degree to which they are capable is what's different.
We're going to talk more about that a little bit later. Reporting on user and course activities. That's definitely a big thing that we've got. Thank you. Video uploads or recordings, how we put the data in the training materials, being able to be downloaded or not, quizzes and surveys attached to courses, automatic generation of certificates and CLE credits can happen for everybody.
The CLE credits does depend on your state and how they require things to happen. Users can be charged, which, Nalani, that was your big thing. Yeah. For us, the biggest thing when we're looking at a platform is, I wanted to make money off of it.
It's sustainability, right? For us it was really, how can we ultimately sustain the platforms that we have? And so one of the ideas that we've really been looking at is social workers need credits, and it's a group that asks for a lot of our trainings for free. If you work with organizations you get requests to go do a presentation for the homeless service provider or for the hospital social workers. And so we thought, well, this is a place that we could actually charge them $10 for half a credit.
They sometimes need 30 credits a year. It'll at least help to sustain the cost of the platform. So we were really looking for a system that we could do something like that, and this training platform allowed us to do it. The cost is a big difference, and I think it's probably the biggest difference between our platforms. Nalani's has a flat rate of what it's going to cost per month.
Learning management systems will vary depending on which company you have, but usually the way they calculate it is number of admins, number of users. What's defined as a user is different among different companies, and if they're active or inactive. And again, all of these definitions will vary company to company. So if you are looking at an LMS, make sure that you're getting apples to apples information and you're understanding what their individual definitions are. Yeah, and let me just talk to that.
Again, cost was important to me. I didn't have a TIG grant to pay for this when we started this up, and so when I started looking at the cost per user I was like, there's no way we could afford it. And so the flat rate that we get, at least with this platform, it costs us $1,000, at most, a year right now.
And that's reasonable for the kinds of product-- for the product that we get. And again, you guys will get to see a little bit of a demo in just a little while about that. Help from the vendor. That's where I definitely thought the LMS was going to have more on it because we have a very close relationship with our vendor, and they help us set up the site. They help us change things if we have questions, especially with the reporting systems. They're there and they will do things really quickly.
And I was sure that you didn't have it, but you've got that. But again, it's a platform solution, right? So they're making their improvements. They're there if we call them and need them to walk us through things. We're not customizing things, so you also have to know that that's a difference with what we're using.
It's not fully customized in the way that Georgia is able to customize their program. But I'm OK with that because I don't have money, so I don't have a problem. I should mention at this point that we did get a TIG grant to do the initial startup cost, which was the biggest part of this. So thank you. And then connect content to a single course and courses to a single program.
So the amount of layering that happens so that when staff or pro bono volunteers or clients, whoever is using the LMS access it, are they able to see how things are connected to one another? Which was a big deal to us. And I think this is one of the major differences. I think with respect to what we do is we can post the training up and say, here's the training if you want to attend. They can click on the training. They'll get access to that training using their username and password, but it doesn't necessarily track all of the different lessons that somebody wants to do, right? So in a traditional learning-- what I understand a traditional learning management system, you're actually tracking courses. You're saying that here's the 10 courses you want to take.
Here's the things that you can manage over the course of time. For us, really, it's more transactional for our pro bono attorneys in our community in terms of as they want to get access to a particular training. And it'll also work for social service providers. It'll work for clients if we decide to open that up in that way.
And I'm going to add to that because it's not in my demo, but on the LMS I also can control if you don't complete this course you don't see this course. So that way if you have it so it has to be in an order, you've got it. And if it doesn't have to be an order they can see whatever pieces they want. And that can be like, you have to take the intro course to see everything else. Or it could be you have to take courses exactly in this way.
So yeah. So there's your-- at least the high level. And again, you guys will get to see the training platforms and the LMS demos in just a little while. And I'm the other example.
So I'm the executive director of CALI, the Center for Computer-Assisted Legal Instruction. We've developed an online learning and teaching platform. We don't like to call it training because law schools don't think of what they do as training. We publish over 1,200 web based interactive tutorials. They're written by law faculty and librarians and academic success folks, and they're written in an authoring platform that we've developed ourselves because we could never find one that would stick around long enough-- the company would go out of business or something like that-- or that did exactly what we needed for our purposes.
It does have the capability to do learning analytics, which is to say, not did someone run this lesson, but did someone-- but down to a very granular level of individual questions, and I'll show you an analytics report in individual students and how they did. And that's really valuable if you want to verify that learning happened. And there's a version of this system that we call Lesson Live where theoretically I could have given you all a URL and then you could all be running the lesson while I'm running the same lesson.
And I could be seeing your scores as you're doing that and maybe even seeing where most of you are doing poorly, and then call you out, like Professor Kingsfield, on those individual questions. All right. So what we're going to do today, and you've kind of gotten a few definitions, is we want to help you consider what type of platform is going to best meet your particular needs based on your goals, your audience, and what you're trying to accomplish, and of course, as Nalani said, what's manageable within your budget. So to help give you a better idea of each type of platform, we're going to do some short demos. So first up, Heidi has put together a demo showing you the Georgia Legal Services program's learning management system. They asked the name of the vendor that we used for our LMS.
Originally when we contacted it was Gnosis, G-N-O-S-I-S. But it's also part of a larger group called InfoPro. And if anyone wants to talk more one on one about the vendor in detail, I'm happy to do that. Yes? Was that second one you were using also InfoPro? It was completely different. It was called Thinkific. OK.
Sorry, could you repeat that? So that being said, I guess my question is something-- when you go to the sign up page that's on your web page, is that through the company portal or is that actually a link for people to sign up? For us it's a separate website that's us, and we have all the background stuff and the IT to support it as well. OK. And as far as user registration, if you're setting it up, let's say your thing is confirmed. Is it individual registration and then taking the AD when 10 users automatically registered? So as an admin I can register people however I want, and I can work with the vendor, or I can-- I think there's a way I can mass upload people. But if not, I could always just send them an Excel file and they'd do it.
But people can also individually register as users, and I can see who's on the system and who's not. So if I was feeling particularly lazy I could just say, hey, you go do it. It's here.
Sorry. There was a question. It's Thinkific. Yeah.
Yeah, no problem. I actually do have a question for each of you. So for Heidi, how do you create the-- is it PowerPoints and then do you create videos and upload them? How-- is that the same as the one? So everything-- there's a wide variety of different types of files that can be uploaded onto this system. SCORM files were what we originally built the site for with a TIG grant, and those are advanced powerpoints that are interactive.
That's the easiest way to explain it. But we do have powerpoints. We have PDFs. I can use MP4s. I can do anything that I really need to. I haven't found a thing yet that I can't upload.
OK. And then you were talking about being able to-- you upload the MP4. You said it was easy to market. Could you expand upon that a little bit more? What we've done is we've created all of our trainings in separate powerpoints outside of Thinkific.
So you can actually create your training directly in Thinkific, but by creating a PowerPoint and then, within PowerPoint, using the record option just to do the audio. If you want to make any changes, you can go into PowerPoint, change the slide itself if you want to change a text, or record over it, and then save it again as a new MP4. And then each of those lessons you just take down the old MP4 and upload the new one that way. So that way you're making your changes outside of it to the PowerPoint itself. And so we have all of that saved as a separate file. But it makes it so that if somebody else needs to go in to edit it outside of me later on, it's really easy to do so when we have all of that saved in a separate location.
I just wanted to clarify. Thank you. There's another question. I just had a specific question regarding Thinkific, I think if you-- I saw-- I see online when you sign up for that, you end up with something you pay a subscription for, and talking about payment is a more serious question.
And you input the training information that you want, try to move on, and get some information-- or is there some additional, I guess, things that come on to support integrations and the like? We create the content ourselves for Thinkific. You can-- I think Heidi's going to talk a little bit later about the other option of using a content creator, but for our site we create that ourselves. Part of why we want we Thinkific is because it makes it really easy for us to do so.
I feel like it's a really user friendly site, which was important for us. Let me just jump on that. I mean, I'll say that two years-- the last time we were alive I went to the LMS training, and I was very confused. And so I started looking at different things, and I started talking to my husband about, this is what I'm looking for. And he said, you're actually looking for a training platform and not an LMS.
And why we like the training platform and why we went in that direction was we just needed something out of the box. I didn't need anything fancy. We needed something easy for us to use.
The tutorial videos are there, and there's lots of different options. And again, we come from a small legal services program. We're only $8 million. We don't have a lot of resources, and we had to find a cheaper solution to be able to do this.
And with Thinkific, like I said-- and there's other training platforms out there. All we pay is an annual fee to them for as many users as we want. We can pay more for more reporting capacity, but the price is really not that high when we look-- and it's not charged by user, and that was really the option. So I think at one point we have-- I think we have, like, 400 or 500 people registered on our site, and we used it a lot during COVID. And to the point about being able to change slides, every time you had a change in eviction, something or other, all we did was we just ripped out that slide, replaced it, and then it took-- it was me. It took me, like, 5 minutes to do and upload, and then we were done with it.
So that's part of why it's-- like I said, I think as you're looking for platforms, just know that there's different types of platforms and know what your goal is, ultimately, in what you're trying to accomplish. And also what your tech capabilities are because that's the difference, right? And some of your programs are huge. You've got that support. We're small.
So we don't have a lot of support. Sorry. We're going to go on to John's demo, which is a little bit demo-ish, a little bit of each, and we're going to talk more about content creation and have more times for-- love having some of these interactive questions too. Cool.
So another viewpoint. So I call this a demo-ish because I'm not going to go to my live website and fumble around. There's too much to show. Rather I captured some slides to basically talk through some of these things at a high level. So CALI's a nonprofit, and our membership is law schools.
The product that we want to make are teaching materials written by law faculty who then make them available in small bits and chunks. A typical CALI lesson takes somewhere between 15 and 45 minutes to run. We found that at that point the students get up and go to the bathroom or make some coffee or we lose them after that. They're organized by subject topics in law school, obviously, but-- oops. There's a link there.
First year topics and things like that. And the students can either come to the website and find their own thing-- there are subject outlines that are like master syllabi-- and they can find the topic that they want, or faculty can find the lesson that they want and insert into their LMS at their school. We're not trying to be an LMS for the law schools.
And there's also lessons-- most of the lessons are about some sort of doctrine area of the law. But what we're expanding into learning about learning, so what we call law school success. These are brand new lessons in the last year or so. Thinking like a lawyer, time management, words matter, things like that. Now, a typical lesson is some expository text, a hypothetical, and then the student gets peppered with multiple choice questions. Does that sound like a socratic dialogue? Because that was how CALI got started 40 years ago was that law professors saw programmed learning and said, that looks like socratic dialogue where I ask questions and then I might change the facts around and then ask the questions anew.
So this is a terrible slide, I realize, at this distance, but basically all of our lessons-- our lessons are not linear. They branch and they loop based on the author saying, I'm going to give them an easy question. If they get it wrong, then I'm going to jump to a remedial branch. I'm going to give them a hard question. If they get it right, I'm going to ask them follow up questions to make sure they got it right for the right reason.
So we're trying to get faculty who are expert teachers-- we have gotten faculty who are expert teachers to imbue the lessons with steering students into the potholes because failure is when the student learns, right? The goal isn't to get the right answer. The goal is learning when they're writing a CALI lesson. All right? So this is what a lesson might look like. There's a hypothetical. There's a yes, no or a true, false sort of question. There's all sorts of varieties of things.
And these are all written in a tool called CALI Author, which is also a web based tool. CALI is also the developer of A2J Author, and in fact, a lot of our thinking about how we develop A2J Author came out of how we developed CALI Author. We found that law faculty have to walk law students through a complicated legal topic has some similarities, at least in how you build the dialogue, to legal aid attorneys walking a self represented litigant through their form filling problem. All right? That was a huge insight. And it kind of worked.
You can upload videos or insert them, but we're experimenting this year with the idea that-- so we have two constituents-- actually, we have three constituencies, right? We have the students who run the lessons. We have the faculty who either recommend or assign them. Sorry, the faculty who wrote the lessons, and then we have the faculty who assign the lessons. And so what we want to do is let faculty insert their own commentary videos inside of CALI lessons. So if they disagree with the author, how they answered the question, they could say, well, you know, Professor Smith is a fine person, I'm sure, but I have my own opinion on this, and for this class you better get it this way. The point there is that even wrong answers are teachable moments in law school.
All right? So when students run the lessons, only they can see the scores. When faculty create a lesson link, then they can see the scores in both aggregate and individually. They can drill down on that even further and get a list of students and how well they did. And then they can sort this and see the people that are all green got all the questions and the people that are more red than green got the questions wrong, and that might be an indicator that they should intervene with the student. It's called formative assessment.
We want the instructor to be informed about how the student is doing. Even better, we want the student to be informed about how they're doing. And so there's a score that motivates the student. And then the weird one. We want the instructor to maybe think, huh, maybe I better look at my own teaching because nobody seems to be getting-- this is the last one. Nobody seems to be able to get in this topic very well.
And so maybe I need to fix how I lecture on that or on what readings I give for that. Right? So we're going to talk a little bit more now about developing the content. Next slide, please.
Next one? Yes. Thank you. OK. I wanted to start by talking a little bit about just what skills do pro bono attorneys need to serve their clients, and then also how do you determine what those skills are.
Sorry. Obviously that the skills your attorneys are going to need is going to depend a little bit on your particular pro bono program and what you're creating the training for, so going back to what is your goal and what do you hope to do with your training. But it's going to likely contain some sort of substantive training or training on substantive skills, and then also training on some practical skills. So substantive skills meaning that obviously your attorneys are going to need to know some specific area of law, and then also need training on practical skills like how to communicate with clients in plain language or how to work with low income clients.
But when looking at creating a course focusing on the skills your attorneys need, we looked at how-- for somebody like me who's creating this content, how do I determine what these skills are? So we didn't necessarily want to have to create a full course for all the different topics that we wanted to create trainings on, but rather wanted to focus on the skills that were most important or the gaps that we were identifying. So the question that we had is, how would we do this and how would we identify these areas? And so one thing that we did was we spoke to the attorneys that were already volunteering at our programs to find out how they felt about training that they already had received from us, where did they feel like they would like additional training, how would they like that training delivered, how were they using the materials that we had in terms of the videos. Also for our substantive materials, our written materials, were they using those, would they like more of them. And then also I looked at the trainings that our organization had already created, what some of our subject matter experts had created, what they felt was important when they had gone out and done trainings, and tried to put all of this together to look at what might need to be included in the training and then test it once we put it out there. So I think that's what I wanted to start with, and I think Heidi has some more specifics. Right.
No, it's on that side still. So when it comes to developing our content, we wanted to give you a little more practical example of how this actually played out. That information is all great. But when it came to our signature projects, which in Georgia Legal that's statewide clinics that travel all around, we wanted to make sure that attorneys from anywhere in the state could view the course and prepare themselves for a clinic no matter where they were helping.
So substantive knowledge was huge, but also in a clinic setting we had to make sure that they had basic DEI training, they had to be prepared for what they were going to see in this one day because that could be very overwhelming. We've had plenty of clinics where the attorneys are like, I help three people a day, usually, and you just gave me 10. And so we looked at it from a perspective of if somebody is only going to be doing this for one day, what do they need? And that led to the templates, the substantive training, the anything that would surprise them or need to be prepared.
And as we do more and more of these events, we're learning more and more about what kinds of things might be more helpful. And there's just a little practical example. And I also want to add to that, too. One of the other things that we did was actually look at the program where the attorneys were going to be volunteering because our particular training that we developed for this-- or for the Thinkific one, the example that you saw there, was for attorneys who would be volunteering at our self-help center. So they were going to be providing limited legal information and legal advice. So just very brief information there.
But I wanted to know what did that actually look like. So what types of questions were they going to be getting, what setting with the information be delivered to them, what did that-- essentially, what was that going to look like for them. And I think that was actually very helpful because I think it helped narrow down what the training actually needed to include. And for example, I realized that they were getting a lot of questions on forms-- where were the forms located, how do they need to fill out the forms.
I realized that the training needed to include more of that information than some of the other substantive information. So that helped us determine what content needs to be created, and then also what needed to be in the actual video training versus what we might want to provide in some additional materials like a substantive modeling. And then we also, in looking with the practical skills, wanted to look at ways that we could possibly integrate the practical skills training into the actual substantive training itself.
So when we were looking at how can we develop content where they're getting the substantive skills training, can we do it in a way where we're actually using that practical-- or using that plain language, and then highlighting to the attorneys, this is the language that you could also use. You can mirror this language when you're communicating with parents or with the clients so that you're sort of getting both of those trainings at the same time to maximize what they're getting out of that content. The reoccurring message here for everyone is, know what reason you're building the training that improves the LMS or the online learning system that you're doing, and that improves the content of your training that you're developing. Cool.
So there's a tension. Every training is unique, and it's purposeful for the situation. It has to be carefully, artisanally crafted for just that situation. On the other hand, there's 50 states all doing lots of training on a lot of the same topics. Is there enough overlap such that some efficiencies of scale or economies of scale can be made here? And so with CALI, no one law school can write its own lessons, and so it's sort of an open repository model.
There's 1,200 lessons that have been written by, oh, about 300 faculty. And we let faculty, if they want, come in and make a copy of a lesson and make changes to it, and then republish it. Now, the interesting thing there is most faculty don't want to do that. They're perfectly fine grabbing what's off the shelf and either recommending or suggesting it, or worse, ignoring us entirely.
Right? But in the legal aid space there is no CALI, to say. Go to the next slide and I'll do it this way. So the reason for the value of an open repository model is some people are just going to want what's on the shelf, but other people are going to say, I need to tweak this because things have changed or things are a little bit different.
So it's sort of like you need a GitHub for training or something like that. That's where the collaboration and sharing comes in, but I wanted to talk instead about what I call the fellowship model. So we don't just go to faculty and say, here's some money. Write us a lesson.
We instead do an open call. We usually get 20 or 30 faculty who apply. We pick five of them who have usually more than 10 years experience teaching a specific subject area. Think, torts, crim law, even the academic success people. And then we bring them together, and to make a very long story short, they write 25 lessons in six months. When they write a lesson, they round robin review each other's lessons.
When they're done writing the lesson and we think it's OK, we send it out to something called an editorial board, the CALI editorial board. And they do anonymous reviews of the lessons, and they're brutal. They want good stuff. And then we publish it.
But what we found is having five people write five lessons each is they share all of their best ideas, so all the lessons are imbued with the value or the smarts of the five people, not just the one person. And they also negotiate which are their favorite areas to teach and what they want to do. And that model has worked fantastically well for us. Our stuff is-- I hold it up to-- it's very high quality material, and they get it done because there's also a social aspect or an ego aspect of competing with each other, or at least of not letting each other down when they're part of a fellowship over six months.
And oh, by the way, we pay them for their lessons. We tried to do, years ago, a volunteer donate your time sort of model, and it just doesn't work because you can't hold someone who's donating your time to any sort of deadline. You got to pay them money. We also pay our reviewers small amounts of money, but they're doing a lot less work. No worry licensing.
So we publish case books and textbooks as well for free. We pay faculty, again, to write books and then give them away as ebooks. You can still buy the paper version of the book, but it costs $20 or $30 because paper. You can download not just the PDF or you can view it on the website, but you can download the Microsoft Word version of the book, and that allows for agency or for collaboration and sharing. So if another faculty wants to adopt one of our books, they can download the Word version and rearrange the chapters or even put comments in the marginalia. Chunks of learning.
So I mentioned that our lessons are all, like, 15 to 45 minutes. That's because you've got to keep things short. In the '80s we used to have lessons that would take four or five hours to complete. Obviously there was lots of saving of space, and I'll come back to it, and it would take a week to finish a lesson.
So things just got to be small for today's audiences. The last one is, how do you do this? Well, the way CALI does it is a whole bunch of law schools give us a little bit of money, and we've been doing this for 40 years. And we've developed tools that we then turn over to the schools and say, if you want to use them, then we develop-- we publish the materials and then they make use of them.
But we act as sort of stewards of the process, but not the subject matter experts on the material. We always have to go out to the membership or to the subject matter experts. And I got to believe that if all of us work together, we're all smarter than any of us individually if we can find the wherewithal to collaborate and cooperate on a consortium of teaching materials. All right.
I'm going to talk about working with a professional content development specialist. For our TIG grant we ended up working with another vendor called ThinkingCap. I've linked them on this slide.
They were fantastic to work with. I should preface this. The reason that we got the TIG grant was to build an LMS and test it with three different user groups, staff, general public, and pro bono volunteers. So everything that we did we were trying to experiment with those different ones.
ThinkingCap specifically helped us work on things for the general public and pro bono volunteers, so our external facing work. This also was to develop SCORM materials, which are those interactive powerpoints I mentioned earlier. Working with them was great because they're professional, they know how to use all the software already, they don't have to be trained, and they can do much more with it than we can. It's like somebody who's just learned what excel is and somebody who's been using it for forever. They identified a lot of technical jargon. In developing each of our courses for those two groups, we worked very hard to use plain language and to make things accessible.
They were still able to tell us, this is not going to work for somebody. This is still too complex. There might be a better way to say this. They helped us identify a lot of information and make our courses more succinct and more accessible to our audiences. Their quality was phenomenal.
I highly recommend it. ThinkingCap all the way. Link is in the slide when you get it. And then also they had access to additional resources with the SCORM files and those powerpoints. We had somebody who was narrating over it.
They had access to the professional narrator, so in our contract was that they would provide that. On the staff side, when we created cases we did the narration. And we did a good job, but it sounds like it.
You can see the demonstrable difference. You've seen them all. It's true. The cons.
It's very expensive. One of the reasons we chose to do the external facing was because they could be branded, they were going to be facing the public in some way or another, and that's how we did it. We also did need to experiment with how sustainable it was. It requires time and strong communication with the vendor. I think some people assume if the vendor's going to create it then you don't have to do a lot of interaction with it.
But if you want effective courses, you are collaborating with them. You're meeting with them regularly. You're doing multiple drafts. Because of the nature of the courses we were doing substantive work, so that meant that we actually had to do a lot of the content creation on the front end, and then they helped us make it a lot better. And the editing materials afterward can be difficult, especially if you're doing narration. One of the first pieces of advice they told us about was if you are going to use narration, make it so that the narrator is only saying generic things that are probably not going to change.
And then if you have text you can always go back into the file and change the text. So when it comes to specific law or talking about that, maybe don't have a narrator. Maybe just have the narrator say, here's more information about the current law. And so navigating that so that the courses are more sustainable and have a longer life is also something to consider. I'm just looking at the slide where my notes were.
It's expensive. The staff side, I'm going to say, cost about a sixth of what the content creator was. I also do think that the content creator work was-- it's going to last longer.
It's got more behind it. It's been vetted more. It's been tested more. If you have an opportunity to use one, I highly recommend taking it. Yes? Have any of you used some of the recent technology like text to speeches that sound really good now? Yes. And also the animated version where-- No.
Yeah, yeah. --they have people talking, a model talking. You just add the text and they speak. It's better than clutch cargo.
Yeah. Oh, boy. That's an old reference. Sorry.
Yes. We're actually working on an experiment where we're feeding our books on our lessons to a TTS, a text to speech system. I love throwing around the acronyms. And it uses SSML, Speech Synthesis Markup Language, which is getting very sophisticated, even to the point where you can add a tag that says breathe and the text to speech will pause every so often to sound like it's taking a breath. So the verisimilitude of listening to a real person get better. And the cool thing about that is it's cheaper than live narrator-- a lot cheaper.
Because you just refeed it the text and then it regenerates it. And I have a feeling-- and we're only a couple of years away where that's going to be just everywhere. Apple has already come out with it, by the way, for a bunch of audiobooks.
So it has two big things going for it. I mean, I have some bad things for it. But the two good things for it are it's too expensive to do human narrators for low use books-- for any other that's not a bestseller. And that's education and law if ever there were two low use books, right? The second thing is-- or the bad thing about it is it's to the point where you can you can still tell it's a robotic voice, but I think we're at the point where you're not going to be able to tell pretty soon and it will be just fine.
Oh, that's right. The other good thing was accessibility. It makes everything into-- everything that's text or visible into audio.
And that's great for the blind, the hard of hearing, or for the-- this is weird. Some people love to listen to these things at one and a half or one and three quarters speed. You know? And also you can listen to it while you're walking the dog or at the gym or something like that. So we're thinking, we don't want to replace what we do with TTS. We want to enhance what we have with text to speech. Sorry.
So I just wanted to talk a little bit about how we have tried to engage our users from the beginning of the process, getting feedback, and then testing that. So the first emblem on here, how can you engage users from the beginning? I do think it's really important to try to talk to those who are going to be using your training and actually find out, like I said earlier, what is it that they actually want, what are their training needs. And then also being open and listening to what they actually have to say. We learned a lot, I think, from our volunteers, and some things that I wasn't expecting to hear. For example, we used to have trainings where there was a person who would talk on it.
There was music. It was a little gimmicky. But almost every single attorney I talked to told me that they hated that. That they thought that was childish.
They did not want that. They did not like it, and to get rid of it. And they actually preferred to have something that was very simple that just gave them the information, and they just wanted to know what they needed to know.
They didn't want any of that. I had the chance to speak with an attorney that had a hearing impairment and told us that some of our trainings were not accessible because there was some music in the background where somebody was speaking, and that that was actually painful for him when he was trying to listen to it. So we got some insight from the beginning that was helpful so that I didn't have to fall into the same traps as I was trying to create content.
And so that was helpful. And then, again, trying to go back to the attorneys throughout the process and making sure that we're checking back in as we go. And then for this last one point on here, can you build opportunities for ongoing testing and feedback into the training. For us that's important because by being able to build that survey into the training, I'm able to constantly check in and adapt and alter the trainings as we go.
And so it's a way to sort of continue to make things better. And I just would encourage those that are creating content to see if there's a way to do that, and then set a timeline. if you're going to check in like every few months or so to-- if you put that in there-- excuse me-- to make sure that you're actually going to check back in. All right. It might seem a little out of place right now, but I promise you it's not.
Whenever you're building a new LMS or developing content, you have stakeholders that you're working with. You have your project team and you have stakeholders. The content or the LMS will not be successful unless you are engaging your stakeholders throughout the life of the development of the project.
Stakeholders are anyone who is impacted by or can impact the project. It is a broad group. If you think it's just two people, I can almost guarantee you you're wrong. It's essential to project success.
If they are included from the onset-- your users, for example, could be a stakeholder group-- they will make everything better and more accessible and more sustainable because of that. You've heard of it, I'm sure, in DEI practices. I'm sure that you've thought about it with the team.
If you're making housing courses you should engage the housing attorneys. But do keep in mind that there are other people who have other ideas, and you don't have to have every individual person involved in a testing process. You could have representative groups. For example, when we were trying to do an assessment with Georgia Legal on a very large scale, rather than have every managing attorney, we had a representative for managing attorneys, and it was very clear to them. You are their representative. You should be talking to them about this and getting feedback outside of it.
So yeah. It's important because of the feedback that you get and how it improves your system. Testing takes a little bit longer, sometimes a lot a bit longer, but it will make the project more effective long term.
How do you keep them engaged? You're not going to have meetings every day. I mean, you could have a stand up, but I do think that's difficult for a lot of organizations, especially if you're over a wide geographic range. Make sure that you have frequent communications, however, and that it's clear and that it is pointed to the purpose of what they need. Not every stakeholder needs the same information. There is a lot more you could say about stakeholders.
This is not a presentation about stakeholders. Just know that you have to identify what information is necessary for each stakeholder, where they can best contribute, and give them access as they can get it. Sometimes you need to repeat the information a couple of times, but it has to be easy, accessible, clear, and frequent.
So I get to talk about using consistent templates. And the point here is-- here, I get to use my-- show you my T-shirt. It says it's not rocket surgery. Right? The point is your training material, your teaching material is there to teach, not to give users a brand new interface that they have to figure out to learn. So if you use a consistent template, then the next time they come back and do training on your site they're going to know where the arrows are, where the Save As is, where the-- whatever it is the buttons are, they're always in the same place. So the other thing is don't start from scratch, right? If you're creating training in almost anything, chances are somebody you know in your community has done something similar, and there's nothing wrong with standing on the shoulders of your friends, of your giant friends, to mix a metaphor.
So slavishly copy from smarter and better people than you, at least as a way before you get started. Everyone is doing similar training but not identical. And I hate to say this, but don't be clever. It's not art. It's training.
And your users will appreciate not the thing that you spend 3 hours picking the right music or making the making things flash or bubble or whatever it is. It's sort of like-- I guess I'm saying, notice how there are no PowerPoint animations that we've been seeing in most of these presentations because we're all sick to death of them, right, ? So don't be clever. All right. So just kind of in conclusion and then we'll have some time for questions. We just want to talk a little bit about sustainability.
And do you want to start, Nalani? Sure. Yeah. Yeah. I'm the cheap one, right? Sustainability was always top of mind for me. I mean, and I think that we all have to think in that way when you're looking at these platforms.
TIG grants are fabulous, but they only last for so long. So I would just say when you're looking for solutions for your problems-- and that's what we did here. We looked for something that we could afford and we knew that we could sustain.
And again, we haven't turned on the purchase capabilities yet, but that's kind of our next step. Really trying to see if we can get it to be self sustaining, at least to cover the cost of the platform, and then also hopefully save our staff time in things that they're being asked to do over and over and over again. So again, sustainability was where we started our journey, and it is something that I feel really confident that our program can continue to maintain because we've thought about it all along the way, even in the building of the powerpoints to be able to just replace slides.
And again, this has just been kind of the constant thinking. And again, we got low resources so we've got to find ways to make it work and continue to be around. I want to echo the comments about start with sustainability.
You don't start a project and then find out where you are at the end you need to start. If you have a TIG grant, for example, you know what the end costs are going to be. Figure out how you're going to keep that in your budget. Figure out which programs you're using if you have to divide it between a ton of grants. And I would say another thing for sustainability and going back to stakeholders, it's buy-in.
If people aren't invested in your product, including your staff, including your clients, you're not going to see a reason to keep it up because they're not going to use it. And then you're going to try and spend money on marketing to try and get it out there. It's not easy to catch up once that falls behind. Yeah. And I think when you're thinking about sustainable and you're thinking about building this project, think about what can you maintain in the future. So you might want to submit a grant proposal and have, oh, we want to create all of this online content and 20 courses and all these things, but is that realistic to be able to sustain in the future? So think about what-- maybe scale down on the front end so that you can then keep it up and make those changes in the future.
And I'm here to make the pitch that this has already been done in legal education, and maybe there's overlap in some of the things that you-- the training materials you create for your associates, for your social workers, for your community, with what law schools are doing with clinics, with what law schools are doing with their law students. And maybe we should work together on something like that. I mean, what we want is a law library of materials. Now, you lose some things when you collaborate. Right? You can't go harrowing off and doing your own thing.
But remember, I said don't be too clever. But maybe this is the place where we can sew together a tapestry of collaboration with legal aid, legal education, and even courts in the training space. Anything to add? No. I think that's it. All right.
I'm going to come around with mics so everyone can hear the questions too. Hi, everybody. Hello.
Test. All right. I wanted to kind of rewind back to ThinkingCap.
And the reason why I ask it is because I've recently been tasked with looking for an LMS and learning platform, and I came across a platform called Moodle, which is free, completely open source. However, you're basically on your own when it comes to that. With ThinkingCap-- There are Moodle consultants.
I have a lot of them out there. It's huge. OK, OK. Well, that's good to know.
But as far as ThinkingCap and having someone that actually creates content, because I tried creating a couple of platforms and I realized I'm not as creative as I thought I was, so you say it's expensive, but how expensive are we talking about here? So it's complicated and it depends. See, I've been in law for a while. So the reason that it was as expensive as it was because of how many courses we were creating, because each of them got a substantial number, and because of the time for each thing. And it was a SCORM file.
So there is no rate. It really depends and is impacted by the product that you want. If you want fancy powerpoints that are just said that, then it's going to be cheaper.
If you want something that's super interactive and you want to make sure that it's got questions in the middle and it's got narration and-- he's very good about-- Tim is the guy we worked with. Hi, Tim. He's very good about walking you through all the things that will impact the price, but I don't feel comfortable talking about one because of the fact that it's so tailored. And just as a follow up, these paid platforms, are there, I guess-- I guess, are people that help you create content on the main platform? So most LMSes that we looked at did have options to create some content.
A lot of them had it built in. But remember, they're learning management systems, not content development systems. There are systems-- like Articulate 360 is very popular for SCORM files, and there are other softwares that you can use. But if you go on an LMS, it's not going to be very fancy. If you want to do content creation, I suggest looking for a specific content creator.
Don't be fancy. Don't be fancy. Be fancy. No, I'm just kidding.
Thank you for a great panel. Really interesting. I'm Adam Stofsky, the CEO of Briefly.
So we use Thinkific. My question is for the Hawaii crew. I'm thrilled that you're using Thinkific. It's an incredible value proposition, right? It's really amazing for, whatever-- I don't know what it is now. $150 a month or something like that.
It's really incredible. My question is specifically about just sort of the-- so you did this without any professional support, right? It was just you doing it? Yeah. Katie took it over so Katie does it now. But I learned it.
We had to initially learn it, yeah. And the idea of monetizing your content through Thinkific is brilliant. I think it's doable. I don't know if you're going to get rich on it, but being able to sustain it I think is totally doable. My question is for legal aid organizations who want to use this, what level of tech savvy do you think you need to be able to do all this yourself? Both the kind of just using it, putting in your content, and then maybe some of the more complex marketing, app ecosystem aspects of Thinkific.
Is it the kind of thing you think pretty much any modern legal aid organization can do? Does it require some something special in terms of technology? I think that it's very accessible. I would not consider myself the most tech savvy. I think if you can use PowerPoint and if you can record audio on PowerPoint, you should be able to do it. There are some tutorials on Thinkific. It did take me a little bit of time to learn it, but I thought the tutorials on there were helpful.
So I think if you take a couple of weeks or so to play around with it and figure that out, then you should be able to do it. That's part of what I like about it. You're able to drag things around, move things around that way, and I think that you could very easily have multiple people take it over without having to have a really big learning curve. What's your marketing-- your market for the monetization? Is it long term? Like, something-- Social workers. That's my market.
I know it sounds crazy, but I mean, that's the group that's always asking us to do trainings, and they need their credits. I mean, at least for our state, our lawyers only need three credits so it really isn't worth monetizing it for our lawyers. But for our social workers, they need 20 hours of credit. Charge them $5 a piece, $10 a piece for 30 minutes. I mean, you know-- I know, again, it's not to make money. It's just to pay for the platform and maybe to pay for some of the content creation in terms of our staff time that's already being put in this area.
So that's kind of the-- at least that's been the thought behind it, and we're getting close to being able to launch. We actually have a whole set of trainings that we're going to hopefully launch in the next couple months that we're going to charge a small fee and see what happens. I think it's brilliant. There was a question back here. John. Have any legal aid or other service providers ever made use of your platform? Yeah, we got a couple of legal aides that are members.
Right now it's free to be a member of CALI, so if you're at a legal aid you should join CALI. Now, you can't produce-- you can't create your own materials with that, and that's sort of what I'm here to say is we're going to start looking at that because we can't give it away for free and support a community of practice doing that. So I'm looking at a membership model for legal aids and courts that could let me put resources on the specific community that would use it. I just looked at your website. I didn't see any kind of demo or sort of legal aid areas.
Is it possible you could do a public demo or two for those of us who are uneducated in it? Sure. Be happy to. Cool.
I'm interested. Hm? The LSM tab? Yeah. LSM tab.
Thank you. So this is a question for Nalani and Katie, and it's about sustainability in terms of staff because you guys have staff developing your trainings. And I would imagine if we did that, it'd be similar. So I get that basically, Nalani, you said you created them. Katie's doing it now.
Do you have-- what is your long term plan for doing updates, new training modules, that sort of thing? Yeah. I mean, we did get a PBIF to do a lot of this new content for along on the-- at least for the work that Katie's been doing in this particular area. But we actually-- our staff is always creating trainings, right? I mean, when you start to go back into it and we started to look at-- I think to what John said, I actually had somebody two years ago look through every training that we had within our program. You know how many eviction trainings that our managing attorneys in each one of our offices did? I mean, we had so much stuff, right? And so I think a lot of it is trying to figure out a way to have it in one platform so that they can come back to it, and then making sure that we continue to embed that.
And so we're working with doing some-- we're doing some reorganization within our entire organization to put a lot of it under more of our intake self-help area to make sure all that content is managed better with the idea that then they can reach out to different people. But again, there is-- I keep thinking about it in the framework of just how much-- if you really looked around your program, how much your-- I mean, every one of your managing attorneys has probably done, like, three trainings on domestic violence and four trainings-- I mean, there's so much content already. It's just making sure that we can find it in one place and then continuing to maintain it on a regular basis.
At least with respect to the training piece of it, right? I think it's a whole different space when we start talking about legal information for our clients. But I think all of it can eventually flow, and that's kind of the goal, right? My theory has been you create really more detailed materials for your pro bono attorneys that then can be translated for your social work audience that then can be used for your client audience. So it's really kind of this pyramid scheme of starting here and willing it down in a way that you're reusing content constantly. And if we can model that, I think we'll be in a good place. But again, it was starting with that whole sweep.
We swept the entire system for everything we could find, and that's part of what Katie was utilizing in starting it out. And then just in terms of maintenance then, you said you were going to reorganize to potentially have this be under your intake self-help. So are you going to designate a staff person who's responsible for that, or is that everybody kind of has some-- yeah.
Yeah. That's Katie. All right. I think we might have time for one last question, otherwise we're just about at the hour.
If not, I would just thank our panelists today. Thank you all for coming. Be sure and complete your evaluations too.
Thank you.
2023-02-07 07:30
