- Well, hello, welcome everyone. My name is Adam Brown and I'm the program director of the Science of Learning Research Initiative, SOLER Of the Office of the Vice Provost for Teaching, Learning, and Innovation. And I'm very pleased to have the opportunity today to tell you about the Provost's SOLER seed grants this program that we launched last year to support faculty research projects in scholarship of teaching and learning, and we're going to be expanding it and continuing with this year and very excited to tell you about that expansion. Sorry, give me one second. Anyways, I want to introduce myself just for a moment at the beginning 'cause I think it will make SOLER make more sense to you. So I have a PhD in neuroscience from the University of Chicago and while I was a graduate student, I got very interested in the scholarship of teaching and learning.
I was a graduate fellow at the Center for Teaching and Learning in Chicago. And I knew after I finished my PhD that I wanted to work in scholarship of teaching and learning in higher education. And when I had the opportunity to apply for this position with SOLER I knew it was the perfect opportunity to combine my scientific perspective my perspective as a scientific researcher with my interest in teaching and learning.
So that's really the core of what SOLER is about. Combining the research perspective with questions about students' learning experiences in higher education. And today we're gonna hear from two faculty members who have collaborated with SOLER over the past year, Brent Stockwell and Alfredo Spagna So we'll start off talking about the origins of SOLER then about this amazing faculty advisory committee that helps guide SOLER as well as our key partnerships around the university.
We'll talk about this thing called DBER, discipline-based education research, what it is and why it's important. Then we'll talk about the Provost's SOLER Seed Grants program, our main focus today. And then I'll answer a question that many of you may have which is what about IRB, the Institutional Review Board.
If you're not sure what that is, we'll talk about it soon. Then we'll talk about what's coming up next for SOLER. What are our future plans and how are we gonna grow. And lastly, we'll hear from the two faculty guests that we have today, Brent Stockwell and Alfredo Spagna. They'll talk about their projects that they've worked on with SOLER over the past year.
And we'll save some time at the end for Q&A. So to understand where SOLER began we have to go back to 2014 when this organization called the Provost Faculty Committee on educational innovation convened and recommended establishing two organizations. One, the Center for Teaching and Learning, which was established in 2015. And I'm sure many of you are familiar with the CTL and all of the amazing work that they do. They've grown rapidly over the past six years and become such a major force for improving teaching and learning at Columbia. And then to compliment the CTL this committee recommended the creation of a research center devoted to building capacity in the learning sciences.
So a key excerpts from that report said that, "Teaching itself should become an object of research "ideally in collaboration with learning sciences "and educational researchers." So SOLER, the Science Of Learning Research initiative was established to be a vibrant hub for cross-disciplinary research that advances the scholarship of teaching and learning. So again, that intersection of the research process and SoTL scholarship of teaching learning. And SoTL was established in 2018 and became fully operational last year around the time that I joined as the program director.
So SOLER is advised by a faculty committee, which is chaired by Soulaymane Kachani, who is the vice Provost for teaching learning and innovation and by Janet Metcalfe, professor of psychology as well as about a dozen other faculty members from across the university representing many of the different schools and departments. So as I alluded to earlier SOLER and CTL are kind of like siblings under the VPTLI and SOLER team is rather small at the moment and may grow in the future. But it consists of me, the program director, Sandesh Tuladhar, who is assistant Provost for online education and Merrell Norden who is the assistant director of online education.
And SOLER collaborates with a number of important offices around the university, the office of research compliance and training, the institutional review board which we work with in developing umbrella protocols. We'll talk more about that later. The office of general counsel and of course CUIT because doing educational research entails the collection of sensitive data. And so we work with CUIT on collection storage and data compliance protocols.
So let's turn our attention to this concept of discipline-based education research, DBER, and talk about what it is and why it's important. So, as I said earlier it is the formal research approach to SoTL. So what is SoTL? SoTL is and we can use this Venn diagram to sort of understand how these different concepts fit together. SoTL is the intersection of the advancement, teaching and learning in higher education and particular disciplinary content meaning learning objectives and skills and ideas that students are learning in the context of particular courses and majors. So SoTL is that intersection in black. And then when we take SoTL which is quite a broad, diverse but online many different ways to approach the scholarship of teaching and learning.
We take that and combine it with the methodology and the frameworks of social science research, social science 'cause we're talking about human behavior, human cognition. We have the intersection of all three, which is DBER. So another way of thinking about it is that, education research is what you get when you combine teaching and learning advancement with social science research. And when you take that education research and place it within the context of a particular discipline, you get DBER.
So for more explanation of this kind Venn diagram framework you are invited to check out the SOLER faculty guide and I think we'll drop the link to that page in the chats. And this is a repository of big picture ideas and best practices and resources for DBER on that SOLER put together and it's a work in progress. So if you have any if you check it out and you have feedback I'd be really happy to hear it from you.
So let's think about why we would wanna engage in this kind of work. Well, we've always had a lot of motivation to wanna understand what works best for our students' learning. But maybe right now there's more motivation than ever because of the shift to remote learning. We're in a new paradigm and it's very crucial to try to understand, well, what can we do to best support our students' learning experiences? But how would we know what works best for our students learning. One approach, and this is kind of the core DBER method is to do an experiments where you compare two pedagogical approaches method A and method B. Which one of these two methods will lead to better learning and attitude outcomes.
Well, if other variables are controlled meaning that the only difference between the two groups is the methods A versus B, then we can actually make a causal inference, right? This is kind of a core idea of the scientific method that if the only difference between two groups is one particular variable method A or method B then we can infer that any differences in the outcomes for those two groups are caused by the difference in method. So that's kind of the idea of applying that core scientific principle to SoTL. And will this kind of leads us to a question of, will we have from this universal insights about learning. So fundamental insights into the learning process. Well, the truth is that learning is a very very context dependent process.
Context dependent because of idiosyncrasies in particular professors, in particular courses and institutions. So context matters a great deal. This means that maybe we don't get sort of generalizable universal insights about the learning process from this work.
However, that's actually more of a feature than a bug I would argue because our focus is intentionally narrow. We want to know about the learning process that is going on in this course in this particular institutional context, academic context and the real results that we want are most relevant to that context and can be used to inform the next iteration of the course. So we're getting information about the learning experience in this particular context which we can use to inform the next iteration of the course. And this iterative process if we incorporate small experiments into each iteration we can gradually build a much stronger course. So over time doing these experiments we can build a better and better course. So to support this DBER process we created the Provost SOLER Seec Grants program to facilitate and support DBER at Columbia.
The goal is to better understand and improve teaching and learning in Columbia courses by doing two things, one, developing an empirically testing the impact of innovative pedagogical interventions. That's the experimental model that I described on the previous slide and also by devising and or implementing learning analytics procedures. This is more on the side of the kind of big data, data science approach, where for example you're collecting lots and lots of data on student engagement with digital platforms like Canvas. And then we have this wealth of data and we try to mine it for insights patterns, and know that you can actually combine these two methods. You can do experiments where you're manipulating some element of the students experience with the platform, and then you're looking at the data generated through that engagements. So they're not mutually exclusive at all.
And these projects are driven by faculty PIs, which could be in some cases, interdisciplinary teams of faculty. And this is really crucial because it's emphasizing the idea that the whole inquiry the question about students' learning is grounded in the disciplinary expertise or interdisciplinary and some cases of the faculty members. The questions are about addressing specific learning objectives. And it's the expertise that the faculty have in the discipline that anchors the whole enterprise. That expertise gives insight into what is it that students need to learn in this particular context and how can they learn it? So this is the deed in DBER the discipline. And to support these projects at this time it's only a modest amount of funding but expensive in kind support.
So that support involves framing the research objectives and matching the study designs to the specific aims, developing or refining methods of assessment or measurables, devising methods for data analysis and visualization, preparing materials for IRB approval, we'll talk more about that in a moment, and developing a plan to disseminate findings, usually in the form of publications and conference presentations. And as a secondary goal, we hope that these projects will provide pilot data for major external grant applications from organizations like the NSF or the Institute for education science. So think about this idea of a seed grants. Well, what are we trying to plant? We're trying to plan something that will grow into an ongoing line of inquiry and the best way to support that ongoing inquiry is with not just continued support from SOLER but the enhanced resources that would come along with a major external grant.
So we hope that some of these projects will grow into these kind of major lines of inquiry with external support. But some of you are thinking, well, what about the IRB? So the IRB, the Institutional Review Board, is an organization that exists at Columbia and at every university to review all human subjects research to make sure that any research that involves humans involves people is adhering to certain protocols and regulations that are mandated by the government. And most DBER projects are subject to what's called exempt review from the IRB. However, don't let that term fool you this is human subject research.
So it's not exempt from review. It must be registered with the IRB but this what they call exempt review, it's a very simple process only a simple consent form is needed. And in the case of archive data, if you're looking at data that came from years ago student's participation in courses years ago you wouldn't even need that consent process. And the review is conducted by one IRB staff member which is a very fast review process. Some projects might be pushed into what's called the expedited category.
Instead it's a more extensive consent form process. The reviews conducted by an IRB committee member not the full convenience committee. So it's a little bit slower view process but still much quicker than the full convenience committee process. And some factors that would push a project into that category would be collection of physiological or physical data like electroencephalography data which we'll actually talk about a little later today or the data not being de-identified for any reason or if there's a risk of breach of confidentiality which could be due for example to a very small class size. If you're doing this research in a very small seminar it might not be possible to protect the anonymity of the students because it would be too easy to match up the data to the individuals.
Again, that's all right, but that would probably push things into the expedited category. And one major thing that I would be as concerned about that you need to know is that in all cases students have the option to exclude their data from disseminate analysis. Every student will be given the option to say, I do not want the data that I generate through my participation in this course to be part of any kind of publication or conference presentation. However, students may or may not have the option to opt out of the study activities themselves. That depends on whether the study activities entail something that is outside of the scope of the normal activities of the course. So if students are being asked if participating in the study entails an extra session which will be the case in one of the projects that we're talking about today they could opt out of that.
But if being in the study just means being in the course then they would not have the opportunity to opt out unless they wanna drop the class. So let's talk a little bit about SOLER next steps. Where are we going from here? Well, one major area is collaboration with the CTL on select projects. We've actually already established something of a model for a project being supported simultaneously by a PSSG and by one of the Provost teaching and learning grants. That's the case with Brent Stockwell's project which we'll hear about very soon, but we wanna establish more of a formal protocol for this kind of collaboration where faculty could be applying for both in parallel.
And there'd be a really clear system for CTL and SOLER too to collaborate on support for the project. So that's something we are developing now. Additionally, we're going to expand our services in several key ways.
First of all, we talked about the IRB. What we want to do is create umbrella protocols. So kind of create a set of different sort of models of projects.
Certain like recurring themes and how the projects are structured so that we can fast track the IRB approval process by saying, okay, this project fits into model A, so it's gonna be a very quick approval process. Do we already have approval for model A projects? And we're also working on developing a learning management system learning analytics dashboard. So we, I alluded to earlier this idea of learning analytics where you're collecting lots and lots of data from student engagement with digital platforms. This is the case across the university with Canvas and with other platforms like NAFTA. So we're working now with CUIT and with the academic technologies leadership group ATLG on developing this dashboard tool which will allow faculty to quickly get access to important insights that they can use in conjunction with this kind of DBER process.
So kind of making the learning analytics data more accessible and useful for faculty. So kind of building these general tools that can be widely use. And the thing that I'm most excited about is the creation of this SOLER fellows program, which we hope will be coming to focus very soon. And we know that DBER is a type of social science research and not all faculty are gonna have the relevant skills depending on what departments they're coming from. So students, whether they would be undergraduates advanced undergrads, or graduate students who have that relevant expertise can do a great deal to enhance the research effort. So the support roles that these students may be providing would overlap with SOLER services, things like designing the research methodologies and assessment instruments, as well as data analysis, more coordinating logistical type work, all of that can do a great deal to enhance the project.
And it's a win-win because where these students are involved in DBER will broaden their training and may lead to publications, which of course is great for students. And I think most importantly this kind of involvement builds the DBER community. It establishes ties between departments and SOLER promotes the idea of DBER as a an important worthwhile and fruitful activity that faculty can engage with and administrative staff and support and students can be part of it as well. And the students will these fellows would be compensated but that money won't have anything to do with the award funds for the PSSG itself it would be a separate allocation. So a little more on the idea of building community and outreach, we're also going to establish a journal club to promote literacy in the field of DBER. So we'll have occasional meetings to discuss one or two peer reviewed articles and participants would be the select faculty committee, of course, as well as anyone who's applying or has been awarded a PSSG people in the CTL and others involved in digital community at Columbia in ATLG et cetera.
So administrative folks as well. And we'll have more information about this coming soon. So next we're going to hear from the first of our two faculty guests. The first will be Brent Stockwell who has a joint appointments.
He's professor of biological sciences and chemistry. He's a member of the Motor Neuron Center and the Cancer Research Center. And most importantly, he's a member of the SOLER faculty committee, which I mentioned earlier and so he's been with SOLER since the beginning.
And he's published work in DBER. He's published articles of his own DBER research already including in the journal cell, which is a very high impact biology journal, which I, again I think emphasizes the idea of the disciplinary grounding of this kind of work and the fact that the DBER work is done is of interest to the biology community. He's also won numerous awards and published many articles in his discipline, of course, and also was the recipient of the Lenfest distinguished faculty award for excellence in teaching.
So thank you so much, Brent Stockwell for joining us and we'll continue now with your presentation. - Great, hanks so much, Adam, for that very kind introduction. So it's great to see everyone here interested in SOLER and education research and teaching. So, I just want to give you a relatively quick overview of the project that we did and we're continuing on this summer and then into the fall as well. But we started looking at virtual reality and whether that would be useful for teaching biochemistry. And actually originally my thought was that in a remote situation it would create more of a sense of presence.
And being with students, sitting around, talking about biochemistry, more like what we experienced sitting in my office in the old days and the before times. And, but what we've learned as you'll see is that the illustrating or eliminating 3D aspects of biochemistry, I think is gonna be one of the important things that we can do in VR in addition to presence and actually having more of a presence. So what we've done so far is last summer we started laying the foundation and applied for the SOLER award and one of the Provosts awards to start thinking about how to do this.
And, there's a lot of groundwork I won't go through and figuring everything out but basically finding the right platform, the hardware the software, and then what, what could we do in virtual reality, that would be beneficial. And then in the fall we actually implemented it. So we recruited students to the study. We offer, it voluntary recruitment. We offered them a small gift card compensation if they participated. And they had to if they participated they would be randomized to the VR or the Zoom groups.
And then we would do they have to do a survey at the end and they'd have to agree to participate as much as they can, obviously, if they drop out it's nothing we can do. I was kind of surprised at the number of people who joined was less than we thought during the study. I think in the end it was about 20, 15 in each group or so, so, and the class size was about a hundred students. So, more students didn't join the study than did.
And so I think, in talking to students and try to understand that I think one aspect was just the scheduling. Like a lot of students were not free at the time. So that's anyway, these are all kind of lessons we've learned and things to think about for the future but the scheduling always turns into to be a big issue.
We, I mean, my thought was that there might be more satisfaction and there might be potentially a benefit in terms of like three-dimensional concepts. I think what we learned in the study was that there is a learning curve with virtual reality. So the students who did the were ended up randomized into the VR group, rated actually the technology as requiring more of a training and learning than they did on Zoom, which I guess makes sense. Everyone's familiar with Zoom at this point. So that led us now for this summer to actually have a training modules.
Oh, here's a demo I can show you if... - Actually why don't we watch the demo then we'll talk about - Let me show you an example of what it looks like. Go ahead here.
So this was the first platform we use, which was spatial where we could import 3D structures or molecules or any really 3D objects. And then there's my avatar. You can see, this is what you're looking at the student point of view and we're looking at this protein together and we're moving around it and we're talking about different aspects of the three-dimensional structure, I can zoom it and make it much bigger and everyone in the room can look at it together and again walk around and talk. And we're in a desert amphitheater there like kind of just enjoying the Vista while we talk about these things. I think there's one more video you have? Yeah, okay, here's one where you can see now I gave everyone their own protein and so each student can actually manipulate their own protein.
And that ended up actually being a lot better because they started to get much better sense of what I was talking about when they had their own model to work with, rather than just looking at a model in the middle of the room. So that's, again, these are all like little lessons about how to do this better. I think some of the insights are shown here. So basically getting a high quality 3D we spent a lot of time figuring that out but basically now we know how to generate protein structures in a way that they can be imported and render them in particular ways to highlight different features.
And that the students in at least in their surveys indicate was the most valuable piece really like looking at proteins which are for them very complicated things and like really see it in three dimensions was it seemed at least intuitively impactful. You know, we still need to evaluate like the impact in a quantitative way, but and that's also part of the challenges that, I've always taught in two dimension. So I didn't really have questions that specifically went into 3D aspects of protein functions. I had no way to teach that really. So now I have to think about what can I ask and what kind of probing questions can we give them that get into that more deeper understanding which you don't want it to just be intuitive. We wanna try to measure that in a way, but that's another challenge.
And then the immersion aspect of really feeling like you're there in spatial, the first platform the software that we use, it didn't have animated avatar. So it sort of felt a little bit static. It didn't have the same presence I was expecting but now we have a new platform called Glue that we're using this summer. We're trying to finalize that and there the avatars are animated. And so when you're talking, your eyes are moving, your eyebrows, your mouth, like your hands are just stipulating.
You really feel like you're there with someone. And also the audio, the volume depends on how close they are to you and his directionality. So it's much more like you're in a space with another person or a group of people talking. There were a couple of issues that came up in the surveys with the students.
One was that it can get uncomfortable wearing the headset for a while it was kind of heavy. So that's one of the reasons we went now with the new headsets and quest two is quite a bit lighter and more comfortable than the original Oculus quest. And so for me, at least that makes a big difference and it's a lot more comfortable.
And then they also they wanted to be able to take notes in VR but of course they can't do that, then have their computers. So we worked out a way to do note taking they were gonna try for this summer, which is towards the end of the session everyone takes off their headset, goes to a shared Google doc. We've got it set up for them to enter notes in a particular section. And then we all come back into VR and we go over the shared doc in VR and then they have that as a takeaway.
So they actually can capture the content from that session. And so, yeah, those are we're still trying to bring in other things like live polling for this summer, we're gonna have some training sessions. So I think still a lot of the focus is on getting the platform, getting the technological pieces there. But then we also wanna do another evaluation and have a test that specifically probes some of the kind of three-dimensional pieces that we hope they would be getting from this.
So I think that's what we had there. Was there another slide Adam or not? - That's it for the slide. So what I'm gonna do now is spend a few minutes just asking Brent a few questions of my own and then we'll have time at the end of the session today for general Q&A. So if you have questions for Brent we'll have time for that at the end.
But Brent, first of all thanks for that great summary of the project. It is really exciting what we've accomplished so far and how much we have planned for the future now I'm starting very soon with summer aid and just a few weeks, but maybe you can tell us a little bit more about what SOLER did specifically to support this project last fall and in the developmental period in the summer. - Yes, SOLER was really key and working with Adam I can tell everyone was key because there was thinking through the strategy of how to implement it, how to evaluate it, how to survey students, getting the IRB done. I mean, I had done IRB before but it always takes just a bit of legwork and as always as a million things to do and Adam was able to really make that happen and then writing up the report and evaluating it. So I feel like it was the missing piece that if we didn't have that, it just, might've never kind of made it to ignition to really be, and not and to have been evaluated in a rigorous way. - Great, yeah, and I also want to emphasize that this project was supported last year and continues to be supported this year by the Emerging Technologies Consortium which is a collaboration of CUIT and the libraries.
It's a really fantastic involvement from Parixit Dave and Vivek Choksi, the two members of each of emerging tech and really great support both in the development of the overall ideas and goals of the project as well as a lot of the technical elements. So it's really a great collaborative team that we have. Brent can tell us a little bit more about your vision for how this project could be scaled up or sort of applied in other contexts in your departments, biology and chemistry and then maybe even more broadly at the university. - Yeah, I think it's the two potentially useful features are this idea of presence of being present with people more so than all of us here on Zoom avoids also the camera issue of are you gonna be your own videographer? You need the lighting, you need the backdrop, you need all this stuff. You know, VR you just got your avatar, you're there.
You don't have to set anything up. You don't have to worry about any of that. So, and then you feel like you are there with other people. So to the extent that would be beneficial for like office hours or short meetings with students, you don't want them to travel all the way to campus, or you're not able to, then I think it could generally be useful. But then the other really unique thing is the 3D objects being able to bring in 3D objects. So I think anyone who has 3D data sets or 3D aspects to what they're teaching could potentially that those would be places where this would be useful.
- Yeah, absolutely, I mean, I have some experience teaching, intro biology concepts myself and I can imagine how useful this would be for students learning things like DNA replication and transcription and translation. All of those are having various spatial 3D elements to them. And I can see how that would really enhance the learning experience. So I think there's a lot of upside for how the art can be expanded, especially in the sciences and maybe and beyond.
We talked a little bit about architecture and other disciplines where that kind of 3D spatial environments is really crucial through learning experience. So maybe on that note, you can tell us a little bit more about these targeted assessments that we're trying to develop for this next iteration. Tell us a little bit more about what exactly is this sort of 3D elements that students need to master in biochemistry. And how can that match with the virtual reality experience? - Yeah, it's so a good example is I do teach this every year and students always struggle with it.
So in the, some of you may remember from your biochemistry the citric acid cycle, the Krebs cycle, there's a step in that cycle where you're going from citrate to isocitrate to metabolite. And the reaction always the isomerization that always goes in one direction on the molecule. Citrate looks like it's symmetric but actually the enzyme always moves there is some ratio onto one side of the molecule not the other side. Students are always puzzled by that because they're like it's asymmetric molecule, it should be random.
And then I can just tell them the answer for that. And then they're like, okay, okay, I got that one but they don't understand the deeper principle there which is that when citrate is bound to the enzyme it's in a chiral environment and the enzyme can actually control that directionality. So that's one of my goals is to be able to basically take a scenario where they don't understand an aspect of a mechanism and show them now on, okay here's the molecule, here is bound. This is the movement we're talking about.
This is how the enzyme controls it. This is why it can be done asymmetrically. And now hopefully rather than just knowing that one fact they'd be able to generalize from that to other contexts where, okay I see any time you have this type of situation I would expect that this could be done in an asymmetric fashion. And then the challenge in evaluating that as you've gotta think of questions that test that more generalized understanding. So that's, you know, it's just a lot that has to go into thinking of these evaluation questions.
- Yeah, it's certainly a really fascinating issue. And I think, just getting the chance to hear you talk in depth about those learning objectives is really exciting because it, again emphasizes the idea of disciplinary expertise and the disciplinary context being such a huge motivator for the work that we're doing. We're interested in how students learn, not just in general, but how they learn specific concepts, how they mastered specific skills. And we can see how the disciplinary expertise of the professor is really such a powerful driving force for the whole enterprise. So thanks very much Brent. And again, we'll have some time for Q&A with Brent at the end of the session.
Our next guest is Alfredo Spagna, who is a lecturer in the discipline of psychology and the director of undergraduate studies in neuroscience and behavior. And Alfredo is a cognitive neuroscientist who is interested in the intersection of cognitive neuroscience and education. So he's really interested in bridging the apparent divide between the laboratory and the classroom. So Alfredo is also working on a project with SOLER and I'm very pleased to introduce him and to give him the opportunity to tell us about that work that he is doing. So thank you, Alfredo - Thank you Adam and can you all hear me well? - I can hear it.
- Okay, awesome. So once again, thank you so much Adam and SOLER for having me today here. First of all, I will just say that what Brent just said and I think that the statement of the holds absolutely true and is whatever you need, Adam will do it. He will solve it and he's able to do everything. So I really, really would like to start by saying that working with SOLER has been so interesting and in great change. We have this idea, I'm going to tell you very briefly what we are going to do.
So I have less data, more the ideas the course we're going to do is going to be delivered in fall. But really working with Adam, especially, but also we have that help from Janet Metcalfe And SOLER in general is just an enriching experienced to really think through how you can make your course how the research component and really study what happens in our case what happens in the student's brain as they're learning. So this is this course is once again. Thank you again SOLER for this opportunity.
This course is kind of the idea these courses can be summarized into one statement which is what happens in the students' brain when they're studying the brain. So what happens in the brain when you are studying neuroscience? So, and this is the circular approach. The circular idea is what really I wanted to do when we're trying to do with this innovative course. So of course I can not be doing this by myself. So I'm working with Xiaofu He who is an assistant professor of Clinical Neurology at UIMC And this grant is supported by SOLER, but also by eight year sided the other scientists collaboratory grant put together by Caroline and Marvin for our department. A year ago, and Getty is the collaborative grant is very big Any particular stories is helping us refine the methodology of what we wanna do in class, which is represented here on the figure on the right.
I'm gonna annotate in a bit, I like to annotate on the slides. So the course is called fundamentals of human brains imaging and somehow is going to be delivery in fall 2021 if it's possible for face-to-face instruction. What we're going to do is we're going to ask our 12 students, so you see, we have 12 students in our seminar course to take a regular neuro imaging course during the manual imaging course we're going to teach them how to collect and analyze data, especially electroencephalographic data.
So EEG data, as well as fMRI on the right data. So functional magnetic resonance imaging course. So you can imagine that without the SOLER component this would be substantially a mirror imaging course in which students walking with their own laptop and then they get some data. They learn how to analyze the data. They have some lectures, so Xiaofu and I will be teaching them the common pipelines how to analyze the data and so on. The major component is that while they are studying EEG we will be recording their EEG signal.
Meaning that we will have the 12 students kind of where this portable EEG headsets, which I completely agree with Brent. These are very light, but having this very light portables for 90 minutes starts being a little bit uncomfortable. So we have an eye on kind of the how this is going to affect or be detrimental for students' learning. Anyway, so the main idea is use portable EEG headsets to study students' neural activity during our seminar and students will be analyzing the same data that they just collected. So again, this circular activity of your study EEG, we explain you what EEG is, as we explained we record your EEG signal and then we give you your own data to analyze.
So you are kind of really experiencing this circular approach. The research design has two components. The first one is kind of trying to study who gets an A-plus, what is attention, what is the behavior of a students in the class? So then we can really understand which are the attentional or physiological or behavioral markers of a student's engagement and learning in class. And to do so, we're going to compare participant performance in a control or a student's attitude or students' engagement in a controlled classroom that we're also going to teach.
So somehow you can imagine what a regular seminar is described here in which dease is a destructor and pretty frontal enviroment versus our enhanced seminar here at the bottom in which each student will have a EEG headsets. Most importantly, and this was also one of the biggest insight we got from Janet Metcalfe is that we're going to record also visually video record our students' behavior in class. So somehow we we'll have a real marker of not only neuron marker, but also objective video recordings of the student's behavior. So for example was Alfredo sitting here following the instructor as the instructor moves in the classroom or was Alfredo completely disengaged and staring at the computer or the phone and so on.
So we expect to produce a very, very, very large data set with neuron marker, behavioral markers, subjective data because students will be reading their interest in the course and so on, and also video recordings. In the attempt to understand really who gets an A-plus what happens in the brain and of our students that will what happens in the brain or what is the new will and behavioral markers will have the neuron behavioral marker. So as soon as they get an A-plus versus a student gets a C-Class or a B-plus and try to circle back their behavior in class during the semester with their final exams.
We hope so and we'll let you know in fall as we start. - Great, thank you so much, Alfredo. I think in the interest of time, we'll move on to the general Q&A, but before we do I just wanna make one note.
So probably many of you notice how high tech, both Brent's and Alfredo's projects are, but I just wanna be clear that there is also very interested in rather low tech types of innovations in the classroom. I personally think it's really amazing to contemplate what can you do that really changes the student learning experience without any technology at all. So just through structuring different kinds of activities through emphasizing different modes of learning in the classroom I think there's just as much to be learned in a very low tech kind of paradigm as there is in the high tech paradigm. So if you're thinking more in terms of low-tech types of innovations, I encourage you to feel like that's valued just as much by SOLER as these rather high tech paradigms are.
So we're gonna move on to a general Q&A and I think that Sandesh and Merrell will sort of moderate the Q&A and your questions can be for me or for Brent or for Alfredo, we have about 10 minutes left. So we welcome your questions and first of all, thank you so much for your attendance and your attention today, looking forward to interacting with many of you in the future. And we're we invite your questions now.
- Just to say while we're waiting for the question about the low-tech out of one of the earlier studies we did was just having students answer questions in a group and discuss a loop or individually and then take a test as individuals. And we found that students were answering a question as a group later performed better on the exact same test as individuals. And so now I always have them do group work in class because of that. And one of the next questions I'd love when I ever get time is to figure out why what is it about the group work that improved their learning in the as individuals later on. - Yeah, thanks Brent.
That's a great example of this conceptual innovation, structural innovation rather than a technological one, just as important in the work that we're trying to do here. - [Wayne] Hi, this is Wayne from the stats department. I have a question about, does the topic have to span an entire course or can it just be a new way to teach a particular topic? Does that make sense? - Yeah, so thank you for your question.
Yes, it could be there are many different ways to structure one of these projects. It certainly could focus on one particular module of the course rather than spanning the entire course any kind of research focused approach to a question about the student learning experience is a valid basis for an application. So certainly it's a great idea to focus on one particular course modular units or components. - And I'll just add to that, that one of the things that Adam and I have talked about is that, everyone as researchers we all have a tendency like you wanna change the world with every experiment, but, and then sometimes that inhibits you from actually getting to do something.
So just small incremental changes accumulated over many years. Like if every year, every semester you do one sort of incremental thing in five years, your course may be unrecognizable, but it's not so hard to get there. - Well, I know from my own teaching experience, that's one of the hallmarks of good teaching is, you have to be able to tolerate silence at times. So, I'll give you all some time to think about questions that you might have.
- Adam, if I wanted to have a project for the fall when would be the ideal time to apply by? - Oh yes it's a great question and something that we should've pointed out we're using a rolling deadline, a system here but we are hoping to have awarded all of our projects by the end of the fiscal year, the end of June. So if you have a project in mind for the fall it would be great to get that application in next month. And then the review process will happen.
The award would go out in June and then we would start working on the research design and all of the preparation throughout the summer in time for implementing the projects in the fall semester. Good question Merrell thank you. - Adam, there's some questions or there's one question from Melissa in the chat.
Any general advice for applicants about approaching the proposal? - So the biggest piece of advice I can give you for applicants is to schedule a consultation with SOLER because we very much want this to be a collaborative process. We know that faculty have a lot on their plates and that also the process of talking through their ideas with the SOLER team is a really fruitful and productive one. So the number one piece of advice is to get in touch, schedule a consultation, we'll talk through everything.
And like I said, we can approach it as a collaborative process with lot of support even already beginning at the application stage. 'cause we want to be working with faculty to develop these ideas. - Yeah, that's a good point 'ause we had an iterative back and forth as we thought about this. What would fit? What can we do is SOLER versus the Provost emerging technologies fund.
So that wouldn't, you don't have to sort of ship it off into the ether, but just have an iterative dialogue about what you wanna do. - And then Adam, another question from the chat from Maurice. Can you say a little more about the analytics dashboard for Canvas? I saw a mention of that in the slides.
- Yeah, so this is an exciting project that's SOLER working on in collaboration with CUIT and Academic Technologies Leadership Group. Again, the idea is to develop a tool that will make sense of this huge amount of data that's generated by student engagement with canvas and with tools like Panopto so that faculty will be able to kind of quickly get insights into their student's behavior. Because right now it's a little bit opaque.
There are ways to generate reports through these platforms, but it's not so easy to do. And sometimes they're incomplete. Sometimes they're not easy ways to filter or slice the data.
So we wanna create a sort of workflow and a dashboard that will facilitate this process. And then faculty will be able to get insights. For example, into who's struggling in a particular course, say, about individual students or about sort of general stats on our students downloading the reading PDFs in the case of like a digital engagement or our students completing quizzes and stuff like that. A lot of that stuff, there are ways to generate it, but we wanna develop a system that will streamline the process and then also do so in a way that will relate to particular goals in individual courses. So it's sort of a general tool, but also we'll have some elements that will focus on the needs of particular faculty or particular departments.
So I know that was a little bit vague but it is just kind of at the very early stages right now and something that we're gonna be developing over the summer and hope to kind of have some sort of test runs with in the coming academic year. - [Participant] Adam, if I can just follow up on that. I was connecting it also to the presentation that Roxanne did on her use of data in her public health research. And it seemed like the difficulty was manipulating and getting all of that data. So the plan here is to allow something like what she was doing in a much simpler and really make it more accessible to everybody else. - Yes, exactly, so wanna highlight the work of Roxanne Russell in the Mailman school of public health who's also part of this collaboration.
And again, the idea is to facilitate the process through which faculty can access data that's generated by student engagement with these platforms and then use it in important ways to either get insights into how students are doing or even to refine the design of the course. To use data as a as basis for evidence-based changes to the structure of the design of the course or the timing of different assignments throughout the semester and stuff like that. - [Brent] And I'm assuming this would also facilitate AB testing and things like that. - Yeah, exactly. So earlier I talked about the idea of the experimental model kind of that AB testing and learning analytics but really one of the most exciting things happens at the intersection of those two, right? So, not just the amount of data that's generated through students using these platforms but what if we implement manipulations to elements of the student experience and then we have combining the experimental approach with the learning analytics approach. That's something that I think we'll certainly be doing in the future.
- Hi Adam, this is showy from CDN. I have a question for you but thank you so much for everyone's presentation. Very informative. Number one is that from our office we have managed a lot more courses. So we have a lot of online mocks data.
I'm just wondering that if we are interested in applying for a research grant, what is the criteria to be a PI? Do we need to find a professor to collaborate with? - So thanks Showy. That's a great question. It's something that we're still kind of refining our ideas about. At this time we're thinking of the faculty PIs as a full-time faculty either tenure track or non tenure track.
But I think as we sort of expand our conception of the work that SOLER does, like in terms of developing this data dashboard, for example I think there'll be opportunities to get other types of administrative staff as PIs on these grants. So I would say if you have an idea about it, be in touch and next Sunday we can develop collaboratively, thank you. - Awesome, I think a follow up question is that, what is the budget of the grant can be applied to? So just, would you be mine to give a little bit of idea about that? Any restrictions or - So the question is what can the money be allocated for? What kinds of expenses? So usually we're thinking that would be allocated towards equipment, towards software licenses, towards in some cases hiring research assistants.
So I mentioned the SOLER fellows but there's also a need to bring in additional personnel like student research assistants, the funding can go towards paying them, can also go towards conference registration fees. And those would probably be the major categories. So thank you so much, everyone.
It's 1:00, so I think we will wrap up. But again, the most important takeaway I hope is that please be in touch. We really wanna connect, with faculty, with administrative folks to talk more about SOLER, to talk about project ideas. SOLER is still in its early stages but hopefully you can tell that we've done a lot of good work already and we have a lot of aspirations for continuing to develop in the future. But that development again, depends in large part on our ability to connect with all of the other people at Columbia faculty and administration.
So we really look forward to those partnerships and I think there's a lot to be optimistic about lots to be enthusiastic about coming up in the next academic year.
2021-05-03