More than Words: Using Voice Comments to Provide Effective Feedback

More Than Words: Using Voice Comments

JASON:    Welcome to today’s webcast More Than Words: Using Voice Comments to Provide Effective Feedback, and I’d like to introduce you all to our presenters today. We’re very pleased to welcome Meagan Kittle Autry. Meagan is the Assistant Director of the first year running program and a campus right-hand speaking program at North Carolina State University where she is also working on her PhD in communication, rhetoric, and digital media. Her research with the Writing Program focuses on teaching with technology in hybrid classrooms and the use of audio visual feedback for writers across disciplines. Her work also focuses specifically on issues for writing program administrators in implementing technology in their programs. Welcome Meagan.

MEAGAN:    Thank you, I’m very glad to be here.

JASON:    And next we have Renee. Renee is a Professor of English at Saddleback College and she has been teaching writing for 15 years now. She collaborates with K-12 teachers in developing curriculum to bridge the gap between high school and college in the areas of reading, writing, and grammar. And Renee also works on creating these professional development sessions that we offer through Turnitin. So welcome Renee.

RENEE:    Thanks Jason, great to be here.

JASON:    And I am Jason Chu. I’m a Senior Ed Manager with Turnitin. I’m going to be your moderator for today’s session. Before we get started, Alexis just pointed out that the web links are not currently active. I’m going to drop in some active web links for you, but in the meantime I’m going to pass the mic over to Meagan. So Meagan, you want to get us started?

MEAGAN:    Sure. Thanks Jason. So thanks everybody for joining us today. We’re really glad to have you. I want to start off by giving you a quick overview of the topics I’ll be covering today. I’ll start with a review of some of the literature on oral response, then I’ll look at some of the new trends in audio visual response, and I wanted to end today with a set of suggestions for using oral response in your own teaching if you’ve maybe never tried that before.

So I’ll begin today with looking at an exigency for oral response. Most research on teacher response to student writing historically has focused on conventional forms of response, and specifically written response to student essays. Moreover, very little research until about the late 1990’s focused on how students experience written response, or what they do with the responses to their writing that we give them. And to continue looking a bit at why we should be thinking about oral response, a lot of the research has found that conventional written response to student writing is not all that effective. Helix who I have cited here says that teacher comment has little impact on student writing, and a later study by Doro Richard also cited here has backed up those findings. More recently a dissertation by Montgomery found that a significant number of students don’t read the feedback thoroughly or seriously, and of those who do, many of them misinterpret the feedback that we’re giving them, and very few think of feedback as an exchange or dialogue between a teacher and a student.

There’s clearly a gap here, right? How can we make response more effective for students? I mean, that is our goal as teachers, right? So I’m going to talk to you today a bit about some of the scholarship in composition studies that has looked at oral response to student writing as perhaps a way to give more effective feedback for students. And they talk about that because oral response sometimes offered in combination with written comments on essays provides opportunities for connecting with students in ways that simply written words just really can’t.

I want to cover a bit of the early oral response literature here, and this literature goes back quite a while. The first publication to mention audio feedback is by Clamor back in 1973 who wrote a short piece about the possibilities of cassettes, then a newer technology in the classroom. Olsen’s piece in ’82 followed this and he reported on his own use of cassettes in two year college English courses where he wrote that his students felt his audio feedback conveyed a sense of caring on the part of him as a teacher.

And then perhaps most notably my own colleague, Chris Anson, of In Composition, he wrote early on about his experience using cassette tapes to record his response to student essays. In his article I’ve cited here in “Our Voices” he explores the range of benefits that he thought of, or he found when giving oral response, benefits for both him as an instructor, and he felt for his own students. And then he went through some considerations for oral response. And, of course, while the technology that he used back then has since changed, right, we don’t use cassette tapes anymore, I’d say that many of the principles he talked about with oral response remain.

So I’d like to review briefly the benefits that he identified of oral response. So the first benefit that he talked about is tone. When you’re offering oral feedback, I mean, you’re literally talking to the student; whereas written words and often the tone that we want to convey with them can be misinterpreted by students. They can have a much better sense of your tone and your message when they can actually hear what you’re saying. Kind of just like with me talking to you here today, hopefully you can hear how excited I am about the concept of oral response to student writing.

The second topic that Anson talked about is the personal nature of this response. He says that offering a narration of your feedback, which is essentially what talking to the student is, you’re telling a story of your feedback, it’s a much more personal approach to responding to students. While written commentary can feel cold or impersonal, oral feedback can feel a lot more personalized for the student and more like an ongoing dialogue on their work, rather than just a definitive evaluation of their effort.

The next concept that Anson talks about is the broad issues or big picture that we want to look at with our feedback. He says that oral response helps instructors to focus feedback on these broad issues or big picture issues of a paper rather than falling into a habit of copy editing every comma splice. I mean, after all we are more concerned with the big ideas, the arguments our students are making rather than silly punctuations mistakes. So responding orally makes that a lot more easy to do.

The next concept he talks about is the idea of teaching as coaching and advising. He said that when he was grading papers orally, he felt his role shift as an instructor. He said he went from feeling more like an evaluator who was judging students, to a coach who was advising students which made him feel more like a mentor and a teacher in the true sense of the word.

He then talks about issues of quality and quantity. He says you can help students so much more with five or ten minutes of spoken response rather than a few pieces of random margin area. Not only do your spoken words amount to pages of material in just a few minutes, you know, rather than a few margin comments or perhaps one end comment. But with spoken words you can provide a fuller context for your comments for your students. I’ve been talking for about five minutes here and I think of all the things that I’ve said; think of how much feedback that would be for a student on a paper rather than just a few margin comments. It certainly adds up and the quality is there as well.

And the last point that Anson talks about in this piece is the concept of offering a reader response perspective. So talking through a student’s paper gives you the chance to show students how readers react as they take in their writing. You can offer feedback as a reader, not just their teacher. You could say things like when I got to this line I stumbled, you know, maybe if I read it back to you, you might be able to see where I was having a hard time too. By doing this you offer students a model for thinking about their writing, which is certainly something that we want to encourage them to be doing.

So beyond the work that Anson has done, which has been quite influential within composition studies, there are scholars in other fields that have conducted case studies to exam oral response to student writing, so this is applicable across the curriculum, not just for composition scholars.

The first article I’ve cited here by Brian Still, he talks about his use of imbedded audio files in a Microsoft Word document. So we’ve moved past cassette tapes all the way up to digital files. So you used to be able to, I think you can still embed audio files in Microsoft Word. So that’s what Still did, and he did this to provide early drafting feedback to a class of technical writers. So this is when they were drafting, not their final product. And he found that offering oral feedback in an early stage when writers were still very new to the genre and the form of technical writing; it was very effective at conveying important points specific to the documents that students were producing. His students also picked up on how the oral feedback allowed him to give more guidance in his commentary. And one of his students said, I quote, “I can tell what I need to do just by listening. Normally I have trouble making sense of what you write, but when you are talking it feels like we are in the classroom, but more on a one-on-one time.” Overall, 79 out of 80 of Still’s students said that they would prefer to get their feedback on their papers in an audio format again.

And the second study that I’ve cited here for you all is by I Said All, and they report, all the authors participate in this study as instructors and they used an online digital audio editing and recording app to provide oral feedback for students. And these instructors all taught online graduate courses in education. So they gave oral response for about half of the assignments in the class and written feedback for the other half, and they randomized that, and all but about two of the study participants indicated that they preferred audio feedback.

They also interviewed the students to follow-up and the interviews revealed a few important themes that I think are worth discussing here. The first theme that emerged was that students understood commentary in a more nuanced way so they could better understand the points that their instructors were getting at.

The second theme that they talk about is students felt more involved in the course, which is definitely important for those of us who teach online classes, you know, we want to be reducing that transactional distance. So for students to feel more involved was definitely a good thing.

The third thing that students talked about was that they understood and retained the content of the feedback better. So it helped them with processing their learning. And then lastly, they felt that the instructor cared more about their own personal learning.

So there are a lot of good points that are coming out of these studies of oral response to student writing.

Now these studies that I’ve talked about have focused solely on providing oral feedback in terms of recorded audio being returned to the student, some kind of recording audio file. But of course, new technology gives us new possibilities for oral response, and most notably now we have the opportunity to have both audio and visual feedback at the same time with what’s called a screen capture program, and some of you may be familiar with this.

So screen capture program essentially records everything that happens on the screen. I think we’re using one here today to record this webcast. So you could have a student paper open on your screen, an electronic copy of course, and then you can hit record and the screen capture program would record you scrolling through the student’s paper, highlighting lines, inserting comments, and also it will record anything that you speak in terms of feedback to a student.

So you can be talking through your response, putting in a couple of notes on the paper or just pointing to certain spots that you’re referring to, and this will all go into a video that students can then view through a unique web link that takes them to securely view that screen capture. So with the use of a screen capture it gives us this added level of detail for providing oral feedback, basically even more context for what we are saying to them because students can sort of see what we’re referring to as your cursor hovers over a certain part on the screen or you highlight a line.

So here at NC State, recognizing how useful this possibly is for students, Chris Anson and myself, along with a couple of other colleagues, have undertaken an about two yearlong study comparing student and teacher response to written versus audio visual or screen capture commentary feedback for students. We studied composition, psychology, and women’s studies classes, and we had instructors first provide solely written commentary, so inserting a comment into a Word doc, and then they, for the second project provided the screen capture feedback for students. Once both projects were completed we had students complete a survey and then some follow-up interviews. And we also interviewed the instructors about their experience with the screen capture technology.

Our preliminary results show that students generally favor the screen capture feedback. Initially students were most familiar with written feedback, which makes sense. They’re, you know, handwritten on a hard copy, or comments inserted into a Word doc. But by the end of the study, a large majority of students said that they would prefer to receive the screen capture comments or feedback in the future. Students also felt that the instructor was more engaged with their work in the screen capture feedback, that they left more constructive feedback for them, that the instructor was more friendly and encouraging to them, and overall more supportive of their writing. They also felt that the student was less distant from them when giving feedback, so there was increased teacher presence, and that the feedback was a lot more personalized for their own learning.

So we see a lot of positives coming from the screen capture, oral response in general, and generally positive affect responses associated with it. But of course, any responsible use of a new technology necessarily includes a critical take on its use. So I want to talk through some of those issues as well. Some students do have less than positive interpersonal reactions to the screen capture because it is a more personal form of feedback, right, you’re talking with the students.

So with one of the students in our study, when she heard constructive criticism on her paper it caused her to feel defensive about her work while she was watching the screen capture feedback. In communications literature we would say that this is a dimension of face saving, so reacting to a faced threat, or a negative face experience, and that impedes student learning. And indeed we can see this when we asked the student what the main message of her teacher’s feedback was, so what was the teacher trying to tell you in the screen capture. The student couldn’t recall any of her teacher’s points. She just felt uncomfortable about the exchange, and that’s about all she could tell us.

So while overall we see positive reactions to the technology, we have to keep in mind complex issues that students face when receiving feedback on their work, and we need to be mindful of it in our practice.

I want to offer maybe a couple of other caveats for you in considering oral and audio visual feedback. Caveats might include your class size. You know, if it takes a few more minutes per student to do a screen capture, you know, is that feasible for your course. Other caveats might include the level of venality of the assignment, the genre, right, it might just not be feasible to do a screen capture feedback for certain genres. Sometimes your institutional context and the support that you have as an instructor for development of writing. So just a few ideas to keep in mind here.

So I want to end my time with a few suggestions for you for trying out oral or audio visual response in your own classes. So first I would suggest reading the student’s paper in advance and organizing your thoughts before you begin recording. Some instructors do live read student essays, but this requires a bit more experience with the technology and the skills to respond on your feet and appropriately while recording.

I think organizing is the way to go. To organize your comments in advance you can pre-insert marginal comments in your student’s paper as a guide for yourself and also sort of points of reference for students, the things that you want to address, certainly those can be the big picture items that you want to talk about with the student. Certainly the downfall to this approach is that you would lose some of the spontaneity in your response, but I think overall the benefits of being organized in advance outweigh that loss of spontaneity.

The next point that I’ve got here is to set a designated time for your response to students. So say I’m just going to talk for five minutes or something. This will help you to organize your comments, obviously sort of prioritize things. Sometimes software has predetermined length of time for which you can speak, which also does help guide you in determining what you want to cover in your comments.

The next point I have here is that at the beginning of your commentary offer an outline of what you’ll be talking about to orient students to your feedback. You know, just liked you’d give an outline for a conference presentation, do this for your students and they’ll have a much easier time understanding the important points that you’re trying to get through with your feedback.

The next point I have here is that giving oral commentary or audio visual commentary allows you to use the full context of your class to situate your response more easily and without having to type really long responses in the margins. So you can just quickly reference something such as I like how you built from the previous assignment with this issue, or great job incorporating our discussion of visuals from class on Monday. Context is really key for students to understanding the guidance that you’re giving them, and so it’s important to include that with your oral commentary too.

I touched on this previously with the next point, but definitely offer students a reader like explanation in your feedback. This helps them to think about writing and to see that you’re not just the judge and the jury of their work, but instead that writing is this ongoing process and that you’re coaching and mentoring them through.

And of course, last but not least, have fun. This is a much more enjoyable way to provide feedback than strictly written comment, after all, having a conversation can be a lot more fun than writing a paper.

So that wraps it up for my comments today. For future reference, I’ve included citations from my discussion if you’re interested in pursuing any of these. So thanks so much for listening.

JASON:    Great! Thank you so much, Meagan. Thank you for sharing all that wonderful information. Again, folks I’d encourage you if you have questions just go ahead and send your questions to us via that Q&A widow. We’re going to switch gears here. I’m going to turn the mic over to Renee, and Renee is going to talk a little bit about the voice comments feature that we will soon be releasing in Turnitin, and also share with you some ways in which you can, in particular can use voice comments to provide feedback within the GradeMark resource that’s part of the Turnitin suite, so Renee.

RENEE:    Great, thanks Jason. So coming this spring, instructors are going to be able to offer a three minute recorded voice comment within Online Grading (GradeMark). And as Meagan discussed, tone is really an important element of students. She kept bringing up the emotional attachment with students. And really in learning and critical thinking, we tend to emphasize the cognitive domain, but the affective domain has significant impacts on learning and motivation. So students really need that positive, comfortable experience to decrease the anxiety and increase learning, and voice comments as Meagan has illustrated can help this.

I’ve often sensed that my students think I’m angry when I grade my papers and I try at the beginning of the semester to explain to them that in fact no, I’m quite supportive, and I’m even happy when I grade their papers. And so now with voice comments I can insure this.

Voice comments can be particularly beneficial in an online environment where students may have never heard their instructor’s voice, and it can facilitate that student’s connection with the instructor, and then of course, the course. Within the Turnitin system, we have the ability within GradeMark to offer various types of feedback or modes of feedback, and this is just another mode for offering feedback that aligns with the instructor’s preferences. This just adds depth to what already exists.

So I’m going to kind of shift and show you what GradeMark looks like if you’re not familiar with it. You have the student’s paper on the left hand side and some of the various features are these quick marks that you see on the right hand side, you can create a bank of comments and drag and drop those comments right onto the student’s paper. And when the student opens their feedback they see these comments, they also might see definitions of what these comments are, and then you also have the ability to attach rubrics that you create as an instructor to the assignment and provide benchmarks for students in terms of how their paper aligns with those benchmarks. And then on top of that a general comment feature, and that’s where this voice comment will be added.

So there’s so many different ways that we can provide feedback on student writing, and these voice comments just add, again, more depth to what exists. And you couple that with the various methods for, you couple all of these methods for grading student writing with what Turnitin already offers, and that is the ability to see originality check and peer mark and peer feedback as well, and that just adds to a rich environment of both evaluative feedback and some formative assessments.

And I think we have a video. Are we going to show that video, Jason, of how this feature works?

VIDEO:    To leave a voice comment, open a paper in GradeMark and mark it up as you would evaluate other student papers with quick mark comments, custom comments, and highlights. Now click the general comments icon at the bottom of the sidebar. You’ll now see that you can leave a voice comment and a general text comment. To record a voice comment, click the blue microphone icon. When the voice comment bar turns red, that indicates it is recording. You can leave a comment of up to three minutes in length.

    If you need to pause the recording, click the pause button. You can then un-pause and continue your recording by clicking the microphone icon again. When you’re done, click the stop button. You can playback what you’ve just recorded by clicking the play button. If you’d like to rerecord your comment, discard the current recording by clicking the trash icon and rerecording. If you’re happy with the recording, click the save button. That’s all there is to it. You’re now ready to give your students a voice comment that is clear, engaging, and easy to understand.

RENEE:    So with that, I think we’re ready to kind of shift gears and answer questions about Meagan’s presentation and all her great research and then I can speak a little bit to how the new voice comments will function within Turnitin, and again that will be available this spring. So it’s pretty exciting for those of us who have been waiting and already using GradeMark to grade papers to add another feature.

JASON:    Thank you so much, Renee. With that, I’m going to just put up this slide here just to provide you with some additional information, and again just to remind you real quick if you’d like copies of the PowerPoint today, then I ask you to simply complete that survey, it’s available to you via the web link window and down in the lower right hand corner, so just click on survey, click browse to, and then you’ll be taken to the survey and you can complete that and then get a copy of the PowerPoint presentation. We also will be sending out an archive, a link to an archive video of today’s session so you’ll have access to that as well.

Before we move to questions, I just want to, I know we’re coming up on the close of our half hour here, I just wanted to thank you all for participating today, and I also wanted to thank especially Meagan as well Renee, thank you both. And also just to say we look forward to seeing you at our next webcast. So with that I bid you all farewell and I’m going to turn attention now to some of the questions that we received. And let me just start here, we received a bunch of questions. Let me start here with a question from C. Brandt Smith, who wanted to know, and this could be directed to both you Renee or to you Meagan, should oral response to a student be followed up with written?

MEAGAN:    Well I would say that what’s great about the screen capture program is that it allows you to have both the oral response and sort of written response at the same time because you can pre-insert some of your commentary for students. And then all the instructors that we worked with in our study also in addition to sending a link to the screen capture video, emailed back the student’s paper or re-uploaded it to the learning management site, whatever they were using. So students still did get their actual paper back along with the video.

RENEE:    And I might add, I teach in an online environment and we have to be ADA compliant, so in that case then we do want to add [break in recording].

JASON:    Great, thank you. I’m going to turn attention now to Michael’s question, and this is directed to you, Meagan. Michael was curious to know as to what app was used in the still study?

MEAGAN:    The still study was with the technical writers and he had just inserted an audio file within the Microsoft Word software. I’m not sure of quite how to do that myself, but I know that he just inserted an audio file. And I’m not sure, they did not name the app in the second study that I had cited there, I Said All, I don’t believe they named the app there. But I could say that the program that we used for our screen capture study, I think that was one of the other questions here, it’s called Jing. It’s a product by TechSmith and they have sort of a whole slew of screen capture products. There’s Jing, Snag It, and Camtasia, and they range from free to paid programs.

JASON:    Great, thank you. Jackie had a question, and this is I think for both you and Renee and Meagan, oral feedback. Will we need to provide a transcript of the oral feedback in order to comply with students with hearing impairments? I mean, what is your advice with respect to that?

RENEE:    I think I was talking about ADA compliance, and especially with the online environment. Yes we require that there is some sort of text to support what you’re writing, what you’re saying. So you can see in the demo that we showed there’s a box to actually type those comments. So you can be typing those comments up and then talk in that voice recording.

JASON:    Okay great! Let’s see here, I’m going to jump down here; there are a bunch of questions that came in. Valencia wants to know is there any information on time management. In other words, how do faculty manage time using audio visual feedback with class size?

MEAGAN:    Yes that’s a really great question and to be honest, you know, sometimes giving audio visual feedback with larger class size just might not be as feasible. Our instructors in our study did find that initially it took about five to seven additional minutes per student when they were first learning the technology. And by the time they got about in the middle of their set, they got it down to about the same amount of time for providing feedback for students in this way, so something to keep in mind.

The other thing I would recommend is just to maybe find an app or a screen capture tool is that’s what you end up using that limits your time and then you know that you’ll only spend five minutes talking to the student, for example, that’s a common time on some of these videos which is really just practical because of, you know, size limits for the video.

RENEE:    And I’ll follow-up. I like that Meagan brought up the point of Chris Anson’s research and that is when you give this type of feedback  you’re focusing kind of more holistically on the paper, you’re not diving into the nitty-gritty, you know, editing the student’s paper. So it encourages you to kind of focus on more global issues within the paper and that might actually, you know, eventually speed up the process as you move away from trying to mark errors and give support.

JASON:    Great, thank you both. I’m just going through some of your questions here. We’ve got, let me see here, we’ve got a few questions, or a few questions that touched on sort of working with ESL students, and I’m wondering Meagan if your research lends any insight into the benefits of using voice comments in particular with ESL students.

MEAGAN:    Yes that’s another, you know issue to certainly think about with using this technology. We did not specifically look at second language students. We couldn’t really sample for them in our study, we just had certain courses, like certain class roles based on the instructors that volunteered for the study. But the student that I talked about with the caveat that she responded negatively or she had that sort of negative face response to the criticism of her paper, she was a non-native speaker of English. But we can’t really make any correlations or connections just based on that one student’s experience that she had felt, you know, that she had to sort of save face in that situation. But that’s something certainly for future research to look at. Sorry, I can’t speak to that more specifically.

JASON:    No that’s great. Thank you for that. Renee, do you have anything you want to add to that as well? I know that this is something that you’ve been thinking and talking about for some time now.

RENEE:    Yes in terms of working with ESL students, you know, I think that actually having the voice, I used to teach English as a second language and I studied linguistics actually as my background, so I find it’s very interesting for students who are second language learners to then have a written text and the voice comments, they can hear that tonality, they can see the writing. Again, what I was talking about with not kind of going in and editing every little mistake, I have students who are ESL learners that come to me and they’re like how will I fix my article errors, and I get to tell them well it’s going to take you five to seven years of exposure to the language before you’re going to learn your article errors.

So as an instructor, I’m not going to go in and mark all their article errors, I get to just tell them, you know, that’s something that they’re still working on and to be patient. And boy, you should see the relief on their faces because I work with a couple of students right now face-to-face and I’m like, you know, the prepositions and the articles just aren’t going to come this semester, and you need to be, need to be aware that I know that and you need to know that as well. And again, I think that speaks to that affective domain, that relief, okay I don’t have to put so much [break in audio].

JASON:    Great, thank you Renee. I guess with that, I don’t see any more questions coming in. I wanted to take the opportunity again to thank you both, Renee and Meagan, thank you very much. And to our participants, thank you for staying with us today and thank you for joining us. And again, I hope that you’ll join us for another webcast. For more information about webcasts, simply click on the web link window on the right and then click browse to. As for the survey, to access the survey, just click on the survey link and then click browse to. And again, if you complete the survey we’ll send you a copy of today’s PowerPoint. We will also be sending out a link to the recorded video of today’s session, so we hope that you’ll take advantage of that. Thank you again everyone and if you’re done with the meeting, just simply close out your browser. Thanks again, take care.