This quarter, UC Davis’ Center for Educational Effectiveness performed a Mid-Quarter Inquiry (MQI) in my BIS 2B class, in which students were given a chance to provide feedback halfway through a course (in addition to the standard end-of-quarter evals). In this post, I will share and reflect on their feedback, which can be reviewed in full here.

The structure of the MQI allows students to free respond to two questions (What helps you learn in this class? and What limits your learning in this class?) and then vote on 10 suggestions for improvement provided by the students in the moment. It is worth noting that ~70% of the class was in attendance for the MQI, so these results represent the majority of students, but not the full enrollment. I did a quick-and-dirty analysis of the open-ended responses, and graphed the results of all three questions. Here, I will discuss the student responses to each section of the MQI and end with a summary of my take-aways and next steps.

What helps you learn in this class?

Top 10 student responses to the MQI question “What helps you learn in this class?” ranked by frequency

There were four components of the class that stood out as consistently helping students learn. The first was me as an instructor (which was a nice ego boost to start the analysis). Every quarter, students comment on my enthusiasm and passion for biology and report the positive impact it has on their learning and this quarter was no exception as over half the comments mentioned an aspect of my personality (enthusiasm, passion, openness) or classroom behavior (moving around the room, responding to questions, checking for understanding) that supports their learning. One student compared me to Bob Ross, which totally made my day. Another referred to my ‘beautiful life aura’ and how ‘the chakras align whenever she’s in the room’ which also made my heart happy. The iClicker questions I sprinkle throughout the lectures and discussions were a close second. Students enjoy the chance to test their knowledge, or work through a math problem together, and a couple explicitly mentioned the opening question each lecture, which jump starts their focus on the material for the day. The students also value the lecture capture videos, which many students make use of for exam/homework prep, supplementing their notes, or going through difficult content multiple times. I operate on the assumption that a difficult concept or example problem can be reviewed at their leisure later, so I’m glad that this response was in the top 4; however, it still only included ~60 responses (out of 215, so only 27%). Students also mentioned they enjoy my examples, stories, and case studies, which provide real-world links to the concepts we discuss in class. I’m glad they like these, because I like them, too and I would be hard-pressed to remove them from my lectures.

What limits your learning in this class?

Top 10 student responses to the MQI question “What limits your learning in this class?” ranked by frequency

Overall, you will note that the top 10 responses to this question represent fewer individual students, because there was much more variety in the responses to this question than to the previous one.  In this section, I will respond to the top 10 concerns expressed by the students, excluding the ones that are included in the next section on recommendations, the lab section concerns, and the structure of class concerns. The structure of class concerns related to things like enrollment numbers, timing of the course, and wi-fi capabilities in the classroom over which I have absolutely no control. I did speak with the educational technology team about classroom wi-fi earlier this quarter and was told that SocSci 1100 was built in such a way that upgrades to the wi-fi are nearly impossible, so I do not believe this concern will be resolved soon (or ever), though they are aware of the issue. The structure of the lab activities and pre/post-lab questions are required to be identical across all sections of the course, so I cannot change anything relating to the lab. I have passed along some of the concerns about pre-lab feedback and grading concerns to Pat Randolph (who oversees the TAs), and I encourage the students to talk directly with their TAs for clarification on grading expectations and feedback on lab assignments. I also encourage the students to voice these concerns on their end-of-quarter evaluations for their laboratory section.

By far the most common response was that my speed is too fast – both in terms of how quickly we move through content and how quickly I talk. The students are right – this is a fast-paced course and I am a fast-paced person. We have a lot of content to cover in 10 weeks, and I talk fast when I’m excited, which is most of the time. I have been working on slowing down ever since I started teaching (as my previous eval reflections show). Perhaps ever since I started speaking. One of the ways I try to get around this is by repeating myself a lot – and some students noticed this and appreciated it on the previous question. I also consider the speed factor to be one of the major benefits of lecture capture – the students can refer back to the videos to spend more time on material, which can even be watched at slower-than-real speed if my speaking is too fast. One issue here, which some students expressed in their comments, is language – both English, and scientific terminology. The lecture capture videos are captioned, but whatever mechanism provides the captioning poorly understands scientific terms. For example, in our discussion of life histories, every time I said “semelparous” the captioning reported I said “simple Paris” or “same in Paris” etc. It also confuses regular English sometimes – for example on one clicker question, I was discussing whether students would ‘choose E’ which the captioning reported as “choosy.” If English is not the language with which a student is most comfortable, mistakes like this in the captioning hinder their understanding even further. I have contacted the office that manages the lecture capture videos to look into how the captioning is generated to see if these issues can be addressed.

The second most common response was nothing – yay!

The third most common concern included a number of concerns relating to answer keys to coursework and feedback on assignments including 1) online answer keys to homework problems and exams, 2) answers to study guides and clicker questions, and 3) feedback on (in)correct answers on the homeworks. These are discussed in the recommendations section next.

The fourth concern was that students do not always enjoy or see the value of the discussions. Interestingly, there is a bit of a bimodal response here, since the discussion activities were also in the top 5 for the previous question on what helps students learn. The discussions are a new component of the course, and very different from other offerings of the course, current or previous. Since I am still calibrating these activities, some of them have been too long, and students worry content we didn’t get to in discussion will be on the exam. It won’t. There was concern that the small group discussions aren’t getting to the ‘right’ answer and then students will have learned the ‘wrong’ thing for the exam. You haven’t learned the ‘wrong thing.’ The primary goal of the discussions is to practice making predictions and interpreting figures/data on your own. In some ways, the exam question that most related to our discussion activities was the one that asked the students to predict the results of a study on plant growth given the hypothesis that nitrogen was the limiting nutrient (even though nutrient limitation was not the focus of any discussions). The students may not be aware of the rather overwhelming body of evidence (ex. Freeman’s recent meta-analysis) that active learning (discussion activities, small group work, etc) significantly improves student understanding and retention of course material, even when they don’t necessarily enjoy it. I understand why not all students enjoy them. The discussion activities are more about uncertainty and real research questions that aren’t yet fully answered. The students can’t memorize terms or facts, they must think through complex problems that don’t have simple answers. They can’t just sit there and stare at me like they can in lecture, they have to do stuff and talk to people. And they have to wallow in uncertainty for a while before it makes sense and/or we talk about it as a group, which is the part they probably hate the most and which is most valuable for their learning and most like real science. Unfortunately, sometimes the pedagogical structure that is most effective for student learning is not the one they enjoy the most (which is they there’s a paper titled “Is active learning like broccoli?“). I’ll do my best to structure the last two discussions to an appropriate length and difficulty, and I ask for the students’ patience and trust that this is a good mechanism for learning, and that I will represent the discussions fairly on the exam.

Concerns relating to incomplete slides on Canvas and the timing and repetition of homework assignments are discussed in the next section on recommendations.

I’m actually pleased that students want more preparation and opportunities for practice as they prepare for homework assignments and exams. There’s perhaps a disconnect between what types of practice the students already have in the class and what they think they have or think they want. For example, comments in this section frequently requested practice exams and review sessions. The study guides I post are essentially a practice exam, they are just called study guides. A study guide that is not a practice exam would be a bullet-pointed list of terms or concepts to study, otherwise, the two are kind of interchangeable. I call them study guides because that sounds less intimidating than a practice exam, and I find more students will use a study guide than a practice exam, even if they contain the same questions. I also do hold review sessions – I just hold them as extra office hours. I assume the students meant evening review sessions. My schedule is such that it is much easier for me to hold office hours or review sessions during the regular work day. So much easier in fact that on exam weeks, I hold 6 hours of office time in the BLC, as opposed to a 1-2 hour evening review session.

Which leads us to our last concern of wanting more office hours. I recognize that many students have class or work during my regular office hours. As already mentioned, I hold extra office hours in exam weeks to be available to more students prior to the exams. Additionally, students can approach me immediately prior to or after lecture or discussion to ask a question (many students come up at the end of class, and I have not once left without addressing all of their questions). Students are also welcome to attend any office hours held in the BLC, whether it is your instructor/TA or not. Students are also welcome to meet up with other students there to work through content or problem sets together. Students are also welcome to post questions to the discussion board. I cannot be physically available in the BLC at all times, which is why we structure as many mechanisms as possible for you to get the support you need. Please take advantage of them.

Student Recommendations

Suggestions provided and voted on by the students during the MQI, ranked by frequency of students who agree with the suggestion

Multiple attempts on homework: nearly 83% of the students present for the MQI wanted additional attempts on the homework assignments. This surprised me, since the mean scores on the homeworks have been in the mid-B range, so I think the students are doing quite well on them. Generally, I am strongly opposed to changing grading structures mid-course; however, so many students want this change that I am willing to consider it. The current grading structure was that each homework quiz can only be taken once, but that the lowest homework grade will be removed from each student’s grade calculation. In my opinion, dropping one homework is a more valuable benefit, since it allows a student to have a weekend where they are ill, or working long hours, etc and have it not affect their grade (which taking the quiz multiple times would not help). If I were to allow multiple attempts on the homework, I would allow only 2 attempts and I would no longer allow students to drop their lowest homework score. Since we only have three homework assignments left, I consider this a poor trade, and I think dropping the lowest homework score would most benefit the students’ final grades; however, I will allow them to vote on it in class on Wednesday.

Full lecture slides posted to Canvas with no hidden content (80% support): it is important to note that I ‘hide’ content on the lecture slides only for the purpose of inspiring student discussion and consideration of a question. The content I ‘hide’ is examples of whether or not real scenarios follow our hypothesized expectations, or examples of answers to a discussion question I pose to students, and occasionally definitions of a term I want them to try to define on their own first. The students have access to the slides during class, so if this content was visible, they would not think about and discuss the possible answers with their neighbors, they would just read ahead to the next slide or two and regurgitate what is on the slide. I do not post full slides after each lecture for two reasons: 1) students who attend class get a little bonus of seeing the ‘hidden’ content and can even take a photo of it in the moment on their phones, and 2) all of the content is visible on the lecture capture videos (which is why I’ve been putting ‘hidden’ in quotations) and so all students can refer back to the videos to see what the slides showed. Consequently, the material is not truly ‘hidden’.

Correct answer on clicker questions (80% support): I’m not going to have a slide with the answer marked, since students will just look at the next slide and not think about the question. I believed I was discussing the answers to each clicker question when we view the results (with the exception of the opening question, because we come back to that one later in lecture); however, several students commented that my discussions after the question are too vague and the answer is unclear. I will endeavor to be more explicit on the answer when I pose a clicker question, and I will ask the students to stop me and ask for clarification if the answer is unclear.

Post study guide answers on Canvas (71% support): when students have asked me for answers to the study guide questions, I encouraged them to go to office hours, or to post to the discussion board and we can talk about the answers there. If students know an answer key is coming, many of them will not work through most of the questions on their own. On a discussion board, the students can start a dialogue about why answers are right or wrong, which helps them identify where their misconceptions or mistakes are far better than just having the answer. I have struggled to get students to use the discussion board in BIS 2B (not just this quarter). I’ve talked with some other faculty, and they’ve had better luck with anonymous discussion boards like Piazza. I may ask the students this week if they would be more likely to use an anonymous discussion board for this purpose.

Post homework answers on Canvas (69% support): It is standard practice across the majority of offerings of this course to NOT provide electronic answer keys on Canvas in order to limit a particular mechanism of academic dishonesty (I’m not going to elaborate on the mechanism because I don’t want to give the students any ideas). I was surprised by how many students considered going to the BLC once a week an egregious barrier to accessing the answer keys. All the students are a few doors down from the BLC once a week for lab, and all the TAs have access to the room and could let students in to check answers even if there aren’t office hours at that time. Still, I can hang the answer keys in the hallway so they can be viewed any time the building is open, not just the BLC.

Provide feedback on pre-lab assignments (63% support): This is a concern that I cannot directly resolve. The pre-labs are graded by your lab TA, and so the best mechanism for feedback on these assignments would be in lab.  I have passed along this concern to Pat Randolph (who oversees the TAs), and I encourage the students to talk directly with their TAs for clarification on grading expectations and feedback on lab assignments. I also encourage the students to voice these concerns on their end-of-quarter evaluations for their laboratory section.

More time on the homework assignments (46% support): The homework assignment availability dates are timed for a specific reason. The assignment does not open until immediately after class on Friday, since the questions that relate to Friday’s content could not be answered before this time. The due date of Sunday at midnight was set so that students cannot get unfair assistance on the homework during office hours. I know that this is a tight window, which is why I structured them to be only 10 multiple choice questions. I have been keeping an eye on the average scores and average time spent on the assignment (which Canvas calculates for me) and adjusting future assignments accordingly.

Include multiple choice questions on exam (34% support): I have never cared for multiple choice questions for a variety of reasons (many described here). Writing fair and clear multiple choice questions is harder than it seems, and there is some anecdotal evidence that they favor students who are less prepared (ie, guessing), and that students who are better prepared frequently talk themselves out of the correct answer. There is also no chance for partial credit. One student commented explicitly on the fairness of this structure, which I would like to address: “Short answer tests in an uncurved class seems unfair when other BIS2B sections have multiple choice tests.” To my knowledge, all other sections of BIS 2B (including my previous ones) have exams that are a mix of multiple choice and short answer. Essentially, what our class does is take the multiple choice sections of the exams and put them as weekly homeworks, which students can complete at their leisure and with the assistance of notes, textbook, lecture slides, and video, leaving the full 50 minutes of exam time for just the short answer.

Include short answer questions on homework (19% support): I agree that this would be good practice for the students prior to the exam; however, I simply do not have the grading manpower to handle 312 short answer questions every week. I support practice on the short answer questions through the study guides, which I encourage students to talk about with me in office hours, before/after class, or through the discussion board, if they would like feedback on their answers.

In Summary
Overall, it appears that the students are generally pleased with the structure of the course. While not overwhelmingly supported, the structural differences in this course from others (including clicker questions, lecture capture, discussion sections, and homework assignments) all appeared in the top 10 things that help students learn. The primary thing I can do to increase student success is slow down. I will do my best to speak more slowly and repeat myself. From the students’ engagement in class, and the high mean grades on the weekly homeworks and our first exam, I believe the students are keeping up with the pace quite well, though it may not feel that way to them.

There are three structural changes I will consider: First, I will post answer keys in the hallway outside the BLC so student access is not limited to when the BLC is open. Second, I will ask the students if they would be more likely to use an anonymous discussion board for discussing study guides and weekly homework questions. Third, I will allow the students to vote on whether they prefer 2 chances on each homework assignment or dropping the lowest homework grade from their assignment.