In Psychology 350, Research Methods & Data Analysis, Professor Cal Garbin makes extensive use of large question banks to provide opportunities for his students to do enough "reps," or practice, that they develop the skills and knowledge needed to approach course assessments with confidence.
Garbin’s approach combines many assignment exercises with strategic proctored testing. For example, students practice terms, concepts, identifications, and decisions using 56 immediate feedback exercises that are each a small collection of question items pulled at random from relevant question groups. To complete an exercise, students must keep practicing until they get a certain number correct. Students are even welcome to work together since this would only mean they would do twice as many items. This type of approach turns each assignment into an important, and sometimes, collaborative, learning event and a low-stakes assessment. A similar approach is used to help them learn to hand calculate statistics and to use a statistical software package.
Students even have the opportunity to complete online exam preparation assignments. These are not worth any course credit, but they are exactly like exam items in form and requirement. When students see these items in an exam, only the “story” of the analysis and the specific “data values” will differ. Hence, there is a compelling incentive for students to make use of these exercises.
Until spring 2020, Garbin used the Digital Learning Commons for in-person proctoring of major exams in his courses. However, with the significantly reduced capacity of the DLC fall 2020, Garbin will limit his use of the resource to only those select items on which he wants to test with explicit “memory load,” i.e., on only those items that he wants students to answer from memory. These include an assortment of terms, short-answer responses, and choosing specific statistical models. The other items, such as “story problems” that he used to include in these exams will be moved to an online, low stakes format like the aforementioned practice exercises.
Garbin’s approach is not only effective for learning, but it also incorporates critical techniques for supporting academic integrity. First, the many low-stakes assignments do not create all or nothing scenarios where students might feel compelled to cheat to move forward. Instead, it gives students a chance to see what they know and to practice with little penalty, thereby helping them learn and develop their confidence in their knowledge. Second, extensive question banks and random selection of items greatly diminishes the utility of collecting and sharing specific items online. Third, the number of exercises combined with their low contribution to the course grade makes the cost of paying someone else to complete them somewhat prohibitive. Finally, and perhaps most importantly, throughout the term students are consistently applying what they are learning and receiving timely, targeted feedback regarding their progress and understanding. This combination improves motivation and engagement and enhances long-term retention that helps students be successful on their own.
The large question banks Garbin has created to support his instructional strategy means that many of the items are quite similar to each other, often differing by only a few words or values. Sometimes the differences change what is the correct answer, and sometimes they don’t. This item-generation strategy means students figure out quite quickly to read every word carefully before choosing an answer.
If you would like to learn more about creating large test banks like these, download “Creating more items more quickly in Canvas,” a guide created by Garbin and Sydney Brown from the Center for Transformative Teaching. To learn more about the details of PSYC 350, visit the course home page.
In the fall 2020 semester, Garbin will be teaching this course in the flexible hybrid format. He will be making use of the Canvas calendar scheduler tool to allow students to sign-up for in-person lectures. Labs for the course will be in-person unless students opt for the online format. In both labs and lectures, the class sessions will be streamed and recorded, allowing students to attend in-person, online synchronously, or online asynchronously. Students may also download Garbin's Powerpoint slides.
