Without clear goals, we cannot readily assess the effectiveness of our efforts or realize when we are off course, how far off we are, and how to get back on the right track.
Although assessments may be administered somewhat differently in an online course than in a face-to-face course, you plan for them in the same way – by asking yourself: 1) what do your students need to know or be able to do? and 2) what would be the best evidence of the extent of their progress toward this goal?
The following resources address both online and in-person classes. However, most in-person techniques have online adaptations and if you'd like assistance, please let us know.
Resources from other institutions & organizations
- Good, Better, Best: Multiple Choice Exam Construction - Great introduction to writing high-quality multiple-choice items with examples.
- Writing Good Multiple-Choice Test Questions - Vanderbilt Center for Teaching Guide. Similar to the above link, but includes a section on writing for higher-order thinking skills.
- Fourteen Simple Strategies to Reduce Cheating on Online Examinations
Course goals are often stated in a "big-picture" way, represent where students should end up, and employ the word "understand". Unfortunately, "understand" does not suggest what type of assessment might be most appropriate. In fact, it often is used when the goal encompasses several different types of knowledge and skills. For example, "understand design thinking," could mean students should be able to describe an ideation process in addition to using design thinking to create a new product. Note that "describe" and "using" both imply ways students might demonstrate their understanding of design thinking. Consequently, learning objectives, which are usually applied to units or topics within a course, are a great place to look for clues as to what type of assessment would be a valid and reliable measure of student learning.
A well-written learning objective has three key components: performance, conditions, and criteria. When students read the learning objective they understand what they will need to do, the conditions under which they will do it, and the criteria by which their performance will be judged.
"Students will be able to use design thinking to address problems after working with dining services to generate a solution for food waste."
This example learning objective could also be a course goal and is authentic in that the assessment resembles or simulates the "real world," for which students are being prepared. Not all course learning can be assessed in hands-on authentic ways, but with a little creativity, much can.
To explore what types of learning goals are essential for a course, try out the iTGI, an interactive version of the Teaching Goals Inventory (Angelo and Cross, 1993). To explore classroom assessment techniques by the type of learning goal you have, make use of the classroom assessment explorer.
Making Assignments Authentic
An assignment is “authentic” if it resembles what a person might do in the real world with the knowledge and skills that you aim to teach students in your course. For example, if your course aims to teach students research methods, then an authentic task might involve students preparing a research design. In the real world, a person with expertise in research methods will likely design, carry out, and evaluate research projects, so an assignment requiring students to do any or all of these things would be authentic. In contrast, many multiple-choice exams tend to be relatively inauthentic. When, perhaps outside of a professional certification process, are people in your field given a limited time period, and no access to resources, and asked to select correctly from a set of available options? If often, then a multiple-choice exam may be the best choice in your course. If not, then think about how you can design assignments that come closer to resembling the real-world circumstances your students will likely encounter in the future. These types of assignments tend to involve high levels of student engagement – they get students thinking, creating, problem-solving, consulting with each other and the instructor, etc. They also tend to contribute to community-building: In the real world, people often work on projects collaboratively, or they at least solicit feedback from peers. If your assignments provide opportunities for this type of collaboration, then they will be more authentic, and the sense of community in your course will also benefit.
The importance of practice
Assessments are often described as falling into two categories: formative and summative. Formative assessments are usually informal, ungraded, or low-stakes assignments used to check students’ understanding of course concepts. They are most often used to “form” the instructor’s instructional plan (i.e., whether it’s time to move on to new concepts or spend a little more time on something). They also give students important information about their progress and about areas in which they may be struggling. Summative assessments, on the other hand, test the sum of a student’s knowledge. These are evaluative, and they are higher-stakes, in that they typically constitute a significant portion of a student’s grade.
More Reps for Better Learning
In Psychology 350, Research Methods & Data Analysis, Professor Cal Garbin makes extensive use of large question banks to provide opportunities for his students to do enough "reps," or practice, that they develop the skills and knowledge needed to approach course assessments with confidence. Garbin’s approach combines many assignment exercises with strategic proctored testing. Read more...
Although there is nothing wrong with relying on summative assessments – almost all courses use them – what is more problematic is the under-use of formative assessments. Formative assessments are, to a great degree, practice. They allow students to attempt a task (like writing a rough draft of an essay or completing quiz questions that resemble what they will later find on an exam) without suffering too great a setback if the attempt does not go as well as it should. In many courses, however, few if any formative assessments are built into the assessment structure. This is not to say that students could not practice their skills in these courses. However, if they do this practice entirely on their own – relying on their motivations, resources, and timelines, and without any feedback from the instructor – then it is not nearly the quality of practice they would enjoy if it were built into the course as formative assessments. Students would then have critical guidance from the instructor, making them much more likely to remain on track and practice in ways that truly move them toward mastery.
In almost any other context, such as athletic competition, we would quickly recognize the absurdity of expecting someone to excel at a task without practice. If the Nebraska volleyball team were to end all of their formal practice sessions, and if the players were told to practice on their own time and then meet up only for matches, then we can be confident that their exemplary performance record would quickly crumble. Yet this is essentially what happens in a course that contains only summative assessments (official matches) and no formative assessments (practice sessions).
As you figure out how to complete assessments in your online course, remember the importance of practice, and build informative assessments wherever you can. Common options include quizzing in Canvas, multiple drafts of paper or projects submitted overtime in preparation for final submission, reflection assignments, polling during a lecture, and blind discussion responses.
Proctoring and Academic Integrity
Resources related to proctoring & academic integrity
Resources from other institutions & organizations
- Our Bodies Encoded: Algorithmic Test Proctoring in Higher Education from HybridPedagogy.org - Explores the values that are embedded in proctoring software systems and the harm they can cause students.
Maintaining high standards of academic integrity is an essential feature of any class. It can be particularly difficult to prevent academic dishonesty in an online environment. Remote proctoring is one option, though it has some downside and is never fail-safe. Another option, with fewer downsides, is to create assessments that make it difficult to cheat in the first place. This section will briefly review both sets of options.
When a higher-stakes assessment, like a final exam, is necessary for your online course, you may need to consider how to proctor it. Remote proctoring options, like Respondus and ProctorU, allow for some degree of control over the environment (both physical and virtual) in which a student takes an exam. They either regulate students’ test-taking conditions (such as by ensuring that they are able to use only a single browser window during the exam) or monitor them while they complete their exam (typically by recording their screen and webcam). As such, they come with the downside of raising privacy concerns: with most remote proctoring, a student’s every move, both on their computer and in their exam-taking space, is being watched and recorded. In addition, there are technological challenges: some students, especially those who live in rural areas, may not have a stable enough internet connection to maintain the bandwidth that the proctoring service requires; and some students may not have the equipment they need (like a webcam) to satisfy the requirements of the proctoring. It may be advisable or necessary to offer a reasonable alternative (such as a directly comparable written exam) for students who do not have the resources necessary to complete a remote-proctored exam. Finally, it is important to realize that there is no way to guarantee that your students are not using prohibited resources when completing the exam. For example, remote proctoring software may provide good oversight over the machine a student is using to complete the exam, but it may have less ability – if any at all – to observe their use of a separate mobile device during the exam. Despite this, if it is necessary to have students completing a high-stakes assessment, in which no other way of ensuring academic integrity exists (such as those discussed below), then remote proctoring may be the best option for protecting against academic dishonesty.
Remote proctoring is not the only option for ensuring the integrity of a high-stakes assessment. After all, you, as the instructor, get to decide what constitutes cheating. If students are not breaking your rules (or those of the University), then they are not cheating. So is there any way to relax your rules while upholding the validity, rigor, and integrity of your assessment? One approach is to make your exam open-book: allow students to use whatever resources they have at their disposal while taking steps to ensure that the exam remains a good test of what they have learned. Even with an open-book exam that consists of multiple-choice questions, there are ways to improve the integrity of the assessment: when possible, use formula-based questions, for which an algorithm can generate unique versions of the question for each student; use application-based questions that require more than simply looking up an answer; avoid using publisher materials, since answers to these questions can typically be easily located online; and create a large test bank, with a subset of questions chosen at random, and in random order, so that no two students receive the same exam.
Avoiding multiple-choice exams altogether, if your class size permits it, is a more reliable way to ensure the integrity of an assessment, even when resources are available to the student. Clearly, a written exam can be administered open-book without much concern for academic dishonesty, provided that at least some originality safeguards are taken. For instance, you could ask students to apply concepts from class to novel real-world scenarios that they come up with, ensuring that each student is writing about something unique; or you could use a program like Turnitin to check for answers that have been copied from elsewhere. Finally, consider outside-the-box options that, by their very design, make academic dishonesty unlikely. For instance, consider oral exams over Zoom, or recorded presentations – both of which allow you to see your students’ faces and voices as they deliver their work. Creative projects completed over the course of the semester – ideally with some direct consultation with you – can be the gold standard: if such a project makes sense with your subject matter, it might be authentic, highly cheat-proof, and an excellent summative assessment of what a student has learned.