Part I of this tutorial gives some tips on writing more questions more quickly. Part II outlines how to do it in Canvas to create larger groups of questions more quickly. This tutorial was created by Cal Garbin, Emeritus Professor of Psychology, and Sydney Brown, assistant director, Center for Transformative Teaching.
Part I Multiplying Items
No matter where you get your multiple-choice items (write them yourself, use the item bank from your current text, get item banks from other texts, gifts from friends, etc), sometimes it is useful to have more items!
It is useful to remember that item banks are written to cover the material in an efficient number of items. Often, when you want more items, you are not looking for efficient coverage. What you’re probably looking for is redundant coverage. And that is our clue for how to multiply multiple choice items!
Below are some examples of how to start with an item and turn it into multiple items that cover the same content domain. For a specific test we are likely to only want to include one or a few of that set of items, but for building a set of alternate tests, or for providing practice with the content domain, more is better!
If you have “enough items” you can sample from the same (large) item pool for both practice and assessments. Or, you can have them do enough “reps” that they are likely to have learned the material (say, if proctored testing is not available, just to pick an example…). One nice feature of using the same item bank to practice and test or to practice extensively, is that you can tell students, “the more you practice, the more likely you will see familiar items again”. Also, the structure of similarities and differences among items constructed this way promotes “reading every word, every time”.
EG#1 Write sets of questions with the same array of responses
Canvas question type: Multiple-Choice
One strategy is to write a set of questions so that each response is correct for one of them.
This example is taken from a chapter on perception.
The original question and choices was:
_________, or expectations, can allow us to recognize objects faster.
- Schema
- Looming
- Proximity
- Size constancy
I looked through other nearby questions and thought about the question and added a few possible answers to the list. e. Motion parallax
- Retinal disparity
- Expectancy
Now I have 7 response options for the question. I like to have longer lists of response options. Because, the idea is to get as much redundant domain coverage as possible and to engage the student in learning for longer, having to take longer to answer each question isn’t a problem!
Last step – write the items for which the other six response options are the correct answers. Now you have seven questions! Each of the 7 resulting questions uses one of the stems below and all seven of the response options above.
_________ a position or motion cue that tells us something is approaching.
_________ items that are closer together are more likely to part of the same group
_________ allows us to perceive an object as being the same size despite a change in the size of the retinal image.
_________ a position or motion cue that uses speed to identify position
_________ uses differences between retinal images to identify position
_________ things that are more common can be recognized more easily
EG#2 Combine for “compare and contrast”
Another way to add items and also make items more difficult and a bit more conceptual is take two items and write them together into a third question, a sort of “compare and contrast” item.
Start with these two questions…
_________, or expectations, can allow us to recognize objects faster
_________ things that are more common can be recognized more easily
Where both have the following seven possible responses:
a. Schema
- Looming
- Proximity
- Size constancy
- Motion parallax
- Retinal disparity
- Familiarity
And write them into a single item…
_________, or expectations, can allow us to recognize objects faster, while _________ means that things that are more common can be recognized more easily.
And use the same seven response options as above, but now ask them to pick the two answers that would fit in the blanks. This approach puts students in the position of identifying and separating similar concepts like these – which is a little more difficult than a single-response style question.
Canvas question types: Multiple-choice, Multiple Drop-downs, Multiple fill-in-the-blank, and Matching may be used for this effect. If using multiple-choice, list the responses like this:
- schema, familiarity
- proximity, looming
- familiarity, schema
.
. .
Using this approach forces students to pick the correct answer for each part of the question. Students are often better at the “contrast” part than the “compare” part of compare and contrast performance. If you use both this kind of item and the single-response type shown above, they are practicing both “compare” and “contrast” across items!! If your testing system has "matching" type items, you can also use it to practice identifying and separating similar concepts like these!
EG#3 “Not a…” for more compare and contrast practice
I like “not a…” questions… they are another kind of practice with “compare & contrast” types of thinking!
For this example, we need only one question to build 5 items…
Which of the following is not a sense?
Here’s how…
Every one of the 5 items will include all of these
- Hearing
- Vision
- Olfaction
- Touch
- Taste
Each item will include one of these (the right answer)
- Analgesia
- Kinesthesia
- Synesthesia
- Induction
- Conduction
So, the first item would look like …
Which one of these is not a sense?
- Hearing
- Vision
- Olfaction
- Touch
- Taste
- Conduction
A second item would look like …
- Hearing
- Vision
- Olfaction
- Anagesia
- Touch
- Taste
Again, you could add items by using a “pick all the correct answers” version, maybe with the question…
Pick each of the following that is not a sense?
- Hearing
- Vision
- Olfaction
- Touch
- Taste
- Analgesia
- Kinesthesia
- Synesthesia
- Induction
- Conduction
EG#4 By writing more involved stories, you can use each story you write for several items….
The point of the item set is to practice identifying 4 kinds of variables – causal variables, effect variables, subject variable confounds, procedural variable confounds. Here’s a story written around 6 variables (see below).
Mary has designed a study to look at the causal relationship between immediate environment and feelings of positivity. All participants were run on the same day, between 11:00 am and 1:00 pm. Mary allows participants to decide whether they will complete the packet while sitting at one of several picnic benches outdoors in the library patio or indoors at one of the individual carrels in the main study room or the library. Mary is working with two other researchers, and the bulk of the packet is collecting data for them. The packet being completed by those outside is about attitudes towards sunbeams and unicorns, and has the 10 positivity questions for Mary’s study at the end. The packet being completed by those inside is about attitudes towards rain clouds and flesh-eating viruses, and has Mary’s 10 positivity questions at the end. Mary noticed that nearly all the females chose to complete the questionnaire inside and the males were about evenly split between indoors and outdoors. She also noticed that people sitting inside tended to be coming from the cafeteria and those sitting outside tended to go to the cafeteria after they were done.
First set of 4 items from this story – telling what kind of variable is each of the six variables in the story…
For the first set of 4 items, we would use the following 4 stems…
- Pick the intended causal variable
- Pick the intended effect variable
- Pick all the subject variable confounds
- Pick all the procedural variable confounds
For each item, these would be the six response options…
- Before vs after eating lunch
- Immediate environment
- Sex of the participant
- Feelings of positivity
- Answering positive questions (sunbeams & unicorns) vs negative questions (rain clouds and flesh-eating viruses)
- Sitting alone (carrel) or with a group (picnic table)
The second set of 6 items just “flips” the questions and response options – this is a great way to double the number of items you have!!
For the second set of 6 items, we would use the following 6 stems…
- What sort of variable is “before vs after eating lunch”?
- What sort of variable is “Immediate environment”?
- What sort of variable is “Sex of the participant”?
- What sort of variable is “Feelings of positivity”?
- What sort of variable is “Answering positive questions (sunbeams & unicorns) vs negative questions (rain clouds and flesh-eating viruses)”?
- What sort of variable is “Sitting alone (carrel) or with a group (picnic table)”?
For each item, these would be the four response options…
- The intended causal variable
- The intended effect variable
- A subject variable confound
- A procedural variable confound
Part II: Using these strategies in Canvas to rapidly make question banks
Question banks are collections of topically related items that allow instructors to create a quiz that randomly pulls a designated number of items from each selected question bank to generate a unique quiz for each student. Questions may also be selected manually from question banks to construct a quiz.
To leverage the aforementioned techniques when constructing Canvas question banks, we’ll do the following four steps:
- Create an item bank with one of each type of the questions we plan to use. This will be our template to copy and change later – instead of writing each question from scratch each time.
- Duplicate the item bank template.
- Go into the item bank and edit the items, use the duplicate item option to rapidly create questions that use the same answer set or question stem.
- Create a quiz/exam for students by pulling from the question bank at random, manually selecting particular items, or selecting all the items.
