When is Multiple Choice a good Choice?

I'm co-developing a workshop on the BB testing tool.  My role is to add the pedagogy to the technology.  While the tool has more to it than multiple choice questions I want to start there -  when is multiple choice a good choice?  

Step 1 for faculty:  Re-read the learning outcomes for your course.  Every course needs to ensure the learning outcomes are covered in the material.  They don't all have to be assessed, but many of them should be.

• Decision Point 1:  Which outcomes need to be assessed?
• Decision Point 2:  Which outcomes require a graded assessment and which are more appropriate in a self-assessment model?

Now you have a list of the outcomes you want to assess.  It's a good idea at this point to think about time, both yours and your learner's.  I am unable to give up the discussion area - this is the place in the course where instructors facilitate the material and interact with learners most directly.  It's critical that you leave enough time in your schedule to actively manage and participate in discussions.  (Note also that discussions do require learners to write.  Discussions can be casual or formal; they can require straight-forward questions or simple comments or they can require essays and complex analysis.)

So, some of your time will be spent in the discussions.  Some will be spent grading.  Requiring essays or significant projects without allowing adequate time for the provision of extensive feedback is not a good idea, so keep that in mind as you design your assessments.   And some outcomes can be assessed with an automatically-graded multiple choice test.

When to use MC:

1. When your goal is to assess straight-forward vocabulary and recall-type requirements/learning outcomes.  Traditional stem/3 or 5 answers multiple questions can give your learners a chance to practice recalling facts and vocabulary questions, particularly for introductory, freshman-level courses.  These types of questions can be productively used both in a graded-assessment and in non-graded, self-assessment scenarios.  Self-assessment quizzes can encourage learners to read the text and other materials.
2. When asking learners to do scenario analysis.  Set the stage with a description of a scenario, then ask a series of MC questions about the scenario.
3. When learners are going to be working in groups.  It is possible to use question pools such that each learner in the group receives a different set of questions.  By the time a group has worked through each permutation of the exam they will understand the material.  Develop question pools by replacing key numbers or phrases in each question with different numbers of phrases.
4. When it is to the benefit of learners to take a quiz or exam more than once.  See question pools above.  This is especially valuable when learners have significant test anxiety or when they might not read at the level required by the quiz or test.
5. When it's appropriate to have a variety of assessment techniques in a course.  Again, particularly in courses where learners need a lot of encouragement or may be more likely to experience test/school anxiety it's a good idea to include as many self-assessment activities as possible.  The automatic grading feature of MC quizzes make them an invaluable tool for self-assessment.

Writing good MC questions:

From Faculty Focus, Creating Better Multiple-Choice Tests for Online Courses, by Patti Shank, PhD, CPT, May 2009

• Provide clear directions. Group questions with the same directions together.
• Include as much of the question as possible in the stem, and reduce wordiness of alternatives.
• Include words in the stem that would otherwise be repeated in each of the alternatives.
• Make sure language is precise, clear, and unambiguous. Include qualifiers as needed, but don’t add unnecessary information or irrelevant sources of difficulty.
• Avoid highly technical language or jargon unless technical knowledge and jargon are part of the assessment.
• Avoid negatives and these words: always, often, frequently, never, none, rarely, and infrequently. When a negative is used, it should be CAPITALIZED, underlined, or bolded to call attention to it.
• Don’t use double negatives or double-barreled questions (asking two things in one question). 

Other examples of what to do or not to do available here:  http://web.utk.edu/~mccay/apdm/mchoice/mc_b.htm.   

Photo Credit: http://www.flickr.com/photos/maistora/3014414972/