Monday, July 30, 2012

Correct answer vs Best answer

Submitted by :
Ahniemay D. Sabile


Basic Tips for MCQ’s (July 20, 2006)
Page 1 of 4

Basic Tips for Writing Effective Multiple Choice Questions (MCQ’s):
A Compilation of the Most Useful Advice
Jennifer Murdock, Department of Economics, U of T


To define terms, here is a sample multiple choice question where the correct answer is (E):

(20) If marks are normally distributed with mean
68 and standard deviation 15, what percent of
students have a mark of 79 or higher?
(A) 15
(B) 17
(C) 19
(D) 21
(E) 23

(1) List the alternatives vertically and in a logical order. These two points are illustrated by
the sample multiple choice question above. Students who know the correct answer can
quickly find it. Also, a jumbled order can be distracting to some students.

(2) Make the question as direct and clear as possible. Questions can degenerate into a test
of logic if you use double negatives and complex structures.

(3) Make sure there is only one unambiguously correct response.

(4) Create plausible distracters that would sound right to an incompetent student but are
clearly wrong. You can include common misconceptions, common mistakes, and
technical jargon to make distracters more effective.

(5) Instruct students to choose the “best answer” rather than “the correct answer.” This helps
avoid argument and allows you to skip specifying rounding conventions (for example, the
answer to the sample question above was 23.1678…).

(6) Write the question stem such that students can quickly and easily deduce what the
question is asking. Students should not have to read all of the alternatives to figure this
out. When possible, write stems as you would an open-ended question such that if a
proficient student were given the question stem s/he could write out the correct response.

(7) Don’t get too fancy with your English and the subtleties of the English language. If you do,
students who do not have English as their first language may get your question wrong
even though they knew the concept you were trying to test.

(8) Use negatives sparingly and emphasize them if used. For example, “For which of the
following population density functions is the population mean NOT equal to the population
median?” There is some evidence that ESL students are particularly disadvantaged by
negatives.

(20) If marks are normally distributed with mean
68 and standard deviation 15, what percent of
students have a mark of 79 or higher?
(A) 15
(B) 17
(C) 19
(D) 21
(E) 23
Question Stem
Distracters
Answer
Alternatives
Basic Tips for MCQ’s (July 20, 2006)
Page 2 of 4
(9) While you may want to emphasize words in the question stem to help comprehension,
emphasis in the alternatives should be avoided.
(10) Keep the question stem and alternatives as short as possible. Use few words. Avoid
repeating words from the question stem in the alternatives.
(11) Before writing a question, think about what it is that you want to test. Lecture notes,
textbook readings, assigned problems, and other course materials can be inspiration.
(12) Use “None of the above” with caution. You can make straightforward numeric calculation
questions more challenging by including “None of the above” as an option (may need to
specify rounding conventions). For other types of questions, you need to think carefully
about whether there are some plausible arguments a proficient student could make to
support choosing this alternative when you intended it as a wrong answer. If you want to
save time writing questions don’t use “None of the above.”
(13) Do not include alternatives such as “Both (A) and (D)” or “All but (C)” as these complicate
the structure of the question and tend to confuse students and/or slow them down. If you
want to convince yourself, look at someone else’s questions that use these and see how
much harder it is to focus on what you’re supposed to be doing.
(14) Be aware of the difficulty level of each question. Make sure you have a sufficient number
of easy and more challenging questions so that you will be able to separate “F” students
from “D” students, “D” from “C” students, “C” from “B” students, and “B” from “A” students.
Easier questions test a student’s knowledge. For example, do they know what selection
bias is. Medium difficultly questions test comprehension. Does a student understand
under what conditions selection bias might arise? Harder questions test a student’s ability
to apply concepts and do analysis. For example, give students a scenario where they
need to realize that selection bias would be a concern (without being told) and to
understand the implications of that bias on inference in that case.
(15) Try to make the first few multiple choice questions relatively quick and easy to help calm
student down so they can focus on the more challenging questions to come.
(16) Avoid the temptation to test many things in one question. If it is possible, try to write more
than one multiple choice question rather than test multiple concepts in one question.
Testing too many things in one question reduces your ability to discriminate amongst
students with differing levels of understanding. Further, students get upset because there
is no partial credit.
(17) Ask more than one question when a fair amount of information must be provided as it
takes time for students to carefully read and understand the information you provide in a
test. For example, you could give them a table of results, a graph, or a scenario and then
ask two or three different multiple choice questions about it.
(18) Five alternatives (A) (E) are recommended. You cannot include more than five with our
Scantron forms. You could include only three or four, but this increases the expected
value of guessing. There is no reason all of your questions have to have the same number
of alternatives.
Basic Tips for MCQ’s (July 20, 2006)
Page 3 of 4
(19) This table shows some common strategies test-wise students, who are skilled test takers
but not proficient in the course material, can use to guess correct answers and how you
can respond. Some are adapted from Russell A. Dewey, “Writing Multiple Choice Items
which Require Comprehension” http://www.psywww.com/selfquiz/aboutq.htm.
Strategy of test-wise student Response (counter-measure)
Pick B Make sure each alternative (A – E) is correct
only about a fifth of the time
Pick the longest alternative
Make sure the longest alternative is correct
only about a fifth of the time or try to make
alternatives roughly the same length
Do not pick A
Make sure A is correct about a fifth of the
time: studies have found that test writers
avoid A as they feel it’s too obvious
Do not pick an alternatives that includes
“always” or “never”
Never use these words or make sure to use
them in the correct answers about a fifth of
the time
Pick the alternative that includes vague words
like "usually," "typically" and "may be" Use vague words in some of the distracters
If there are two alternatives which express
opposites, pick one or the other and ignore
other alternatives
Sometimes offer opposites when neither is
correct
If in doubt, guess Use five alternatives instead of three or four
Pick the answer that uses technical words or
phrases you learned in class (i.e.
heteroscedasticity, endogenous …)
Use such words or phrases in wrong
answers
Don't pick an answer which looks too simple
or obvious
Sometimes make the simple, obvious
answer the correct one
Pick “All of the above” Make sure that “All of the above” is correct
only about a fifth of the time
Pick “None of the above” Make sure that “None of the above” is
correct only about a fifth of the time
Basic Tips for MCQ’s (July 20, 2006)
Page 4 of 4
(20) Do not try to write the entire test in one day: it takes time, creativity and thought to write
good multiple choice questions. You could write a few each week as you teach the
material or as you get ideas based on students’ questions or performance on homeworks.
(21) Come back to the questions you’ve written a day or two later with a fresh eye.
(22) Ask trusted TA or a colleague to try your questions to sort out any ambiguities (especially
if you’re using these questions for a final examination).

1 comment:

  1. Good job! can you post the site that you got this from.

    ReplyDelete