Correct answer vs Best answer

Submitted by :
Ahniemay D. Sabile

Basic Tips for MCQ’s (July 20, 2006)

Page 1 of 4

Basic Tips for Writing Effective Multiple Choice Questions (MCQ’s):

A Compilation of the Most Useful Advice

Jennifer Murdock, Department of Economics, U of T

To define terms, here is a sample multiple choice question where the correct answer is (E):

(20) If marks are normally distributed with mean

68 and standard deviation 15, what percent of

students have a mark of 79 or higher?

(A) 15

(B) 17

(C) 19

(D) 21

(E) 23

(1) List the alternatives vertically and in a logical order. These two points are illustrated by

the sample multiple choice question above. Students who know the correct answer can

quickly find it. Also, a jumbled order can be distracting to some students.

(2) Make the question as direct and clear as possible. Questions can degenerate into a test

of logic if you use double negatives and complex structures.

(3) Make sure there is only one unambiguously correct response.

(4) Create plausible distracters that would sound right to an incompetent student but are

clearly wrong. You can include common misconceptions, common mistakes, and

technical jargon to make distracters more effective.

(5) Instruct students to choose the “best answer” rather than “the correct answer.” This helps

avoid argument and allows you to skip specifying rounding conventions (for example, the

answer to the sample question above was 23.1678…).

(6) Write the question stem such that students can quickly and easily deduce what the

question is asking. Students should not have to read all of the alternatives to figure this

out. When possible, write stems as you would an open-ended question such that if a

proficient student were given the question stem s/he could write out the correct response.

(7) Don’t get too fancy with your English and the subtleties of the English language. If you do,

students who do not have English as their first language may get your question wrong

even though they knew the concept you were trying to test.

(8) Use negatives sparingly and emphasize them if used. For example, “For which of the

following population density functions is the population mean NOT equal to the population

median?” There is some evidence that ESL students are particularly disadvantaged by

negatives.

(20) If marks are normally distributed with mean

68 and standard deviation 15, what percent of

students have a mark of 79 or higher?

(A) 15

(B) 17

(C) 19

(D) 21

(E) 23

Question Stem

Distracters

Answer

Alternatives

Basic Tips for MCQ’s (July 20, 2006)

Page 2 of 4

(9) While you may want to emphasize words in the question stem to help comprehension,

emphasis in the alternatives should be avoided.

(10) Keep the question stem and alternatives as short as possible. Use few words. Avoid

repeating words from the question stem in the alternatives.

(11) Before writing a question, think about what it is that you want to test. Lecture notes,

textbook readings, assigned problems, and other course materials can be inspiration.

(12) Use “None of the above” with caution. You can make straightforward numeric calculation

questions more challenging by including “None of the above” as an option (may need to

specify rounding conventions). For other types of questions, you need to think carefully

about whether there are some plausible arguments a proficient student could make to

support choosing this alternative when you intended it as a wrong answer. If you want to

save time writing questions don’t use “None of the above.”

(13) Do not include alternatives such as “Both (A) and (D)” or “All but (C)” as these complicate

the structure of the question and tend to confuse students and/or slow them down. If you

want to convince yourself, look at someone else’s questions that use these and see how

much harder it is to focus on what you’re supposed to be doing.

(14) Be aware of the difficulty level of each question. Make sure you have a sufficient number

of easy and more challenging questions so that you will be able to separate “F” students

from “D” students, “D” from “C” students, “C” from “B” students, and “B” from “A” students.

Easier questions test a student’s knowledge. For example, do they know what selection

bias is. Medium difficultly questions test comprehension. Does a student understand

under what conditions selection bias might arise? Harder questions test a student’s ability

to apply concepts and do analysis. For example, give students a scenario where they

need to realize that selection bias would be a concern (without being told) and to

understand the implications of that bias on inference in that case.

(15) Try to make the first few multiple choice questions relatively quick and easy to help calm

student down so they can focus on the more challenging questions to come.

(16) Avoid the temptation to test many things in one question. If it is possible, try to write more

than one multiple choice question rather than test multiple concepts in one question.

Testing too many things in one question reduces your ability to discriminate amongst

students with differing levels of understanding. Further, students get upset because there

is no partial credit.

(17) Ask more than one question when a fair amount of information must be provided as it

takes time for students to carefully read and understand the information you provide in a

test. For example, you could give them a table of results, a graph, or a scenario and then

ask two or three different multiple choice questions about it.

(18) Five alternatives (A) – (E) are recommended. You cannot include more than five with our

Scantron forms. You could include only three or four, but this increases the expected

value of guessing. There is no reason all of your questions have to have the same number

of alternatives.

Basic Tips for MCQ’s (July 20, 2006)

Page 3 of 4

(19) This table shows some common strategies test-wise students, who are skilled test takers

but not proficient in the course material, can use to guess correct answers and how you

can respond. Some are adapted from Russell A. Dewey, “Writing Multiple Choice Items

which Require Comprehension” http://www.psywww.com/selfquiz/aboutq.htm.

Strategy of test-wise student Response (counter-measure)

Pick B Make sure each alternative (A – E) is correct

only about a fifth of the time

Pick the longest alternative

Make sure the longest alternative is correct

only about a fifth of the time or try to make

alternatives roughly the same length

Do not pick A

Make sure A is correct about a fifth of the

time: studies have found that test writers

avoid A as they feel it’s too obvious

Do not pick an alternatives that includes

“always” or “never”

Never use these words or make sure to use

them in the correct answers about a fifth of

the time

Pick the alternative that includes vague words

like "usually," "typically" and "may be" Use vague words in some of the distracters

If there are two alternatives which express

opposites, pick one or the other and ignore

other alternatives

Sometimes offer opposites when neither is

correct

If in doubt, guess Use five alternatives instead of three or four

Pick the answer that uses technical words or

phrases you learned in class (i.e.

heteroscedasticity, endogenous …)

Use such words or phrases in wrong

answers

Don't pick an answer which looks too simple

or obvious

Sometimes make the simple, obvious

answer the correct one

Pick “All of the above” Make sure that “All of the above” is correct

only about a fifth of the time

Pick “None of the above” Make sure that “None of the above” is

correct only about a fifth of the time

Basic Tips for MCQ’s (July 20, 2006)

Page 4 of 4

(20) Do not try to write the entire test in one day: it takes time, creativity and thought to write

good multiple choice questions. You could write a few each week as you teach the

material or as you get ideas based on students’ questions or performance on homeworks.

(21) Come back to the questions you’ve written a day or two later with a fresh eye.

(22) Ask trusted TA or a colleague to try your questions to sort out any ambiguities (especially

if you’re using these questions for a final examination).