Tuesday, August 14, 2012

Article:Table Specification


PLANNING THE UNIT TEST
  • Start with curriculum guide
  • then ope rationalize by writing up series of objectives
  • can do a quick review of writing Behavioral Objectives, if you would like.
SPECIFICATIONS CONSTRUCTION
  • A table that shows what will be tested (and taught)
  • Theoretically, a completely detailed table of specifications would have every learning objective listed for every lesson for the whole year
    • things haven't gone quite that far here in Alberta
    • not sure there is really a point to having a document that lists every single fact students are to know
    • not only is this too inflexible -- because it wouldn't allow for any room for teacher to respond to student needs,

      it is also reductionism
      • try to reduce learning to individual skills, misses that education is more than the sum of its parts
      • part of difference between training and education I talk about in Social Context

    • currently this totally detailed approach is dominant one in England
    • some movement toward that end of the continuum here: "competency based" education is an attempt to move towards defining education in terms of a finite number of specific competencies
    • so we do not need that level of detail --> main topics for year, main concepts for a unit plan good enough
Sample Table of Specifications


Bloom's Taxonomy
Subject
Content
Knowledge & Comprehension
Application
Analysis, Synthesis &Evaluation
TOTALS
Topic A
10%
20%
10%
40%
Topic B
15%
15%
30%
60%
TOTALS
25%
35%
40%
100%

  • usually a two sided chart used in construction of tests
  • content down one side, cognitive levels across the top
  • common format in Alberta, but no rule: could have content across the top, Bloom' down the side
    • usually group Bloom' categories: in this example, knowledge, understanding, and higher mental activity
    • I prefer grouping knowledge/understanding (because straight recall usually too simple to count as real learning) and than application, then analysis, synthesis and evaluation as higher level
    • for more on Bloom's Taxonomy, please see Glossary

  • Content usually much more detailed than this, but will use two categories here to keep illustration simple
  • totals tell you at a glance what percentage of course emphasis given to each topic and what percentage lower and higher level mental processes


Here is an example of more detail


CONTENT
Economic Growth: USA
Economic Growth: USSR
CANADA: Respondind to Change
TOTAL
PROCESSES
Industrialization
Market Economy
Quality of Life
Geography
Industrialization
Centrally Planned Economy
Quality of Life
Technology
Mixed Economy
Quality of Life

KNOWLEDGE AND
COMPREHENSION

Recall Facts

Understand Concepts and Generalizations
17%
17%
18%
52%
PROCESS SKILLS A

Locating

Interpreting

Organizing
8%
8%
8%
24%
PROCESS SKILLS B

Analyzing
Etc...










  • Example of running content across the top, Bloom' down the side
  • notice that some curriculum' translate Bloom into subject specific taxonomy, but principle is the same


Table of Specifications Relates the Outcomes to the Content and Indicates the Relative weight of each area
  • weight is usually based on how much time devoted to teaching concept
    • but also how important it is that students remember, transfer to other contexts, courses --> some important ideas may be easy to teach but still important to include
    • also determined by type of material --> don't put a lot of weight on higher mental activity category for unit on memorizing state capitals --> don't put a lot into recall for drama class on risk taking and creative dance
    • weight -- start simple --> four topics, divide into 4, then maybe add bit more to topic you are particularly interested in, or figure students will be interested in, etc.
Sample Table of Specifications
For
Art History and Appreciation



Bloom's Taxonomy
Subject
Content
Knowledge & Comprehension
Application
Analysis, Synthesis &Evaluation
TOTALS
Art History
40%
5%
5%
50%
Art
Appreciation
5%
5%
40%
50%
TOTALS
45%
10%
45%
100%


Rationale:
  • Both topics equally important so = 50% each
  •  
  • art history requires more memorization of names and dates, so relatively more weight on knowledge and comprehension than on analysis, synthesis, or evaluation.
  •  
  • art appreciation requires student to analyze art object and synthesize personal response to produce critique (evaluation), so this unit will have relatively more weight on higher mental activities
  •  
  • table achieves overall balance between lower and higher mental activities
  • weight usually given in %, but you can use marks (e.g., 50) if you like
  • usually out of 100%, but might make two separate blueprints, one for 70 multiple choice, and second 30% for written response
  • Acts as a:
  • blueprint for teaching --> don't just start teaching page one on day one, or suddenly discover that its Easter and you're still on first unit --> need to figure out how much time you're going to allocate per unit, per concept within units
  • blueprint for the test
  • So that we get:
  • representative sample of course content -->not all random sample
    • this is important so that you don't just choose questions from last two weeks before exam

  • representative sample of skills, cognitive levels across content
    • not just rote memorization; or just high level stuff
    • often sabotage great course by teaching high level skills (sculpting, acting, playing solo) then giving rote memorization test (date that Mozart composed 43rd symphony) that does not reflect actual time spent

      kids learn quickly what actually &#34counts"is stuff on test, so if you have rote memorization test, don't try to get class discussion going!

  • analyze results by level and content area
    • if students getting all lower level questions but missing higher level, then you're not doing your job; if all have got answers to one unit but not another, may have to reteach that unit, etc.
Do classroom teachers actually do this?
  • No, but most of them have not had the benefit of your training.
  • Part of my job when I worked for Student Evaluation branch was to do inservice workshops at PD days and teacher conventions and on item writing committees; teachers were always surprised and pleased by this obvious concept.
  • So more teachers are doing this each year.
  • Now, most principals will want to see your year plans, and expect some evaluation planning as part of it.
  • It is becoming a standard part of unit planning
Strictly speaking there is a difference between a Table of Specifications and a Blueprint:
  • Specifications refer to a plan of what is to be taught/tested by weighting
  • A blueprint is the plan of the specific test, i.e., which questions test which concept
  • So same specifications could give rise to several different blueprints
  • But mostly we use terms interchangeably.
Sample Blueprints For
Art History and Appreciation




Bloom's Taxonomy
Subject
Content
Knowledge & Comprehension
Application
Analysis, Synthesis &Evaluation
Total # of Items
Art History
1,2,3,4,5,6,7,8
9
10
10
(50%)
Art
Appreciation
13
12
11,14-20
10
(50%)
Total # of Items
9 (45%)
2 (10%)
9 (45%)
20 (100%)




Bloom's Taxonomy
Subject
Content
Knowledge & Comprehension
Application
Analysis, Synthesis &Evaluation
Total # of Items
Art History
1,2,3,4,5,6,
7,8,9,10,19,20,
21,22,23,24
11,14
12,13
20
(50%)
Art
Appreciation
15,16
17,18
25-40
20
(50%)
Total # of Items
18 (45%)
4 (10%)
18 (45%)
40
(100%)

·  NOTE how the same Table of Specifications can give rise to any number of different Test Blueprints - the same proportions on each test, but different questions

PHYSICS 30 DIPLOMA EXAM BLUEPRINT
The following table is used by Alberta Education in reporting Diploma exam results.

Distribution of Questions by Percentage of Total Mark



Cognitive Level
Concept&#178
Knowledge
Comprehension
and
Application
Higher
Mental
Activities
Exam
Emphasis %
Nature and Behavior
of Light
6
12
5
23
Electric and Magnetic
Fields
6
16
5
27
Electromagnetic
Radiation
4
8
3
15
Structure of Matter
5
11
4
20
Modern Physical
Theories
4
8
3
15
Total
25
55
20
100


Sociology Specifications Table

Learning Outcomes
Subject Content
Knowledge &
Comprehension
Higher Mental
Activities
Totals
I. Introduction
0%
2%
2%
II. Sociological Perspectives
1. Overview of Sociological
Paradigms
A) Consensus Theories
B) Conflict Theories
C) Interaction Theories
8%
12%
20%
III. Schooling & Political Eco.
1. Schooling & the Human
Capital Model
2. Education Policy in Canada
3. The Alberta Context
8%
10%
18%
IV. Educ., Social Mobility, &
Equality of Ed.Opp.

1. Social Mobility
2. Equality of Educational
Opportunity
3. Educational Assessment
and the Social Construction
of Individual Differences
4. Schooling, Social
Reproduction & Legitimation
5%
30%
35%
V. Hidden Curriculum
1. Content
2. Process
2%
18%
20%
VI. Professionalism
2%
3%
5%
TOTALS
25%
75%
100%
  • class time spent discussing Hidden Curriculum = 20%

    - so I have a term paper on hidden curriculum worth 20% of course grade
  •  
  •  

V. Hidden Curriculum
1. Content
2. Process
10%
(2%)
90%
(18%)
100%
(20%)
  •  
  • -structure the assignment so that 10% deals with their understanding of concept of hidden curriculum; 90% on higher level thinking.
Reference:
http://www.uleth.ca/edu/runte/tests/specf/specf.html



2. www.biz.colostate.edu/MTI/summer/.../3-2Grading.pdf - United States

No comments:

Post a Comment

Post a Comment