EEA 622�EDUCATIONAL ASSESSMENT��Chapter 3 & 4�Video B� �

​

Dr Kim Teng Siang

kskim2007@gmail.com

0124661131

Chapter 3�USING OBJECTIVE TESTS�

CHAPTER 3 OVERVIEW

3.1 What is an Objective Test? 

3.2 Multiple-choice questions - What is a MCQ?

- Construction of MCQs

- Why are MCQs widely used?

- Limitations of MCQs

3.4 True-false questions

- What is a true-false question? 

- Construction of T-F questions

- Advantages of T-F questions

- Limitations of T-F questions

3.5 Matching questions

- What is a matching question?

- Construction of matching questions

- Advantages of matching questions 

- Limitations of matching questions

3.6 Planning Your Test

​

3.1 What is an OBJECTIVE TEST?

• a written test consisting items or questions which requires the respondent to select from a list of possible answers
• Objective = accurate, not biased

​

Figure 3.1 Common Formats of Objective Tests,

pg 2

• MCQ (multiple-choice question)
• True and False Questions
• Matching Questions

​

3.2 MULTIPLE-CHOICE QUESTIONS�(MCQ)

Anatomy of a Multiple Choice Question, pg 3

- Stem

- Choices of answer

• keyed response (answer)
• distractors (wrong answers)

MCQ - Example

1. What is the capital of China?

a. Hong Kong

b. Shanghai

c. Beijing

d. Kun Ming

​

Stem

Key

(answer)

Distractors

Options or

alternatives

(Choices)

The Stem should

a) be in the form of a question or of a statement to be completed

​

b) be expressed clearly and concisely, grammatically correct, not be ambiguous and free from double negatives

c) be presented generally as positive question. (If a negative is used it should be emphasised with italics or underlining.)

​

d) generally ask for one answer only (the correct or the best answer)

​

The Options or Alternatives (which contains the answer and distractors)

• There should be either four or five alternatives, all of which should be mutually exclusive and not too long

​

• All alternatives should follow grammatically from the stem and be parallel in grammatical form

​

• All alternatives should be expressed simply enough to make clear the essential differences between them, and must be unambiguous

​

​

• The distractors should appear plausible (likely) solutions (or answers) to the problem for those learners who have not achieved the objective (knowledge or skills) being measured by the item (or question).

​

• The distractors should appear as implausible (unlikely) solutions (or answers) for those learners who have achieved the objective of the item.

​

• The solution (or answer) should appear plausible (likely) to these those learners who have achieved the objective of the item (or answer).

​

b) Construction of Multiple-choice Questions, pg 4-10

SEE guideline 1- 13 in the module

​

Why are Multiple-choice Questions Widely Used

• Learning outcomes from simple to complex can be measured.
• Highly structured and clear tasks are provided.
• A broad sample of achievement can be measured.
• Incorrect alternatives or options provide diagnostic information.
• Scores are less influenced by guessing compared to true-false items.
• Scores are more reliable than subjectively scored items (e.g. essays).

​

• Scoring is easy, objective and reliable. 

​

• Item analysis can reveal how difficult each item was and how well it discriminated between the strong and weaker students in the class.

​

• Performance can be compared from class to class and year to year.

​

• Can cover a lot of material very efficiently (about one item per minute of testing time).

​

• Items can be written so that students must discriminate among options that vary in degree of correctness.

​

Limitations in the Use of Multiple-choice Questions

• Constructing good items is time consuming.
• It is frequently difficult to find plausible distractors.
• MCQs are not as effective for measuring some types of problem solving skills and ability to organise and express ideas.
• Scores can be influenced by reading ability.
• There is a lack of feedback on individual thought processes – it is difficult to determine why individual students selected incorrect responses.

​

• Students can sometimes read more into the question than was intended.
• It often focuses on testing factual information and fails to test higher levels of cognitive thinking.
• Sometimes there is more than one defensible “correct” answer.
• They place a high degree of independence on the student’s reading ability and the constructor’s writing ability.
• Does not provide a measure of writing ability.
• May encourage guessing.

​

Procedural rules for the construction of MCQ (Table 3.2 pg 12) - Things to do and avoid

3.3 TRUE-FALSE QUESTIONS

• only two possible responses (yes or no; true or false) for each item

​

WHY ARE TRUE-FALSE USED?

• well suited for testing student recall or comprehension.
• they are easier to construct
• Can be easily written and cover a lot of content

​

LIMITATIONS OF TRUE-FALSE QUESTIONS

• Guessing
• Tendency to include trivial material
• May not suitable for complex materials
• May not discriminate students well

Suggestions for Construction of �True-false Questions (p 14)

• Include only one main idea in each item.
• As in multiple choice questions generally, use negatives sparingly.
• Try using in combination with other material, such as graphs, maps, written material. This combination allows for the testing of more advanced learning
• Avoid lifting statements directly from assigned reading, notes or other course materials so that recall alone will not permit a correct answer.
• Avoid ambiguous or vague statements and terms
• Avoid verbal clues (word, length or detail)

​

3.4 MATCHING QUESTIONS

Example:

Directions: Column A contains statement describing selected Asian cities. For each description find the appropriate city in Column B. Each city in Column B can used only once.

​

Column A Column B

_____ 1. The ancient capital of Thailand A. Ayuthia

_____ 2. The largest city in Sumatera. B. Ho Chih Min City

_____ 3. The capital of Mynmar. C. Karachi

_____ 4. Formerly known as Saigon D. Medan

_____ 5. The former capital of Pakistan E. Yangdon

F. Hanoi

G. Surabaya

Advantages Of Matching �Questions, pg 16

• good at assessing a student's understanding of relationships

o Definitions - terms

o Historical events- dates

o Achievements - people

o Statements- postulates

o Descriptions - principles

• ability to apply knowledge by requiring a test-taker to match the following

o Examples - terms

o Functions - parts

o Classifications - structures

o Applications - postulates

o Problems - principles

​

• a variation of the multiple choice format
• easy to write
• Highly efficient as large amount of knowledge can be sampled in a short amount of time
• One matching question can replace several true-false questions.
• Can allow responses to be used more than once

​

Limitations Of Matching Questions �Pg 16

• limited to material that can be listed to two columns
• Last answer could be a give away through elimination
• Difficult to differentiate between effective and ineffective items
• Often leads to testing of trivial facts or bits of information
• Often criticised for encouraging rote memorisation

Suggestions for Writing Good Matching Questions pg 17�

• Provide clear directions
• Information in each column as homogeneous as possible
• Include more responses
• Correct answers should not be obvious
• No keywords appearing in both a premise
• All of the responses and premises should appear on the same page.

​

Assignment Part C

The purpose of Part C is to assess your ability to build a test and conduct item analysis on the items to determine its appropriateness.

INSTRUCTIONS:

1. Construct 10 multiple-choice questions (MCQ) in any subject area of your choice The test could be for learners in a primary or secondary school, learners in a college or university or participants in a training programme.

2. The test should be an Achievement Test for a particular subject such as Science, Mathematics, Statistics, History, Management, Language and so forth.

3. The questions should be constructed by you (i.e. original) and not taken from available sources. Attach the test.

​

4. Each stem for an item or question should have 4 alternatives (A, B, C & D) with three distractors and one answer.

5. Administer the test to a group of at 15 students or more.

6. Mark the test and analyse the test scores:

Compute the mean & median

Compute the standard deviation

7. Conduct an item analysis of the items. (refer to Chapter 8: Item Analysis):

• Compute the Difficulty index for each item
• Compute the Discrimination index for each item
• Evaluate the Effectiveness of each option or alternative

8. Write a report on your findings.

9. Attach a copy of the test.

​

Exam 2 Question 6

Chapter 4�USING ESSAY TESTS�

CHAPTER 4 OVERVIEW

4.1 What is an essay question?

4.2 Types of essay

4.3 Why are essay questions used?

4.4 Deciding whether to use essay questions or objective questions

4.5 Limitation of essay questions

4.6 Some misconceptions about essays

4.7 Guidelines for constructing essay questions

4.8 Common verbs used in essay questions

4.9 Marking or scoring essay questions

​

4.1 What is an ESSAY QUESTION?

• is “a test item which requires a response composed by the examinee usually in the form of one or more sentences of a nature that no single response or pattern of responses can be listed as correct, and the accuracy and quality of which can be judged subjectively only by one skilled or informed in the subject”.

Criteria by Reiner, Bothell, Sudweeks �and Wood (2002)

• The learner has to compose
• consist of one or more sentences.
• no one single correct response or answer
• must be judged subjectively by a specialist in the subject:

4.2 TYPES OF ESSAYS

Coursework Essay

• Essay Outlines- 500 words – basic, organise materials
• Standard Essay – 1000-2500 words, analyse relationship, give judgment, solve familiar problem, make comparison
• Extended Essay – 2500-5000 words – less familiar materials or problem , research

Examination Essay

- short essay written as part of a formal examination

– one or a few, choice or no choice, long or short duration, closed book or opened book

4.3 Why are ESSAY QUESTIONS used?

• effective way of assessing complex learning outcomes or higher order thinking skills
• students to demonstrate their reasoning
• provide authentic experiences in problem solving and making decisions

​

4.4 DECIDING WHETHER TO USE ESSAY QUESTIONS OR OBJECTIVE QUESTIONS

• Check the learning outcomes
• Which priority -Ability to construct or to select
• Check the verb(s) that best describe the desired ability to be assessed

4.5 What are Some Limitations of Essay Questions?�

• not possible to assess the student's mastery of the complete subject matter domain with just a few questions
• Essay questions have limitations in reliability
• Essay questions require more time for marking student responses.
• does not allow them to give appropriate attention to the complex process of organizing, writing, and reviewing their responses.

4.6 Some Misconceptions About Essay Questions in Examinations

• By their very nature essay questions assess higher-order thinking.
• Essay questions are easy to construct
• The use of essay questions eliminates the problem of guessing
• Essay questions benefit all students by placing emphasis on the importance of written communication skills
• Essay questions encourage students to prepare more thoroughly.

4.7 Some Guidelines for Constructing Essay Questions

ENSURE THAT STUDENTS HAVE A CLEAR IDEA ON WHAT THEY ARE EXPECTED TO DO AFTER THEY HAVE READ THE QUESTION OR PROBLEM PRESENTED

• Clearly Define the Intended Learning Outcome to be Assessed by the Question.
• Avoid Using Essay Questions for Intended Learning Outcomes that are Better Assessed with other Kinds of Assessment
• Clarity about the Task and Scope (see eg. Pg 11)
• Questions that are Fair

​

​

• Specify the approximate time limit and marks allotted for each question
• Use several relatively short essay questions rather than one long one
• Avoid the use of optional questions
• Improving Essay Questions Through Preview and Review, pg 14

​

4.8 Verbs Describing Various Kinds of Mental Tasks (See pg 15-16)�

• Defend
• Critique
• Construct
• Apply
• Describe
• Design
• Explain
• Evaluate
• Generate
• Summarise
• Create
• Compose

4.9 Marking AN ESSAY QUESTION

1) The Analytic Method

• Figure 4.2 Sample of a Marking Scheme Using the Analytical Method, pg 18

​

​

2) Holistic Method (Global or Impressionistic Marking)

• Table 4.1 Sample of a Marking Scheme (Rubric) Using the Holistic Method, pg 20

​

Suggestions For Marking Or Scoring Essays

• Grade the papers anonymously
• Mark the question one by one
• Mark one question for all without interruption.
• Shuffle the papers after each item is scored throughout all the papers
• handle extraneous factors and be consistent in applying the rule.
• Be on the alert for bluffing.
• Write comments on the students' answers

• Be aware of the order in which papers are marked which can have an impact on the grades that are awarded
• when students are directed to take a stand on a controversial issue, the marker must be careful to insure that the evidence and the way it is presented is evaluated, NOT the position taken by the student

Exam May 2018

Question 4

Discuss FIVE (5) specific guidelines that can help you construct good essay questions.

​

[Bincangkan LIMA (5) garis panduan tertentu yang dapat membantu anda menggubal soalan

esei yang baik]

• [TOTAL: 10 marks]

Exam 1 Question 6