Difference between revisions of "REAP Study on Question Types (Summer 2006)"

From LearnLab
Jump to: navigation, search
(REAP Study on Question Types)
(Abstract)
 
(2 intermediate revisions by the same user not shown)
Line 24: Line 24:
 
=== Abstract ===
 
=== Abstract ===
  
This study examined the differences between four types of assessment question types for measuring knowledge of individual vocabulary items.  The four types of questions were cloze, synonym, sentence production, and self-assessments.  Twenty words were tested using all question types on pre-, post-, and retention-tests.  Ten of the words (the same set for all students) were practiced during an eight week period between pre- and post-tests.
+
This study, described in further detail by Heilman and Eskenazi (2008), examined the differences between four types of assessment question types for measuring knowledge of individual vocabulary items.  The four types of questions were cloze, synonym, sentence production, and self-assessments.  Twenty words were tested using all question types on pre-, post-, and retention-tests.  Ten of the words (the same set for all students) were practiced during an eight week period between pre- and post-tests.
  
 
=== Glossary ===
 
=== Glossary ===
Line 46: Line 46:
 
=== Hypotheses ===
 
=== Hypotheses ===
  
=== Explanation ===
+
=== Findings ===
  
 
It was found that self-assessments are accurate when the learner claims that a word is NOT known.   
 
It was found that self-assessments are accurate when the learner claims that a word is NOT known.   
Line 60: Line 60:
  
 
=== Annotated bibliography ===
 
=== Annotated bibliography ===
 +
 +
Heilman, M., Eskenazi, M. (2008). Self-Assessment in Vocabulary Tutoring. Ninth International Conference on Intelligent Tutoring Systems. (poster)

Latest revision as of 12:59, 18 April 2008

REAP Study on Question Types

Logistical Information

Contributors Maxine Eskenazi, Alan Juffs, Michael Heilman, Lois Wilson
Study Start Date May, 2006
Study End Date July, 2006
Learnlab Courses English Language Institute Reading 4 (ESL LearnLab)
Number of Students 42
Total Participant Hours (est.) 195
Data in Datashop no


Abstract

This study, described in further detail by Heilman and Eskenazi (2008), examined the differences between four types of assessment question types for measuring knowledge of individual vocabulary items. The four types of questions were cloze, synonym, sentence production, and self-assessments. Twenty words were tested using all question types on pre-, post-, and retention-tests. Ten of the words (the same set for all students) were practiced during an eight week period between pre- and post-tests.

Glossary

Self-assessment: The act of a learner informing the tutor about the degree to which a particular knowledge component is known. In this study, the REAP tutor asked students for each word X, "Do you know the word X? Yes/No". A finer grained self-assessment question with multiple levels would probably be preferable.

Cloze Question: A fill-in-the blank question. A target word is replaced with a blank in a sentence containing that word. The student must select the target word from a list of distractors.

Research question

What relationships exist between the four types of vocabulary assessment types?

Dependent variables

Assessment performance on each question type on pre-, post-, and retention-tests.

Independent variables

Question type, Whether a word was practiced or unpracticed.

Hypotheses

Findings

It was found that self-assessments are accurate when the learner claims that a word is NOT known. Self-assessments are not accurate, however, when the learner claims that a word IS known. The accuracy of self-assessments was measured by performance on cloze exercises given after a pre-test but prior to practicing the word for the first time. Learners claiming that a word was unknown INCORRECTLY answered a subsequent exercise on that word over 90% of the time (5% chance of guessing). Learners claiming that a word was known, however, CORRECTLY answered a subsequent exercise about 30-40% of the time. Learners appear overconfident in terms of their vocabulary, and their self-assessments should only be used to determine a list of unknown words or when asking if further practice on a word is necessary.

Further analysis of the relationships between question types is pending.

Notes

In this study, students were given practice exercises (cloze and sentence production) before readings rather than after readings as in previous studies. This is actually not preferable according to teachers and students.

Descendents

Annotated bibliography

Heilman, M., Eskenazi, M. (2008). Self-Assessment in Vocabulary Tutoring. Ninth International Conference on Intelligent Tutoring Systems. (poster)