Difference between revisions of "REAP Study on Focusing of Attention (Fall 2006)"

From LearnLab
Jump to: navigation, search
(Hypotheses)
 
(21 intermediate revisions by the same user not shown)
Line 1: Line 1:
== REAP Study on Focusing of Attention ==
+
== REAP Pilot Study on Focusing of Attention ==
 
   
 
   
 +
 
=== Abstract ===
 
=== Abstract ===
  
There are competing goals in providing a text for practice of target vocabulary itemsFirst, there is a goal to maximize the density of practice opportunities in a given textThat is, texts should be preferred that contain many target vocabulary words so that the learner can encounter as many words as possible in the time availableSecond, there is a goal to provide texts that can be comprehended by the learner in order to prevent frustration and allow deep processing of the meaning of the context around the target words.  If too many unfamiliar words appear in a text, target words or not, then the reader will not be able to comprehend or recall the meaning of the textThird, there is a goal not to overload the student with too many new words to remember, even if a text can be recalled.
+
This study examines the effect of highlighting target vocabulary words in practice readings for vocabulary practice.  Previous research has examined the highlighting of words to facilitate incidental acquisition (incidental because the main task was reading comprehension)Knight (1994) showed positive results for both incidental vocabulary acquisition and reading comprehension when words were marked to [[focusing|focus]] student attention on themStudents were thus encouraged to access definitions for those words.  However, time on task was not controlled, and students in that study for whom vocabulary was marked spent much longer because they were looking up more definitionsOther studies of highlighting vocabulary report weaker effects of marking vocabulary words to focus attention.
  
Therefore, the tutor should seek a proper balance between the goals of providing sufficient quantity and difficulty of practice.  This study was aimed at locating this balance.   
+
The task of students using the REAP tutor is slightly different in that their primary goal is to learn vocabulary.  Authentic texts are made available as well so that students can coordinate [[implicit instruction]] information from context with [[explicit instruction]] information from dictionary definitions.  In previous studies with REAP, target vocabulary words have been highlighted.  As for all words in this study, the highlighted words could be clicked on to bring up a dictionary definition.  Many students in these studies  do not read the entire text given to them, but instead focus entirely on the highlighted target vocabulary words.  Thus, these students may be ignoring valuable [[implicit instruction]] about those words available from context.  Therefore, focusing of attention on vocabulary words may in fact hurt the learning of those words when the student's task is to learn vocabulary.
 +
 
 +
A pilot study showed evidence of shallow learning of highlighted vocabulary words.  Students with highlighting accessed more definitions for target vocabulary words and performed better on immediate post-reading vocabulary practice exercisesHowever, [[normal post-test]] measures showed no differences in learning.  Therefore, students in the highlighting condition seemed particular prone to forgetting. 
 +
 
 +
This study examines the issue of highlighting with a larger sample size and longer period of instructionThe goal of the study is to determine whether or not highlighting of target vocabulary words promotes [[robust learning]] of those words.  The effects of highlighting on learner behavior will also be measured (dictionary accesses for target and non-target words, time spent per reading, etc.).
  
 
=== Glossary ===
 
=== Glossary ===
  
''Vocabulary Stretch:'' of a text, the percentage of unfamiliar or unknown vocabulary words that appear.
 
  
 
=== Research question ===
 
=== Research question ===
  
What percentage of potentially unknown target vocabulary words is optimal in a text given for practicing target vocabulary words?
+
Does focusing attention on target words by highlighting improve vocabulary learning in a reading task aimed at the intentional vocabulary acquisition?
  
 
=== Dependent variables ===
 
=== Dependent variables ===
  
Post- and retention test cloze question performance for words answered incorrectly on the pre-test.
+
[[Normal post-test]] : performance on cloze questions for words identified as unknown on a pre-test  
  
Post- and retention test sentence production performance for words answered incorrectly on the pre-test.
+
[[Transfer]] test measures : Sentence production, use of target words in writing for other classes.
 
+
Number of words practiced.
+
  
 
=== Independent variables ===
 
=== Independent variables ===
  
Preferred number of target vocabulary words in a given text (2 or 4 words per text).
+
Highlighting of target vocabulary words in practice readings.
  
 
=== Hypotheses ===
 
=== Hypotheses ===
  
=== Explanation ===
+
Attention focusing through the highlighting of target vocabulary words will result in large improvements on post-reading practice exercises but only minimal gains in [[normal post-test]] scores.    Highlighting will also reduce students' likelihood of deeply processing [[implicit instruction]] contextual information, as measured by reading-check questions and time spent per reading.  Long-term retention and transfer measures will show negative effects for highlighting vocabulary.
  
The study found no significant differences between the two conditions (2 or 4 words/text).  This null result may be due to statistical power and mistakes in the study design.  The tutor had difficulty providing high quality readings with four vocabulary words.  Thus, the tutor often had to give texts with 3 or even 2 words to students in the 4-word condition.  Additionally, the optimal number of words per text (the density of practice) may be higher than 4 words.  There may be a threshold at which the density of practice becomes too high.  This threshold may actually be at 5 or even 10 words given the average length of the texts used by the REAP tutor.  Some research suggests that up to 5% of the words in a text can be unknown before comprehension is impeded (Laufer, 1992).  Although the task for that study was different (reading comprehension rather than vocabulary acquisition), it suggests that the optimal level of stretch may be such that as many as 20 words may be unknown in a text with 400 word types (which is approximately the mean number of word types in REAP texts).  Other research, however, suggests higher thresholds up to 98% (reference missing).
+
=== Findings ===
  
Although the results for stretch from this study were inconclusive, a great deal of interest data was gatheredSome interesting results have been found, and further analysis is possible (pending).  
+
These preliminary findings are from a PILOT study with a small sample over a short period of timeThe current study is with a larger sample and over a longer period of time.
  
A step-wise linear regression showed that the most significant predictor of acquisition of new vocabulary was the size of the student's vocabulary prior to instruction.  This finding corroborates findings of Stanovich (1986) and James (1996) who refer to the phenomenon as the "Matthew effect" (after a Bible passage), whereby the rich get richer and the poor get poorer.
+
Students were randomly assigned to control and treatment groups. In the control group, as defined here, the target vocabulary words were not highlighted (or linked to definitions) in the readings. In the treatment group, target words were highlighted and linked to dictionary definitions of the words.
  
It was also found that students frequently access dictionary definitions for non-target words that are well below their expected reading level (e.g., "hitting").  This may lead to revisions of curricula for ESL vocabulary learning.
+
There were 27 students who participated in the study. Only 18 came to class on the day of the post-test. Unfortunately, due to absences, there was a significant difference in the time on task between the control and experimental groups (Control Group: N=10, M=163.95 minutes SD=98.7 min; Treatment Group:N=8, M=82.9 min, SD=46.2 min; p=0.49).  
  
=== Descendents ===
 
  
=== Notes ===
+
The number of target words looked up in an online dictionary (either by clicking or by typing them into an HTML form element) was statistically significantly different between groups (p<0.001). Students for whom the target words were highlighted looked up 2.26 words per reading, while those students without highlighting only looked up 0.42 target words per reading. In contrast, the mean number of non-target words looked up (none of which were highlighted) was essentially equal (3.75 for control, 3.69 for treatment).
 +
 
 +
Students with highlighting of target words also performed significantly (p=0.001) better on post-reading vocabulary exercises (M=95% correct) than did students without highlighting (M=56%). However, students in both groups performed approximately equally well on the post-test (M=52.9% correct for control, 59.1% for treatment with highlighting), and the difference between groups was non-significant (p=0.22).
 +
 
 +
=== Explanation ===
  
Target vocabulary lists were determined by a lengthy pre-test using multiple choice cloze questions.  Each question takes 30-60 seconds on average, depending on the student.  In later studies, self-assessment pre-tests were preferred due to time constraints and initial impressions of the students.  Lengthy pre-tests for determining target word lists leave a poor initial impression of the tutor.
+
=== Descendants ===
  
 
=== Annotated bibliography ===
 
=== Annotated bibliography ===
  
Laufer, B. (1992). How much lexis is necessary for reading comprehension? In Vocabulary and Applied Linguistics, Pierre J. L. Arnaud and Henri Béjoint (eds.), 126–132. London: Macmillan.
+
De Ridder, I. (2002). Visible or Invisible Links: Does the Highlighting of Hyperlinks Affect Incidental Vocabulary Learning, Text Comprehension, and the Reading Process?  Language, Learning & Technology, Vol. 6, 2002

Latest revision as of 12:58, 29 March 2007

REAP Pilot Study on Focusing of Attention

Abstract

This study examines the effect of highlighting target vocabulary words in practice readings for vocabulary practice. Previous research has examined the highlighting of words to facilitate incidental acquisition (incidental because the main task was reading comprehension). Knight (1994) showed positive results for both incidental vocabulary acquisition and reading comprehension when words were marked to focus student attention on them. Students were thus encouraged to access definitions for those words. However, time on task was not controlled, and students in that study for whom vocabulary was marked spent much longer because they were looking up more definitions. Other studies of highlighting vocabulary report weaker effects of marking vocabulary words to focus attention.

The task of students using the REAP tutor is slightly different in that their primary goal is to learn vocabulary. Authentic texts are made available as well so that students can coordinate implicit instruction information from context with explicit instruction information from dictionary definitions. In previous studies with REAP, target vocabulary words have been highlighted. As for all words in this study, the highlighted words could be clicked on to bring up a dictionary definition. Many students in these studies do not read the entire text given to them, but instead focus entirely on the highlighted target vocabulary words. Thus, these students may be ignoring valuable implicit instruction about those words available from context. Therefore, focusing of attention on vocabulary words may in fact hurt the learning of those words when the student's task is to learn vocabulary.

A pilot study showed evidence of shallow learning of highlighted vocabulary words. Students with highlighting accessed more definitions for target vocabulary words and performed better on immediate post-reading vocabulary practice exercises. However, normal post-test measures showed no differences in learning. Therefore, students in the highlighting condition seemed particular prone to forgetting.

This study examines the issue of highlighting with a larger sample size and longer period of instruction. The goal of the study is to determine whether or not highlighting of target vocabulary words promotes robust learning of those words. The effects of highlighting on learner behavior will also be measured (dictionary accesses for target and non-target words, time spent per reading, etc.).

Glossary

Research question

Does focusing attention on target words by highlighting improve vocabulary learning in a reading task aimed at the intentional vocabulary acquisition?

Dependent variables

Normal post-test : performance on cloze questions for words identified as unknown on a pre-test

Transfer test measures : Sentence production, use of target words in writing for other classes.

Independent variables

Highlighting of target vocabulary words in practice readings.

Hypotheses

Attention focusing through the highlighting of target vocabulary words will result in large improvements on post-reading practice exercises but only minimal gains in normal post-test scores. Highlighting will also reduce students' likelihood of deeply processing implicit instruction contextual information, as measured by reading-check questions and time spent per reading. Long-term retention and transfer measures will show negative effects for highlighting vocabulary.

Findings

These preliminary findings are from a PILOT study with a small sample over a short period of time. The current study is with a larger sample and over a longer period of time.

Students were randomly assigned to control and treatment groups. In the control group, as defined here, the target vocabulary words were not highlighted (or linked to definitions) in the readings. In the treatment group, target words were highlighted and linked to dictionary definitions of the words.

There were 27 students who participated in the study. Only 18 came to class on the day of the post-test. Unfortunately, due to absences, there was a significant difference in the time on task between the control and experimental groups (Control Group: N=10, M=163.95 minutes SD=98.7 min; Treatment Group:N=8, M=82.9 min, SD=46.2 min; p=0.49).


The number of target words looked up in an online dictionary (either by clicking or by typing them into an HTML form element) was statistically significantly different between groups (p<0.001). Students for whom the target words were highlighted looked up 2.26 words per reading, while those students without highlighting only looked up 0.42 target words per reading. In contrast, the mean number of non-target words looked up (none of which were highlighted) was essentially equal (3.75 for control, 3.69 for treatment).

Students with highlighting of target words also performed significantly (p=0.001) better on post-reading vocabulary exercises (M=95% correct) than did students without highlighting (M=56%). However, students in both groups performed approximately equally well on the post-test (M=52.9% correct for control, 59.1% for treatment with highlighting), and the difference between groups was non-significant (p=0.22).

Explanation

Descendants

Annotated bibliography

De Ridder, I. (2002). Visible or Invisible Links: Does the Highlighting of Hyperlinks Affect Incidental Vocabulary Learning, Text Comprehension, and the Reading Process? Language, Learning & Technology, Vol. 6, 2002