Difference between revisions of "Optimizing the practice schedule"

From LearnLab
Jump to: navigation, search
(Optimizing the practice schedule)
(Independent variables)
 
(100 intermediate revisions by 4 users not shown)
Line 1: Line 1:
== Optimizing the practice schedule ==
 
 
 
=== Abstract ===
 
=== Abstract ===
The studies in this cluster concern the design and organization of instructional activities that direct the learner’s attention to critical knowledge components. A general assumption of this cluster’s work is that these knowledge components can be acquired, strengthened, and refined more effectively through the analysis of learning tasks that identify the knowledge components in relation to the learners’ background knowledge. A corollary assumption is that under many circumstances, instruction is more effective when it directly instructs the knowledge components than when it is structured to allow only indirect learning. However, the cluster’s more general concern is that instruction needs to be based on considerations of the unique demands of the material to be learned and the learner’s relevant prior knowledge.
+
This project plan extends dissertation work of Pavlik. In this initial work, a model-based algorithm was described to maximize the rate of learning for simple facts using flashcard like practice by determining the best [[instructional schedule]] for a set of facts. The goal of this project plan is to develop this initial work to allow this tutor with [[optimized scheduling]] to handle more complex information and different types of learning in more natural settings (like LearnLabs). Specifically, this project plan describes extensions to the theory in two main areas.  
  
Our general hypothesis is that the structure of instructional activities, including practice, affects learning. A slightly more specific hypothesis is that structures that require the learner to attend to the valid features of a complex stimulus lead to more robust learning than structures that do not. In some situations, the knowledge components are supported by the learner’s prior knowledge; in other situations, the learner’s prior knowledge provides a hindrance to learning. However, in all situations, the effective structure of learning events requires an analysis of the domain to be learned, one that reveals the unique demands of the to-be-learned material (relative to a learner’s background) and highlights the critical knowledge components. Learning events are then organized that recognize the demands of the task and draw attention to these components.
+
:1. Specification of a theory of [[refinement]]
 +
::a. Generalization practice (multimodal and bidirectional training)
 +
::b.  Discrimination practice (detailed error remediation)
 +
:2.  Specification of a theory of [[co-training]]
 +
::a. Effect of [[declarative]] memory chunk [[schedule of presentation]]  during learning
 +
::b. Effect of [[declarative]] memory chunks on [[procedural]] learning
  
At the micro-level, this hypothesis can be rephrased in terms of the PSLC general hypothesis, which is that robust learning occurs when the learning event space is designed to include appropriate target paths, and when students are encouraged to take those paths. In most of the studies in this cluster, few paths are available to the student and these are well structured to lead to the acquisition and refinement of knowledge components. However, in some cases, the analysis of the task leads to paths that are not obvious.
+
These theoretical directions are intended to enhance the [[FaCT System]] tutor by greatly extending its capabilities.
 +
 
 +
A secondary goal of the project is to link the optimization algorithm used in this project with the larger [[CTAT]] project. In this linkage the optimization algorithm would be integrated onto the current [[CTAT]] system as a curriculum management system that could select or generate problems according to the algorithm, but using [[CTAT]] interfaces. This integration will make it easier for people to use the [[optimized scheduling]] system and therefore increase its impact and usefulness.
  
 
=== Glossary ===
 
=== Glossary ===
Forthcoming.
+
* [[Optimal spacing interval]]
 +
* [[Expanding spacing interval]]
 +
* [[Optimized scheduling]]
  
 
=== Research question ===
 
=== Research question ===
How can analyses of task and learner’s knowledge lead to a structuring of instructional events that lead to robust learning?
+
How can the optimal sequence of [[learning event]]s be computed? The descendants section below links to LearnLab and laboratory research tracks that have employed and invetigated these methods of optimal sequencing.
  
=== Independent variables ===
+
=== Background and significance ===
Alternative structures of instructional events based on alternative analyses of task demands, relevant knowledge components, and learner background. Assessing the learner’s background is essentially part of the learning task analysis.
+
 
 +
Since the early 60's researchers in learning theory have been describing models of practice which attempt to capture the effect of [[practice]] on performance at a later time. These models are applicable to describing many types of learning situations, but are easier to apply where information to be learned can be broken up into small chunks that can be learned independently. For instance, Atkinson (1972) applied a Markov model of learning to schedule [[drill]] of German vocabulary.
 +
 
 +
More recently there has been a renewed emphasis on repeated practice. For instance, the National Council of Teachers of Mathematics new report [http://online.wsj.com/article_email/SB115802278519360136-lMyQjAxMDE2NTE4MjAxMjIyWj.html WSJ article] emphasizes the importance of this type of learning for simple math skills.
 +
 
 +
More information and demonstrations of tutors in this project can be found at [http://optimallearning.org Lab Website]
  
 
=== Dependent variables ===
 
=== Dependent variables ===
Measures of normal and robust learning.
+
 
 +
[[Long-term retention]] -- These measures are usually taken in the tutor after at least one day of retention (much longer intervals occur in some of the most recent studies).
 +
 
 +
[[Transfer]] -- Many of the studies in this project will look at how learning in the tutor transfers to situations where that knowledge can be applied in a different configuration.
 +
 
 +
[[Accelerated future learning]] -- Some studies in this project will investigate the effect of tutor practice on the learning of items that depend upon the tutor practice.
 +
 
 +
=== Independent variables ===
 +
Alternative structures of [[instructional schedule]] for [[practice]] based on the predictions of an ACT-R based cognitive model. Further independent variables include how the material is presented for [[learning events]] and the assumptions of the model used to compute the [[instructional schedule]]. The assumptions of the model include alternative analyses of [[task demands]], the structure of relevant [[knowledge components]], and learner [[individual differences]].
 +
 
 +
Example screen shot of instructional event presentation (Demo versions at [http://optimallearning.org/demos/ Demo Page]):
 +
[[Image:Examplescreen1FaCT.JPG]]
  
 
=== Hypothesis ===
 
=== Hypothesis ===
Robust learning is increased by instructional activities that require the learner to  attend to the relevant knowledge components of a learning task.
+
[[Robust learning]] occurs more quickly when [[practice]] is scheduled efficiently. In this case efficiently means according to a complex model of the [[robust learning]] gain and time cost of possible scheduling decisions. Given a single type of learning event, such schedules tend to have an [[expanding spacing interval]], since as [[practice]] accumulates knowledge components gain [[stability]]. See [[optimal scheduling]] for a discussion of learning principles and other examples.
  
=== Explanation ===
+
=== Findings ===
Attention to features of the task domain as a knowledge component is processed leads to associating those features with the knowledge component.  If the features are valid, then forming or strengthening such associations facilitates retrieval during subsequent assessment or instruction, and thus leads to more robust learning.
+
  
=== Descendents ===
+
This is a summary of the main findings for the various lines of research associated with this project. The following work has utilized the Java based [[FaCT System]] for trial based learning to deliver experiments. This system is described here: [http://optimallearning.org/ website].
  
* Using syntactic priming to increase robust learning (de Jong, Perfetti, DeKeyser)
+
* [[Applying optimal scheduling of practice in the Chinese Learnlab]] (Pavlik, MacWhinney, Sue-mei Wu, Koedinger)
 +
**This section discusses our efforts (a series of classroom studies) to show that the [[optimized scheduling]] provided by the [[FaCT System]] is better at producing robust learning than various [[Ecological control group|Ecological Control Group]]s. Initial results indicate that the system improves student performance for vocabulary quizzes, results in more practice by students and has better participation than control practice conditions.
  
* Basic skills training (MacWhinney)
+
* [[Understanding paired associate transfer effects based on shared stimulus components]] (Pavlik, MacWhinney, Bolster, Koedinger)
 +
**This study shows how a [[knowledge component]] analysis leads to predictions about [[transfer]] that are supported experimentally. After making a model of these effects, the results of this study will be applied in the classroom to improving the [[optimized scheduling]] algorithm. Three effects were found: Unit knowledge component learning - This hypothesis proposes that the stimulus items (sound file, Hanzi character, pinyin, or English) are learned as individual components somewhat independent of the pairings they occur in. Supports the notion of knowledge decomposition. Resonant learning - This hypothesis proposes that people spontaneously recall related knowledge components (spreading activation) when prompted to recall a specific pair. Further, this covert practice results in measurable learning. Stimulus mapping - This is the straightforward notion that learning of the connection between an orthography and a sound is advantaged because there are mapping rules (knowledge components) that allow this conversion.
  
* First language effects on second language grammar acquisition (Mitamura)
+
* [[Understanding encoding inhibition, retrieval inhibition and destructive interference effects of errors during practice]] (Pavlik)
 +
**This study used a complex design to see the effects of errors on learning. If errors should have an effect on learning it will require revisions of the model (i.e. if an error on practice at time t has an effect on practice at time t+1, then the model's accuracy will be increased if this is accounted for.)
  
* [[Optimizing the practice schedule]] (Pavlik)
+
* [[French gender cues]] (Presson-MacWhinney)
 +
**This project is part of Nora Presson's dissertation research and explores how to optimize practice for a skill that generalizes to multiple exemplars using the FaCT system.
 +
 
 +
* [[Providing optimal support for robust learning of syntactic constructions in ESL]] (Levin, Frishkoff, De Jong, Pavlik)
 +
**This project will use the FaCT system to explore a learning paradigm where multiple general factors compete to determine the response (whether to produce the NP PP or NP NP construction).
 +
 
 +
=== Explanation ===
 +
The algorithm for scheduling practice uses a mathematical model of learning to predict when new practice should occur for recall to be optimal later. This model accounts for:
 +
 +
When prior practice occurred
 +
*How many prior [[learning events]] occurred
 +
*[[Temporal spacing]] between prior [[learning events]] was
 +
*Whether prior [[learning events]] occurred as testing or passive study
 +
*Duration of prior [[learning events]]
 +
*An individuals history of success or failure with tests
 +
*What type of practice occurs (phonological, orthographic, English to Foreign or Foreign to English, [[implicit instruction]], [[explicit instruction]]).
 +
 +
Optimized scheduling is mainly controlled by the benefit of wide [[temporal spacing]], which results in better [[long-term retention]] and the benefit of short [[temporal spacing]], which reduce time cost.
  
* Semantic grouping during vocabulary training (Tokowicz)
+
=== Descendants ===
  
* Mental rotations during vocabulary training (Tokowicz)
+
* [[Applying optimal scheduling of practice in the Chinese Learnlab]] (Pavlik, MacWhinney, Sue-mei Wu, Koedinger)
 +
* [[Understanding paired associate transfer effects based on shared stimulus components]] (Pavlik, MacWhinney, Bolster, Koedinger)
 +
* [[Understanding encoding inhibition, retrieval inhibition and destructive interference effects of errors during practice]] (Pavlik)
 +
* [[French gender cues]] (Presson-MacWhinney)
 +
* [[Providing optimal support for robust learning of syntactic constructions in ESL]] (Levin, Frishkoff, De Jong, Pavlik)
  
 
=== Annotated bibliography ===
 
=== Annotated bibliography ===
Forthcoming
 
  
[[Category:Project]]
+
*Atkinson, R. (1972) Optimizing the learning of a second language vocabulary. Journal of Experimental Psychology, 96, 124- 129.
 +
*Pavlik Jr., P. I., Presson, N., Dozzi, G., Wu, S., MacWhinney, B. & Koedinger, K. (2007, accepted). The FaCT (Fact and Concept Training) System: A new tool linking cognitive science with educators. In D. McNamara & G. Trafton (Eds.), Proceedings of the Twenty-Ninth Annual Conference of the Cognitive Science Society. Mahwah, NJ: Lawrence Erlbaum. [http://www.learnlab.org/uploads/mypslc/publications/pavlik_1_31.pdf (Article)]
 +
*Pavlik Jr., P. I. (2006). Transfer effects in Chinese vocabulary learning. In R. Sun (Ed.), Proceedings of the Twenty-Eighth Annual Conference of the Cognitive Science Society (pp. 2579). Mahwah, NJ: Lawrence Erlbaum. [http://www.learnlab.org/uploads/mypslc/publications/pavlik-transfereffects.pdf (Article)]
 +
*Pavlik Jr., P. I. (in press-a). Timing is an order: Modeling order effects in the learning of information. In F. E., Ritter, J. Nerb, E. Lehtinen & T. O'Shea (Eds.), In order to learn: How order effects in machine learning illuminate human learning. New York: Oxford University Press.
 +
*Pavlik Jr., P. I. (in press-b). Understanding and applying the dynamics of test practice and study practice. Instructional Science.
 +
*Pavlik Jr., P. I., & Anderson, J. R. (2005). Practice and Forgetting Effects on Vocabulary Memory: An Activation-Based Model of the Spacing Effect. Cognitive Science, 29, 559-586 [http://optimallearning.org/people/Articles/2005%20Pavlik%20Anderson.pdf (Article)]
 +
*Pavlik Jr., P. I., & Anderson, J. R. (2004,November). Optimizing Paired-Associate Learning. Poster presented at the 45th Annual Meeting of the Psychonomic Society, Minneapolis, MN.

Latest revision as of 17:30, 8 December 2008

Abstract

This project plan extends dissertation work of Pavlik. In this initial work, a model-based algorithm was described to maximize the rate of learning for simple facts using flashcard like practice by determining the best instructional schedule for a set of facts. The goal of this project plan is to develop this initial work to allow this tutor with optimized scheduling to handle more complex information and different types of learning in more natural settings (like LearnLabs). Specifically, this project plan describes extensions to the theory in two main areas.

1. Specification of a theory of refinement
a. Generalization practice (multimodal and bidirectional training)
b. Discrimination practice (detailed error remediation)
2. Specification of a theory of co-training
a. Effect of declarative memory chunk schedule of presentation during learning
b. Effect of declarative memory chunks on procedural learning

These theoretical directions are intended to enhance the FaCT System tutor by greatly extending its capabilities.

A secondary goal of the project is to link the optimization algorithm used in this project with the larger CTAT project. In this linkage the optimization algorithm would be integrated onto the current CTAT system as a curriculum management system that could select or generate problems according to the algorithm, but using CTAT interfaces. This integration will make it easier for people to use the optimized scheduling system and therefore increase its impact and usefulness.

Glossary

Research question

How can the optimal sequence of learning events be computed? The descendants section below links to LearnLab and laboratory research tracks that have employed and invetigated these methods of optimal sequencing.

Background and significance

Since the early 60's researchers in learning theory have been describing models of practice which attempt to capture the effect of practice on performance at a later time. These models are applicable to describing many types of learning situations, but are easier to apply where information to be learned can be broken up into small chunks that can be learned independently. For instance, Atkinson (1972) applied a Markov model of learning to schedule drill of German vocabulary.

More recently there has been a renewed emphasis on repeated practice. For instance, the National Council of Teachers of Mathematics new report WSJ article emphasizes the importance of this type of learning for simple math skills.

More information and demonstrations of tutors in this project can be found at Lab Website

Dependent variables

Long-term retention -- These measures are usually taken in the tutor after at least one day of retention (much longer intervals occur in some of the most recent studies).

Transfer -- Many of the studies in this project will look at how learning in the tutor transfers to situations where that knowledge can be applied in a different configuration.

Accelerated future learning -- Some studies in this project will investigate the effect of tutor practice on the learning of items that depend upon the tutor practice.

Independent variables

Alternative structures of instructional schedule for practice based on the predictions of an ACT-R based cognitive model. Further independent variables include how the material is presented for learning events and the assumptions of the model used to compute the instructional schedule. The assumptions of the model include alternative analyses of task demands, the structure of relevant knowledge components, and learner individual differences.

Example screen shot of instructional event presentation (Demo versions at Demo Page): Examplescreen1FaCT.JPG

Hypothesis

Robust learning occurs more quickly when practice is scheduled efficiently. In this case efficiently means according to a complex model of the robust learning gain and time cost of possible scheduling decisions. Given a single type of learning event, such schedules tend to have an expanding spacing interval, since as practice accumulates knowledge components gain stability. See optimal scheduling for a discussion of learning principles and other examples.

Findings

This is a summary of the main findings for the various lines of research associated with this project. The following work has utilized the Java based FaCT System for trial based learning to deliver experiments. This system is described here: website.

  • Understanding paired associate transfer effects based on shared stimulus components (Pavlik, MacWhinney, Bolster, Koedinger)
    • This study shows how a knowledge component analysis leads to predictions about transfer that are supported experimentally. After making a model of these effects, the results of this study will be applied in the classroom to improving the optimized scheduling algorithm. Three effects were found: Unit knowledge component learning - This hypothesis proposes that the stimulus items (sound file, Hanzi character, pinyin, or English) are learned as individual components somewhat independent of the pairings they occur in. Supports the notion of knowledge decomposition. Resonant learning - This hypothesis proposes that people spontaneously recall related knowledge components (spreading activation) when prompted to recall a specific pair. Further, this covert practice results in measurable learning. Stimulus mapping - This is the straightforward notion that learning of the connection between an orthography and a sound is advantaged because there are mapping rules (knowledge components) that allow this conversion.
  • French gender cues (Presson-MacWhinney)
    • This project is part of Nora Presson's dissertation research and explores how to optimize practice for a skill that generalizes to multiple exemplars using the FaCT system.

Explanation

The algorithm for scheduling practice uses a mathematical model of learning to predict when new practice should occur for recall to be optimal later. This model accounts for:

When prior practice occurred

Optimized scheduling is mainly controlled by the benefit of wide temporal spacing, which results in better long-term retention and the benefit of short temporal spacing, which reduce time cost.

Descendants

Annotated bibliography

  • Atkinson, R. (1972) Optimizing the learning of a second language vocabulary. Journal of Experimental Psychology, 96, 124- 129.
  • Pavlik Jr., P. I., Presson, N., Dozzi, G., Wu, S., MacWhinney, B. & Koedinger, K. (2007, accepted). The FaCT (Fact and Concept Training) System: A new tool linking cognitive science with educators. In D. McNamara & G. Trafton (Eds.), Proceedings of the Twenty-Ninth Annual Conference of the Cognitive Science Society. Mahwah, NJ: Lawrence Erlbaum. (Article)
  • Pavlik Jr., P. I. (2006). Transfer effects in Chinese vocabulary learning. In R. Sun (Ed.), Proceedings of the Twenty-Eighth Annual Conference of the Cognitive Science Society (pp. 2579). Mahwah, NJ: Lawrence Erlbaum. (Article)
  • Pavlik Jr., P. I. (in press-a). Timing is an order: Modeling order effects in the learning of information. In F. E., Ritter, J. Nerb, E. Lehtinen & T. O'Shea (Eds.), In order to learn: How order effects in machine learning illuminate human learning. New York: Oxford University Press.
  • Pavlik Jr., P. I. (in press-b). Understanding and applying the dynamics of test practice and study practice. Instructional Science.
  • Pavlik Jr., P. I., & Anderson, J. R. (2005). Practice and Forgetting Effects on Vocabulary Memory: An Activation-Based Model of the Spacing Effect. Cognitive Science, 29, 559-586 (Article)
  • Pavlik Jr., P. I., & Anderson, J. R. (2004,November). Optimizing Paired-Associate Learning. Poster presented at the 45th Annual Meeting of the Psychonomic Society, Minneapolis, MN.