Difference between revisions of "Providing optimal support for robust learning of syntactic constructions in ESL"

From LearnLab
Jump to: navigation, search
Line 2: Line 2:
 
|-  
 
|-  
 
! PIs
 
! PIs
| Pavlik, Koedinger
+
| Levin, Frishkoff, De Jong, Pavlik
 
|-
 
|-
 
! Faculty
 
! Faculty
| Koedinger
+
| Levin
 
|-
 
|-
 
! Postdocs
 
! Postdocs
| Pavlik
+
| Frishkoff, De Jong, Pavlik
 
|-
 
|-
 
! Others with > 160 hours
 
! Others with > 160 hours
Line 14: Line 14:
 
|-
 
|-
 
! Learnlab
 
! Learnlab
| None (stimuli from Chinese learnlab)
+
| ESL
 
|-
 
|-
 
! Number of participants
 
! Number of participants
| 10
+
| 30
 
|-
 
|-
 
! Total Participant Hours
 
! Total Participant Hours
| 15
+
| 40
 
|-
 
|-
 
! Datashop?
 
! Datashop?

Revision as of 13:24, 26 March 2007

PIs Levin, Frishkoff, De Jong, Pavlik
Faculty Levin
Postdocs Frishkoff, De Jong, Pavlik
Others with > 160 hours n/a
Learnlab ESL
Number of participants 30
Total Participant Hours 40
Datashop? Expected date 7/15

Abstract

The goal of this project is to examine how second-language learners acquire context-appropriate use of syntactic constructions*. In some cases, learning when to use a syntactic construction is straightforward. In other cases, use of a syntactic construction is seldom mastered, even by advanced students. The main challenge, in such cases, is to learn which contextual cues predict the occurrence of a particular form (for example, I am here for two years vs. I have been here for two years). That is, students must learn the “meanings” or functions of grammatical constructions, in order to use them in appropriate contexts.

This project focuses on acquisition of the dative alternation* ( give someone a book vs. give a book to someone) by ESL students. Recently, Bresnan and associates (Bresnan & Hay, 2006; Bresnan & Nikitina, 2003; Bresnan, Cueni, Nikitina, & Baayen, 2005) identified 14 features that combine to form a linear regression model of the dative alternation for native English speakers. Using this model as a blueprint, we propose to develop a student-centered, cognitive–linguistic model that will determine optimal presentation of example texts to support acquisition of the dative alternation in ESL. We propose to use a constraint-based model ( the Bresnan model*), combined with a model of learning (the Pavlik model), to select training examples that we will present in a way that is hypothesized to result in optimal refinement, transfer, and retention, of proficiency with the dative alternation. This approach will comprise the following steps:

(A) The native-speaker model will constitute the target for ESL acquisition of the dative alternation.
(B) We will calculate distances between student performance and the native speaker model.
(C) We will select training items that maximize learning, i.e., that reduce the distance between the ESL student models and the native-speaker model.

Background

The stimuli for the experiment are based on Switchboard, which is a corpus of telephone conversations between native speakers of English. Switchboard was recorded by the Linguistic Data Consortium in the early 1990's for the purpose of speech research. The Bresnan team selected 2600 sentences from the Switchboard corpus that contain agent, theme, and recipient arguments. For each sentence, they annotated the values of fourteen features (is the action concrete or abstract, is the theme a pronoun, is the recipient a pronoun, etc.) A linear regression model trained on the features has 92% accuracy in predicting whether the native speaker chose the double object or prepositional option in each sentence. We want to know whether we can get non-native speakers to make the same choice that the native speaker made. However, we can't show the original dialogue to the non-native speaker because it contains disfluencies, culture-specific information, and advanced vocabulary. We are therefore adapting the Switchboard sentences for ESL students, but we must be sure not to change the values of any of the 14 features. Following is an example of a Switchboard segment and the adapted version for our experiment. The subjects make a forced choice between the double object and prepositional options.

Original

Well, the thing I think that annoys me the most is, I have, I have young children, a baby in the house and, and inevitably as soon as they're asleep, someone calls on the phone trying to sell me something.


Adapted

I have young children at home. As soon as they are asleep at night,

  • someone calls on the phone trying to sell me something.
  • someone calls on the phone trying to sell something to me.

Significance

Novelty of the project: Almost all previous studies of the dative alternation in ESL have focused on the form of the dative alternation, rather than whether it is used appropriately in context. Our study will be the first to —

  • Teach the use (meaning or function) of the dative alternation;
  • Provide an implemented model of how the dative alternation is learned; and
  • Test whether models of native speaker use can serve as the basis for effective instruction augment the native speaker models with a model of how the dative alternation is learned

In the course of this study, the Pavlik and Anderson model will be applied to a new area: the interaction of multiple features in second language acquisition We will build a framework (theory, model, and tools) that can be re-used in subsequent studies of difficult-to-use constructions.

Relevance for Second-Language Learning Research: While the dative alternation itself represents a relatively small part of the English grammar, it is one instance of a broader (and relatively productive) class of constructions known as resultatives, which include non-prototypical uses of verbs like “sneeze” to express change-of-state — e.g., “He sneezed the letter across the table” (cf. Goldberg, 1995). Resultative constructions have received considerable attention among linguists (see Goldberg & Jackendoff, 2004 for a recent review), particularly in recent research on grammar acquisition (e.g., Goldberg & Casenhiser, 2004, 2005). Still more broadly, the dative alternation involves transitivity relations (also referred to as voice, or diathesis), which are represented cross-linguistically using several important syntactic devices (Givón, 1984). Voice markers are often associated with complex meanings and functions (cf. Frishkoff, 1997), and acquisition of grammatical voice in some languages occupies a large part of the curriculum. Therefore, although our selection of the dative may suggest a narrow focus, our studies should have wide-spread implications for acquisition of transitivity and diathesis relations in many (perhaps all) languages.

We view this project as seeking to establish “proof of concept,” which will justify work in other domains of grammar learning using this same approach. Our goal is to tune our procedures to our findings, particularly with respect to the degree to which training items affect student behavior, and the proportional effects of student performance vs. experience. After we tune our procedures for the dative alternation, we will extend our methods to address errors in the use of articles and tense-aspect constructions.

Contribution to the theory of robust learning This project relates to refinement and fluency in the learning process. Our hypothesis is that we can strengthen and refine feature representations to approximate native speaker competence. The instructional goals of this project focus on long term retention and transfer. Following the Pavlik and Anderson model, training items are selected and spaced to optimize long-term gain. Transfer is expected from trained to untrained items, from comprehension to production, and from prototypical feature instances to less prototypical feature instances.

Glossary

Alternation Pair
A pair of sentences with the same verb, agent, recipient, and theme, where one sentence of the pair is a double object construction and the other is a prepositional dative construction:
  • Gretchen sent her a form.
  • Gretchen sent a form to her.
Argument
A participant in the action described by the verb. The sentence I gave a book to him has three arguments, I, book, and him.
Bresnan Model
A logistic regression model including fourteen features that predicts whether native English speakers will use the double object or prepositional variant of a dative sentence.
Dative Alternation
There are two ways to express sentences that contain agent, recipient, and theme arguments. In the double object construction, the recipient comes first, followed by the theme, with no prepositions. In the prepositional dative construction, the theme comes first, followed by the preposition to and then the recipient.
  • Gretchen sent her a form. (double object)
  • Gretchen sent a form to her. (prepositional dative)
Double object construction or NP NP construction
In the double object construction, the recipient argument comes first, followed by the theme argument with no prepositions.
  • Gretchen sent her a form.
  • This music gives me a headache.
  • The light gave her features a healthy glow.
NP NP Construction
See Double object construction
NP PP Construction
See Prepositional dative construction
Prepositional dative construction or NP PP construction
In the prepositional dative construction, the theme comes first, followed by the preposition to and then the recipient.
  • Gretchen sent a form to her.
  • The teacher told a story to the children.

The following sentence is also an instance of the prepositional dative construction, but it has undergone an alternation called Heavy NP Shift. The recipient retains its preposition and the theme moves to the end of the sentence because it is long (heavy).

  • Gretchen sent to her a long form containing many confusing questions.
Recipient
The argument that receives something in an abstract or concrete way. The recipient arguments are in bold face in these examples:
  • Gretchen sent her a form.
  • Gretchen sent a form to her.
  • The teacher told the children a story.
  • The teacher told a story to the children.
  • This music gives me a headache.
  • The light imparted a healthy glow to her features.
Syntactic Construction
A recognizable configuration of words and morphemes (prefixes and suffixes). Constructions may contain fixed expressions ( What a ADJ NOUN!, What a nice dress!). They may also be normal parts of the syntax of the language such as using a noun phrase before a verb phrase to make a predication ( The girl ran). Constructions have a form and a meaning or use. Many second language textbooks are organized around constructions and how they are used. For example, ESL textbooks may have lessons on when to use the present perfect ( have Verb-ed, I have eaten).
Theme
The argument that is moved, given, or communicated to the recipient in an abstract or concrete way.
  • Gretchen sent her a form.
  • Gretchen sent a form to her.
  • The teacher told the children a story.
  • The teacher told a story to the children.
  • This music gives me a headache.
  • The light imparted a healthy glow to her features.

Research Question(s)

Our central research question concerns the learning of complex form-meaning mappings, particularly those as complex as the dative shift, which involves fourteen meaning-related features (knowledge components) with different weights (cue strengths). We want to know whether a model of native speaker behavior can be used as a target for non-native speaker behavior and what instructional interventions (learning events) will bring non-native speakers closer to that target.

Hypotheses

We propose the following hypotheses:

(A) An algorithm that selects training items on the basis of student performance, as well as on the basis of training history, will lead to better performance than one that selects training items based on the history of training alone.
(B) Selection of examples based on student performance will be superior in:
  • transfer from trained to untrained items
  • transfer from comprehension to production
  • transfer from prototypical to less prototypical examples, provided they are presented at the right times, with the right frequencies.

Study 1 will calibrate the learning model. The effects of learning will be measured in terms of feature weights that characterize the student's responses. We will measure the size of the effect that is caused by exposure to specific types of examples. Study 1 also measures medium term forgetting between blocks of stimuli.

Study 2 will test whether trial selection by the learning model leads to native-like behavior. The learning model uses information about which trials have been presented as well as the students' performance. In the control condition, trials are selected based on the history of practice only, not on student performance (i.e., the performance sensitivity in the model will be turned off). In other words, in the control condition, the trial-selection is model-based, but not personalized, as in a classroom where all students see examples in the same order. It is expected that both conditions will result in learning, but that there will be higher gains in the experimental condition than in the control condition.

A follow up session will test two types of transfer: 1) from comprehension to production, and 2) from prototypical to non-prototypical features. The first type of transfer will be tested by including a posttest for production, in which students put sentence constituents into the preferred word order. The second type of transfer will be tested by including a number of test items with non-prototypical features. Some features are binary, such as pronominality ( the book is non-pronominal, whereas it is pronominal), while other features can be prototypical or non-prototypical. For example, in a context where only a house and a man are mentioned, the house is highly accessible in subsequent context, whereas, for instance, the clouds is not. However, in the same context, the kitchen and his wife have intermediate accessibility because they are implied by the mentioning of a house and a man.


Experiment Methods

  • Stimulus development
  • Dative Model: PCA variable reduction
  • Experiment Design & Protocol
  • Study Participants

Independent variables

For Study 1, the independent variables are factors, contrast, session, half of session, and half of block, as explained below.

  • Factors: Using principal components analysis (see Methods for details), the fourteen features of the Bresnan model were reduced to four factors.
    • Factor 1: Definiteness, pronominality, and discourse accessibility (givenness) of the theme.
    • Factor 2: Definiteness, pronominality, and discourse accessibility (givenness) of the recipient.
    • Factor 3: Ratio of lengths of the theme and recipient noun phrases.
    • Factor 4: Abstractness vs. concreteness of the action and of the theme argument.

Each factor has two values, which we can call plus and minus: e.g., action and the theme are abstract ( pay attention to someone) or concrete ( give something to someone).

  • Contrast: If the Bresnan model assigns a score close to zero, both the double object and prepositional variants of the sentence are generally acceptable ( send American Express a check vs. send a check to American Express). If the Bresnan model assigns a score farther from zero, one of the two options will be highly preferable ( give it to anyone who comes in vs. give anyone who comes in it).
  • Session, half of session, and half of block: The first experiment consists of two sessions, each session having eight blocks, and each block having two halves.
    • First half of first session: One block for each feature, with the plus value for the factor in one half of the block and the minus value of the factor in the other half.
    • Second half of first session: An additional block for each feature (medium term forgetting).
    • Second session: One more block for each feature (longer term forgetting.)

For Study 2, an additional independent variable is whether training examples are chosen based on student performance (the student's feature weights) or based on history only (which examples have been seen).

Dependent variables

  • Accuracy on a forced-choice (NP-NP vs. NP-PP) task
    • Learning (improvement in accuracy from early to later trials)
    • Long-term retention (improvement in accuracy from the beginning of Session 2, cf. with beginning of Session 1.
    • Transfer: comprehension to production; prototypical to nonprototypical;

Results

  • Pilot study


  • Experiment 1


  • Experiment 2

Explanation & Discussion

This experiment seeks to increase fluency by adjusting cue strengths (weights of the fourteen features of the Bresnan model). It also seeks to achieve long term retention. The Pavlik-Anderson model predicts the number and spacing of training items needed to change cue strength with long term retention.

Annotated Bibliography

Bresnan, J. & Hay, J. (2006). Gradient grammar: An effect of animacy on the syntax of give in varieties of English. [draft downloaded from http://www-lfg.stanford.edu/bresnan/download.html]

Bresnan, J. & Nikitina, T. (2003). On the gradience of the dative alternation. [draft downloaded from http://www-lfg.stanford.edu/bresnan/download.html]

Bresnan, J., Cueni, A., Nikitina, T., & Baayen, R. H. (2005). Predicting the dative alternation. Paper presented at the KNAW Academy Colloquium: Cognitive Foundations of Interpretation, Amsterdam.

DeKeyser, R. M. (1994). How implicit can adult second language learning be? AILA Review, 11, 83–96.

Goldberg, A. E. (1995). A construction grammar approach to argument structure. Chicago: University of Chicago.

Goldberg, A. E. & Jackendoff, R. (2004, to appear). The English resultative as a family of constructions. Language.

Inagaki, S. (1997). Japanese and Chinese learner's acquisition of the narrow-range rules for the dative alternation in English. Language Learning, 47, 637-669.

Marefat, H. (2005). The impact of information structure as a discourse factor on the acquisition of the dative alternation by L2 learners. Studia Linguistica, 59(1), 66-82.

Morris, C. D., Bransford, J. D., & Franks, J. J. (1977). Levels of processing versus transfer appropriate processing. Journal of Verbal Learning and Verbal Behavior, 16, 519-533.

Pavlik Jr., P.I. & Anderson, J.R. (2005) Practice and Forgetting Effects on Vocabulary Memory: An Activation-Based Model of the Spacing Effect. Cognitive Science, 29, 559-586.