# Difference between revisions of "Composition Effect Kao Roll - old, please keep"

Line 4: | Line 4: | ||

=== Abstract === | === Abstract === | ||

− | + | Composite problems, i.e., problems that require the application of more than one skill, are shown to be harder than a collection of single-step problems requiring the application of the same set of skills. | |

+ | A common explanation is that the composition itself imposes another difficulty level. However, an alternative explanation suggests that the composition makes the application of the individual skills harder. According to that explanation, poor feature validity and shallow domain rules make it harder on students to apply the individual skills correctly in the cluttered environment of composite problems, regardless the need to apply additional skills. | ||

+ | Our study investigates these issues in two ways: (1) Having a DFA which evaluates performance on composite problems and single-step problems using the same data, and (2) By evaluating the effect of instruction targeting a common misconception in single-step problems on composite problems. | ||

=== Glossary === | === Glossary === | ||

+ | - Composite problems: Problems which require the application of several skills, such as solving 3x+6=0 for x. | ||

+ | |||

+ | - Single-step problems: Problems which require the application of a single skill, such as y+6=0 or 3x=-6 | ||

+ | |||

+ | - DFA (Difficulty Factor Analysis): A test that includes pairs of items varying along one dimension only. It allows to evaluate the difficulty level of the single dimensions along which the problems differ. | ||

+ | |||

+ | - The Composition Effect: The effect according to which composite problems are harder than a set of single-step problems using the same skills. | ||

=== Research question === | === Research question === | ||

+ | What is the main source of difficulty in composite problems? | ||

=== Background and Significance === | === Background and Significance === | ||

+ | Significance: This study can shed some light on the source of difficulty on composite problems, and thus can inform the design of relevant instruction and remediation. | ||

=== Independent Variables === | === Independent Variables === | ||

+ | An instruction in the form of solved-example, targeting a common misconception - identifying base and hight in a cluttered environment. | ||

=== Dependent variables === | === Dependent variables === | ||

− | + | Three tests are used in the study: | |

− | - | + | - Pre-test: given before all instruciton |

− | - | + | - Mid-test: given after students learned about single-step problems and before composite problems |

− | + | - Post-test: after students have learned and practice all material. | |

− | + | ||

− | + | ||

− | - | + | |

− | + | ||

− | + | ||

− | + | The tests include the following items. Some of which are transfer items, evaluating robust learning, since they require and adaptive application of the knowledge learned and practiced in class. | |

− | + | ||

− | + | ||

− | + | ||

+ | - Simple diagram: no distractors, canonical orientation | ||

+ | - Simple diagram: distractors, canonical orientation | ||

+ | - Simple diagram: no distractors, tilted orientation | ||

+ | - Simple diagram: distractors, tilted orientation | ||

+ | - Given complex diagram, ask for skill A | ||

+ | - Given complex diagram, ask for skill B | ||

+ | - Given steps A and B, ask for skills C (which requires A and B) | ||

+ | - Given complex diagram, ask for C (which requires A and B) | ||

=== Hypothesis === | === Hypothesis === | ||

+ | The difficulty level in composite problems originates in poor feature validity of the single skills. | ||

=== Findings === | === Findings === | ||

Line 53: | Line 66: | ||

=== Annotated bibliography === | === Annotated bibliography === | ||

− | + | Bransford (2000). How people learn: brain, mind, experience, and school National Academy Press. | |

− | + | Heffernan, N.T., & Koedinger, K.R. (1997) The composition effect in symbolizing: The role of symbol production vs. text comprehension. in proceedings of Nineteenth Annual Conference of the Cognitive Science Society, 307-12. Hillsdale, NJ: Erlbaum. | |

− | + | Koedinger, K.R., & Anderson, J.R. (1997). Intelligent Tutoring Goes to School in the Big City. International Journal of Artificial Intelligence in Education 8, 30-43 | |

− | + | Koedinger, K. R. & Cross, K. (2000). Making informed decisions in educational technology design: Toward meta-cognitive support in a cognitive tutor for geometry. Presented at the annual meeting of the American Educational Research Association, New Orleans, LA. | |

− | + | Owen, E., & Sweller, J. (1985). What do students learn while solving mathematics problems? Journal of Educational Psychology, 77, 272-284. | |

− | + | Simon, H. A., & Lea, G. (1974). Problem solving and rule induction: A unified view. In L. W. Gregg (Ed.), Knowledge and cognition. Hillsdale, NJ: Erlbaum. | |

− | + | ||

− | + | ||

[[Category:Empirical Study]] | [[Category:Empirical Study]] |

## Revision as of 08:41, 18 September 2006

## The Composition Effect - What is the Source of Difficulty in Problems which Require Application of Several Skills?

Yvonne Kao, Ido Roll, Kenneth E. Koedinger

### Abstract

Composite problems, i.e., problems that require the application of more than one skill, are shown to be harder than a collection of single-step problems requiring the application of the same set of skills. A common explanation is that the composition itself imposes another difficulty level. However, an alternative explanation suggests that the composition makes the application of the individual skills harder. According to that explanation, poor feature validity and shallow domain rules make it harder on students to apply the individual skills correctly in the cluttered environment of composite problems, regardless the need to apply additional skills. Our study investigates these issues in two ways: (1) Having a DFA which evaluates performance on composite problems and single-step problems using the same data, and (2) By evaluating the effect of instruction targeting a common misconception in single-step problems on composite problems.

### Glossary

- Composite problems: Problems which require the application of several skills, such as solving 3x+6=0 for x.

- Single-step problems: Problems which require the application of a single skill, such as y+6=0 or 3x=-6

- DFA (Difficulty Factor Analysis): A test that includes pairs of items varying along one dimension only. It allows to evaluate the difficulty level of the single dimensions along which the problems differ.

- The Composition Effect: The effect according to which composite problems are harder than a set of single-step problems using the same skills.

### Research question

What is the main source of difficulty in composite problems?

### Background and Significance

Significance: This study can shed some light on the source of difficulty on composite problems, and thus can inform the design of relevant instruction and remediation.

### Independent Variables

An instruction in the form of solved-example, targeting a common misconception - identifying base and hight in a cluttered environment.

### Dependent variables

Three tests are used in the study: - Pre-test: given before all instruciton - Mid-test: given after students learned about single-step problems and before composite problems - Post-test: after students have learned and practice all material.

The tests include the following items. Some of which are transfer items, evaluating robust learning, since they require and adaptive application of the knowledge learned and practiced in class.

- Simple diagram: no distractors, canonical orientation - Simple diagram: distractors, canonical orientation - Simple diagram: no distractors, tilted orientation - Simple diagram: distractors, tilted orientation - Given complex diagram, ask for skill A - Given complex diagram, ask for skill B - Given steps A and B, ask for skills C (which requires A and B) - Given complex diagram, ask for C (which requires A and B)

### Hypothesis

The difficulty level in composite problems originates in poor feature validity of the single skills.

### Findings

### Explanation

### Descendents

### Annotated bibliography

Bransford (2000). How people learn: brain, mind, experience, and school National Academy Press. Heffernan, N.T., & Koedinger, K.R. (1997) The composition effect in symbolizing: The role of symbol production vs. text comprehension. in proceedings of Nineteenth Annual Conference of the Cognitive Science Society, 307-12. Hillsdale, NJ: Erlbaum. Koedinger, K.R., & Anderson, J.R. (1997). Intelligent Tutoring Goes to School in the Big City. International Journal of Artificial Intelligence in Education 8, 30-43 Koedinger, K. R. & Cross, K. (2000). Making informed decisions in educational technology design: Toward meta-cognitive support in a cognitive tutor for geometry. Presented at the annual meeting of the American Educational Research Association, New Orleans, LA. Owen, E., & Sweller, J. (1985). What do students learn while solving mathematics problems? Journal of Educational Psychology, 77, 272-284. Simon, H. A., & Lea, G. (1974). Problem solving and rule induction: A unified view. In L. W. Gregg (Ed.), Knowledge and cognition. Hillsdale, NJ: Erlbaum.