Using Elaborated Explanations to Support Geometry Learning (Aleven & Butcher)

From LearnLab
Revision as of 07:13, 18 April 2007 by Koedinger (Talk | contribs) (Dependent variables)

Jump to: navigation, search

Using Elaborated Explanations to Support Geometry Learning

Vincent Aleven and Kirsten Butcher 

Summary Tables

Pilot Study

PIs Vincent Aleven & Kirsten R. Butcher
Other Contributers Graduate Students: Andy Tzou (CMU HCII), Carl Angioli (CMU HCII), Michael Nugent (Pitt, Computer Science)

Research Programmers/Associates: Octav Popescu (Research Programmer, CMU HCII), Grace Lee Leonard (Research Associate, CMU HCII), Thomas Bolster (Research Associate, CMU HCII)

Study Start Date January 24, 2006
Study End Date January 31, 2006
LearnLab Site Central Westmoreland Career & Technology Center (CWCTC)
LearnLab Course Geometry
Number of Students 90
Total Participant Hours 90
DataShop n/a (Pencil & Paper Test)


Classroom Study

PIs Vincent Aleven & Kirsten R. Butcher
Other Contributers Graduate Students: Carl Angioli (CMU HCII), Michael Nugent (Pitt, Computer Science)

Research Programmers/Associates: Octav Popescu (Research Programmer, CMU HCII), Grace Lee Leonard (Research Associate, CMU HCII), Thomas Bolster (Research Associate, CMU HCII)

Study Start Date November 28, 2006
Study End Date January 15, 2007
LearnLab Site Central Westmoreland Career & Technology Center (CWCTC)
LearnLab Course Geometry
Number of Students 90
Total Participant Hours 540
DataShop Status: In Progress. Log files and CTAT online test files have been anonymized, passed to DataShop, and are awaiting conversion and input.


Abstract

Does integration of visual and verbal knowledge during learning support deep understanding? Can interactive communication using visual and verbal knowledge components support the development of integrated representations? In our research, we are exploring how student explanations in an intelligent tutoring environment that includes visual (pictorial) and verbal (textual) information can support robust learning. In domains like Geometry, where visual (diagram) and verbal (textual and conceptual) information is critical to deep understanding and successful problem solving, focusing student communication acts on conceptual relationships between critical knowledge components should support deep understanding. Thus, our research is investigating explanations that link the diagram features (visual knowledge components) and geometry rules (verbal knowledge components) related to successful problem-solving steps. We call these visual-verbal explanations "Elaborated Explanations." In our studies, students give a verbally-focused explanation (by stating a relevant geometry rule) as well as a visually-focused explanation (by stating the diagram features that used to apply the stated rule). The intelligent tutor provides feedback and hints on these elaborated, verbal-visual explanations. We hypothesize that this type of interactive communication will support integration of visual and verbal knowledge during learning, leading to better transfer and long-term retention.

Background & Significance

A rich body of prior research has demonstrated that students develop deeper understanding of instructional materials when they self-explain to themselves during learning (e.g., Bielaczyc, Pirolli, & Brown, 1995; Chi, Bassok, Lewis, Reimann, & Glaser, 1989; Chi, de Leeuw, Chiu, & LaVancher, 1994). Although much of this research has focused on students self-generated and natural language explanations, research has also shown benefits of simple, menu-based explanations in supporting learning. An existing version of the Geometry Cognitive Tutor implements student self-explanations in this simple manner. After correctly answering a geometry problem step, students must select the geometry rule or theorem that justifies their answer from a glossary menu of terms.

Despite the limitations of simple, menu-based explanations, they have been shown to promote student learning in the Geometry Cognitive Tutor (Aleven & Koedinger, 2002). However, these menu-based explanations may not be supporting the development of deep, expert-like connections between conceptual geometry principles and visual features of geometry problem diagrams. Students complete these explanations after they have successfully solved a numerical problem-solving step. There there is no requirement that students make explicit connections between the rule being applied and the visual features that are required for application. Because prior work has shown that experts' geometry problem solving is driven by recognition of key diagram features that cue relevant conceptual knowledge (Koedinger & Anderson, 1990), supporting deep student understanding in geometry may require that students explicitly connect relevant conceptual knowledge to visual diagram features.

Our approach is to support students' connections between and integration of visual and verbal knowledge components through interactive communication, where students' explanations of their problem solving steps reference both verbal, conceptual knowledge (in the form of a relevant geometry rule) and its relationship to key diagram features. Thus, our question is whether explicit forms of communication that link verbal and visual knowledge components can be more successful at promoting robust learning than communication that requires expression of verbal, declarative knowledge only.

Glossary

See Visual-Verbal Learning Project Glossary

Research questions

  1. Can elaborated explanations that connect visual (pictorial) and verbal (textual/conceptual) knowledge components support robust learning better than explanations that reference verbal information only?
  2. Are the effects of elaborated explanations strongest when considering long-term retention and transfer as opposed to results on a normal post-test?

Independent Variables

  • Type of Explanation
    • Simple, Textual Explanations (students state geometry principles only)

Figure 1. Screen shot of Geometry Cognitive Tutor interface with simple, verbal-only explanations.
Diagram SimpleExpl.jpg
vs.

  • Elaborated Explanations (students state geometry principles and the diagram features which allow the principle to be applied)

Figure 2. Screen shot of Geometry Cognitive Tutor interface with elaborated explanations
Diagram ElabExpl.jpg

Dependent variables

  • Normal post-test, and immediate transfer test measuring student performance on:
    • Problem-solving items isomorphic to the practiced problems (immediate retention)
    • Problem-solving items unlike those seen during problem practice (immediate transfer)
  • Delayed long-term retention posttest, measuring student performance on:
    • Problem-solving items isomorphic to the practiced problems (long-term retention)
    • Problem-solving items unlike those seen during problem practice (long-term transfer)
  • Log data collected during tutor use, used to assess:

Hypothesis

  • elaborated explanations promote integration of visual and verbal knowledge components during problem-solving. Thus, students who engage in this form of interactive communication (explaining verbal and visual information, and receiving feedback on all explanations) will demonstrate deeper understanding as evidenced by improved transfer and long-term retention.

Findings

Pilot Study: Paper-based Difficulty Factors Analysis

  • Study Summary
    • Participants: Three 10th grade geometry classes in a rural Pennsylvannia school
    • Design: In vivo experiment: Paper-based Difficulty Factors Analysis (DFA) covering Angles content from the Geometry Cognitive Tutor
    • Problems varied along two dimensions:
      • Problem Format: Diagram vs. Table. In Diagram-format problems, students entered answers in the appropriate location within the geometry diagram. In the Table format, students entered answers in a table separate from the geometry diagram.
      • Explanation Type: Simple vs. Elaborated Explanations. Simple explanations required students to name only the geometry rule that justified their problem-solving steps. Elaborated explanations required students to name the geometry rule and the known diagram features that allowed them to use the stated geometry rule.
    • Implementation: Students using the standard version of the Geometry Cognitive Tutor took the paper-based DFA test midway through their completion of the Angles unit on the tutor.
  • Findings
    • Although there is an overall trend that, with no practice, students perform best in the familiar Table format that they were using during their Cognitive Tutor practice (F (1, 88) = 3.47, p = .07), the type of explanations that students provided significantly influenced performance (F (1, 88), = 6.75, p = .01). Students who gave elaborated explanations (the geometry rule and relevant diagram features) performed better on problem-solving (M = .42, SE = .04) than students who gave simple explanations (the geometry rule only, M = .34, SE = .04).

Classroom: Elaborated Explanations with Tutored Practice

  • Study Summary
    • Participants: Six 10th grade geometry classes in a rural Pennsylvannia school
    • Design: In vivo experiment: 2 (Simple vs. Elaborated Explanations) X 2 (Contiguous vs. Noncontiguous tutor interface)
    • Implementation: Students were randomly assigned to one of the four experimental conditions. Time in tutor was held constant: students completed three classroom sessions with their assigned version of the tutor over a three week period. Immediate posttesting was conducted one week after the final tutor session. Delayed posttesting was completed after a four-week delay.
  • Anticipated Findings
    • (Study data is currently being uploaded to the Datashop.)
    • We anticipate that elaborated explanations will support students' performance on transfer tasks, especially the ability to reason about conceptual relationships in a geometry diagram.
    • Further, we anticipate that long-term retention performance will show an advantage for elaborated explanations, because integrated visual-verbal knowledge should support recall of relevant geometry principles based on critical diagram features.

Explanation

We have evidence from the (paper-based) difficulty factors analysis that elaborated explanations support problem solving performance. We hypothesize that the classroom test of elaborated explanations will show support for robust learning. That is, we anticipate that the classroom test will show not only support for long-term retention but also transfer of geometry knowledge.

From an Interactive Communication Cluster perspective, the Elaborated Explanations help students focus on the right content for problem solving. Students will be less likely to engage in shallow strategies or to focus on irrelevant features in problem solving when tutored to make these explanations. Further, by relating these critical knowledge components via an explicit, communicative act, students also should integrate these knowledge components into a rich knowledge representation.

From a broader PSLC perspective, elaborated explanations support coordination through self-explanation of visual and verbal information. Communication that makes use of multiple representations during learning likely affects path choice. For example, when students are required to explain the application of geometry principles using diagrams, there will be only small differences in estimated effort of shallow and deep strategies since shallow strategies are unlikely to achieve the correct answer. Further, we anticipate that elaborated explanations also produce path effects: the processes that students employ via path choice are more effective when the materials support use of visual and verbal information during sense making. Specifically, scaffolds or materials that support sense making with visual and verbal information should promote integration.

Further Information

Connections

Implicit Support for Visual-Verbal Knowledge Integration:
Our research is investigating multiple methods with which student learning can be supported by interactions with pictorial information during geometry learning. Our work also includes more implicit methods for supporting student integration visual and verbal knowledge components. These methods include Contiguous vs. Noncontiguous Representations and Integrated Hints [Link to be added].

Specificity in Interactive Communication:
The addition of visually-related explanations may force students to be more specific in explaining how a geometry principle applies to a problem-solving step. This potential interpretation is related to the hypothesis that elaborative interaction supports learning by increasing the specificity of another person's comments. In fact, Hausmann & Chi showed that elaborative interaction improves deep learning.

Application Information in Interactive Communication:
The addition of visually-related explanations may help students by directing their attention to content that governs the application of information during problem solving. This potential interpretation is related to the hypothesis being tested by Ringenberg & VanLehn, that completely justified examples support robust learning by demonstrating the reasoning and all application steps needed for problem-solving.

Annotated Bibliography

  • Presentation to the PSLC Advisory Board, Fall 2006. Link to Powerpoint slides
  • Presentation to the PSLC Interactive Communication Cluster, Fall 2006.

References

  • Aleven, V., & Koedinger, K. R. (2002). An effective metacognitive strategy: Learning by doing and explaining with a computer-based Cognitive Tutor. Cognitive Science, 26(2), 147-179.
  • Bielaczyc, K., Pirolli, P. L., & Brown, A. L. (1995). Training in self-explanation and self-regulation strategies: Investigating the effects of knowledge acquisition activities on problem solving. Cognition & Instruction, 13, 221-252.
  • Chi, M. T., Bassok, M., Lewis, M. W., Reimann, P., & Glaser, R. (1989). Self-explanations: How students study and use examples in learning to solve problems. Cognitive Science, 13, 145-182.
  • Chi, M. T. H., de Leeuw, N., Chiu, M.-H., & LaVancher, C. (1994). Eliciting self-explanations improves understanding. Cognitive Science, 18, 439-477.
  • Koedinger, K. R., & Anderson, J. R. (1990). Abstract planning and perceptual chunks: Elements of expertise in geometry. Cognitive Science, 14, 511-550.

Future Plans: June 2007 - December 2007

  1. Once data is uploaded to DataShop, analyze log and assessment data
  2. Transcribe data think-aloud from lab study
  3. Code and analyze think-aloud data from lab study
  4. Prepare manuscript