Jan 20 2012DataShop 5.2 released!

DataShop terms of use, project pages, PIs, and bug fixes

Terms of Use

DataShop now has a terms of use, which you will be asked to agree to next time you sign in. The terms say in plain language many of the things that we generally communicated over time—for example, that data is for research purposes.

Project pages and PIs

To make the concept of a "project" clearer, we've created project pages for each project in DataShop, which are linked to from the My/Public/Other Datasets tabs. For now, a project page lists the datasets in that project and the principal investigator's name, but in the future, it could hold much more information (papers and files, for example).

As part of this change, we've also moved the principal investigator (PI) field from the dataset to the project. There is now one PI for each project in DataShop.

Minor changes

KC Models subtab. You will notice a new "KC Models" subtab beneath the main "Learning Curve" tab. This is the same page as the one currently below "Dataset Info"; we've just added a link to it so that you can move between KC Models, Line Graph, and Model Values more easily.

Importing inputs greater than 255 characters. For new data where a value in the "Input" column is greater than 255 characters, DataShop will split the text into multiple "Input" columns, each no greater than 255 characters.

Bug fixes. We've made various bug fixes to the code that determines the Problem View and Problem Start Time columns from raw data.

More Information

Jan 16 2012New Course: 05-438/05-838: The Role of Technology in Learning in the 21st Century

How can educational games and technology impact global poverty?

05-438/05-838: The Role of Technology in Learning in the 21st Century
Tuesdays and Thursdays, 12 noon to 1:20 PM, GHC 5222
Prof. Matthew Kam (mattkam@cs.cmu.edu)
Human-Computer Interaction Institute, CMU

Take an entrepreneurial approach to designing an educational initiative that uses technology to change the face of education in both the developing and industrialized world. Rapid changes in today’s world are fast rendering previously-acquired knowledge and skills obsolete, thereby creating an impetus for continual, lifelong learning. This challenge calls for leaders in society who can effect meaningful learning, in their roles as workplace managers, parents, and agents of change in their local communities.

The course will equip participants with an integrated breadth of multidisciplinary knowledge about local educational contexts, the psychology behind how humans acquire expertise, educational computing technologies, human-centered design thinking, as well as advanced topics such as business models, education policy, reading literacy, second language acquisition and STEM (science, technology, engineering and mathematics) education. Understanding of the course material will be reinforced through case studies from all over the world. By the end of the course, participants will have a “survival level” of knowledge about the latest scientific research on human learning to design and implement high-impact educational programs in the workplace, home and classroom.

This course is being offered for the third time in Spring 2012. It is open to undergrads and graduate students from all majors. No computer programming experience is necessary. Participants are encouraged to bring their own project ideas to the course.

Jan 04 2012John Stamper's Invited Talk at EDM is Now Online

Abstract: Technology advances have made the ability to collect large amounts of data easier than ever before. These massive datasets provide both opportunities and challenges for many fields and education is no different. Understanding how to deal with extreme amounts of student data in the EDM field is a growing problem. The 2010 KDD Cup Competition, titled "Educational Data Mining Challenge", included data for over 10,000 students. The students completed over 30 million problem steps collected over a year long courses from Carnegie Learning Inc.'s Cognitive Tutors. We believe these are the largest educational dataset at this level of granularity to be released publicly. The competition drew broad interest from the data mining community, but it was also clear that many in the research community could not handle datasets of this size. In this talk, John will discuss the 2010 KDD Cup and the impact of larger and larger amounts of data coming available for educational data mining and how this will drive the direction of educational research in the future.

More Information

Jan 04 20122011 James Chen Annual Award for Best UMUAI paper

The UMUAI James Chen Award is bestowed annually to the author(s) of the best paper of the year. The award has been donated by the Chen family in commemoration of James R. Chen, a creative researcher in the area of user modeling and information retrieval, and twice a UMUAI author. The award comes with a cash prize of U.S.$ 1,000. To select the winner, an award committee is formed from the editorial board of the journal which reviews all articles that were nominated by reviewers and the editorial board.

This year the winning papers were both written by LearnLab affiliated authors. Congratulations to Ken, Kurt and Brett.

UMUAI James Chen Award recipients:

2011

Brent Martin, Dept. of C.S. and Software Engineering, Univ. of Canterbury, Christchurch, New Zealand
Antonija Mitrovic, Dept. of C.S. and Software Engineering, Univ. of Canterbury, Christchurch, New Zealand
Kenneth R. Koedinger, HCI Institute, Carnegie Mellon University, Pittsburgh, PA, USA
Santosh Mathan, Human Centered Systems Group, Honeywell Labs, Minneapolis, USA
Evaluating and improving adaptive educational systems with learning curves
UMUAI 21:3, 2011, 249–28

Kasia Muldner, Department of Psychology, Arizona State University, USA
Winslow Burleson, School of Computing, Informatics and Decision Systems Engineering, ASU, USA
Brett Van de Sande, School of Computing, Informatics and Decision Systems Engineering, ASU, USA
Kurt VanLehn, School of Computing, Informatics and Decision Systems Engineering, ASU, USA
An analysis of students’ gaming behaviors in an intelligent tutoring system: predictors and impacts
UMUAI 21:1-2, 2011, 99-135

More Information

Dec 07 2011Call for Papers for EDM 2012: June 19-21, 2012

======================================
Call for Papers
EDM 2012: The Fifth International Conference on Educational Data Mining
19-21 June 2012 in Chania, Crete, Greece
http://educationaldatamining.org/EDM2012/
=======================================

We invite submissions to the 5th International Conference on Educational Data Mining (EDM2012), to be held on 19-21 June 2012 in Chania, Crete, Greece.

The EDM 2012 conference is organized under the auspices of the International Educational Data Mining Society.

The EDM 2012 conference is a leading international forum for high quality research that mines large data sets of educational data to answer educational research questions. These datasets may come from learning management systems, interactive learning environments, intelligent tutoring systems, or any system used in a learning context. EDM 2012 is a highly disciplinary conference that brings together researchers from computer science, machine learning and data mining, artificial intelligence in education, intelligent tutoring systems, education, learning sciences, psychometrics, statistics and cognitive psychology.

The theme of the EDM 2012 conference is “From Data to Information: Empowering Learning Environments and Settings”. We particularly solicit submissions that describe how EDM approaches transform the educational setting and empirical studies.

EDM may require adaptation of existing or development of new approaches that build upon techniques from a combination of areas, including but not limited to statistics, psychometrics, machine learning, information retrieval, recommender systems and scientific computing.

EDM 2012 will immediately follow the Eleventh International Conference on Intelligent Tutoring Systems (ITS 2012), 14-18 June 2012, at the same location.

================== TOPICS OF INTEREST ==================

Topics of interest to the conference include, but are not limited to:
- Generic frameworks, methods and approaches for EDM
- Improving educational software. Many large educational data sets are generated by computer software. Can we use our discoveries to improve the software’s effectiveness?
- Domain representation. How do learners represent the domain? Does this representation shift as a result of instruction? Do different sub-populations represent the domain differently?
- Evaluating teaching interventions. Student learning data provides a powerful mechanism for determining which teaching actions are successful. How can we best use such data?
- Emotion, affect, and choice. The student’s level of interest and willingness to be a partner in the educational process is critical. Can we detect when students are bored and uninterested? What other affective states or student choices should we track?
- Integrating data mining and pedagogical theory. Data mining typically involves searching a large space of models. Can we use existing educational and psychological knowledge to better focus our search?
- Improving teacher support. What types of assessment information would help teachers? What types of instructional suggestions are both feasible to generate and would be welcomed by teachers?
- Replication studies. We are especially interested in papers that apply a previously used technique to a new domain, or that reanalyze an existing data set with a new technique.
- Best practices for adaptation of data mining techniques to EDM, information retrieval, recommender systems, opinion mining, and question answering techniques

==================== SUBMISSION PROCESS ====================

All submissions should follow the ACM SIG KDD Explorations submission format (examples at the website)
- Full papers (up to 8 pages). Should describe original, substantive, mature and unpublished work.
- Short papers (4 pages). Should describe original, highly promising and unpublished work, whose merit will be assessed in terms of originality and importance rather than maturity and technical validation.
- Posters and Demos (2 pages). Posters describe original and unpublished work in progress and last minute results. Demos describe educational data mining tools and systems, or educational systems that use EDM techniques.
- Doctoral consortium (up to 3 pages). Should describe the graduate/postgraduate student’s research topic, proposed contributions, results so far, and aspects of the research on which advice is sought. Should be solely authored by the student.

Submissions will be accepted through easychair (https://www.easychair.org/conferences/?conf=edm2012)

Each submitted paper will be reviewed by at least three reviewers. Accepted papers will be published in the EDM2012 proceedings and will also appear online on this website.

================ IMPORTANT DATES ================

5 February 2012, Abstract submissions due
12 February 2012, Full and short paper submissions due
19 February 2012, Doctoral consortium submissions due
2 April 2012, Notification of acceptance (Full, short, doctoral consortium)
5 April 2012, Poster and demo submissions due
16 April 2012, Notification of acceptance (Posters and demos)
22 April 2012, Final papers due
19-21 June 2012, Conference days

======================= CONFERENCE ORGANIZERS =======================

Conference chair: John Stamper, Carnegie Mellon University

Program chairs: Kalina Yacef, University of Sydney Osmar Zaiane, University of Alberta

Poster and demo chairs: Arnon Hershkovitz, Worcester Polytechnic Institute Michael Yudelson, Carnegie Mellon University

Doctoral Consortium chairs: Art Graesser, University of Memphis Zachary Pardos, Worcester Polytechnic Institute

Web chair: Michael Bett, Carnegie Mellon University

Local Organization: Kitty Panourgia, Neoanalysis

More Information

Dec 07 2011Spring Course: Research Methods for the Learning Sciences 05-748

Research Methods for the Learning Sciences 05-748

Goals
The goals of this course are to learn data collection, design, and analysis methodologies that are particularly useful for scientific research in education. The course will be organized in modules addressing particular topics including cognitive task analysis, qualitative methods, protocol and discourse analysis, survey design, psychometrics, educational data mining, and experimental design. A key goal is to help students think about and learn how to apply these methods to their own research programs.

Course Prerequisites
To enroll you must have taken 85-738, "Educational Goals, Instruction, and Assessment" or get the permission of the instruction.

Textbook
"The Research Methods Knowledge Base: 3rd edition" by William M.K. Trochim and James P. Donnelly. Other readings will be assigned in class.

Dec 07 2011Spring Course: Cognitive Modeling & Intelligent Tutoring Systems CMU/HCII 05832 / 05432


Course information

* 9 credits or 12 credits
* Tuesday and Thursday, 10:30AM - 11:50AM, GHC 4211

This course addresses the use of cognitive psychology and cognitive task analysis to create computer-based intelligent tutoring systems. Students will learn data-driven and theoretical methods for creating cognitive models of human problem solving. Such models have been used to create educational software that has been demonstrated to dramatically enhance student learning in domains like mathematics and computer programming. This type of software, which originated at CMU and is now widely used in US high schools and middle schools, is probably the premier application of cognitive science in education.

In addition to discussion and readings on methods and models of problem solving, learning, and tutor design, the course will have a substantial “learning by doing” component. Students will be analyzing data, designing cognitive models and interfaces, and implementing an intelligent tutoring system.

This course is offered as a 9-credit version and a 12-credit version. Students in both the 12-credit and the 9-credit version will use CTAT (the Cognitive Tutor Authoring Tools, see http://ctat.pact.cs.cmu.edu) to build a prototype intelligent tutoring system. The 9-credit version of the course does not involve programming, the 12-credit version involves rule-based programming to construct cognitive models.

The course targets students in Human-Computer Interaction, Psychology, Computer Science, Design, or related fields, who are interested in educational applications. Students should either have programming skills, or experience in the cognitive psychology of human problem solving, or experience with instructional design.

More Information

Dec 07 2011Carolyn Rose Guest Speaker

Guest Speaker, Detecting Social Dynamics in Speech, Indo-US Workshop on Analytics, IISc, Bangalore, Dec 2011

Invited Talk, Detecting Social Dynamics in Speech, IBM Delhi, Spoken Web group, December 14, 2011.

Oct 14 2011DataShop 5.1 released!

Redesigned KC Models page, cross validation, citations, and an important change to the transaction format

Redesigned KC Models page

We redesigned the KC Models page as a table to make it easier to compare models. You can sort the models by any of the statistics in the table using the combobox at the top of the page. By default, KC models are now sorted by AIC instead of BIC (lowest to highest, or best fit with fewest parameters to worst fit with additional parameters) and then by model name. The sort order chosen also affects the order of models in the KC Models combobox seen in the navigation area of various reports in DataShop.

Cross validation

Cross validation statistics for KC models have been expanded to include two new methods of cross validation: student stratified and item stratified. Cross validation is a technique for assessing how well the results of a statistical model (in this case, AFM for a particular KC model) will generalize to an independent dataset from the same tutor. It's reported as root mean squared error (RMSE). Lower values of RMSE indicate a better fit between the model's predictions and the observed data. More information on these new statistics is available on the Model Values help page.

Tell other researchers how to cite your dataset

We've added two new fields to the Dataset Info page: Acknowledgment for Secondary Analysis and Preferred Citation for Secondary Analysis. Using these fields, you can display a citation and/or acknowledgement that others should use if they publish research based on your dataset. Once filled in, the citation/acknowledgement is shown in two other places: on a new subtab called Citation, and in a text file that is included with each export of the data.

Changes to the transaction format

We've added two columns to the transaction format, Problem View and Problem Start Time. Problem Start Time identifies when a problem is shown to a student. This information was missing from the tab-delimited transaction format even though it was possible to log using the XML version of the format. It's now possible to export a dataset that has problem-start information and re-import it without losing any data. Similarly, you can now import a new dataset in the tab-delimited format that describes when a problem is shown to a student. Problem View serves a complimentary purpose. It counts how many times the current problem has been encountered by the student. The addition of these columns fixes a long-standing limitation of the tab-delimited transaction format.

You will find updated definitions for Problem View, Problem Start Time, and Step Start time on the export help page.

Data changes

As part of the changes to the transaction format, we've made changes to how problem view, problem start time, and step start time are calculated. These changes will modify metrics for datasets that were created via the tab-delimited format (or, in rare cases, if logged in the XML format without problem start information). These datasets will change in the following ways:

• For the first step of the problem, the step start time is no longer indeterminate, so instead of seeing a dot (".") for these steps in the student-step table, you'll see a time value. The same applies for the step's duration.
• Because of how problem start times and problem views are calculated, it's possible for the "attempt at step" count (seen in the transaction table) to be different. This difference means that metrics for KC models such as as AIC and BIC may be slightly different.

Bug fixes

• Web Services: requesting both custom fields and specific columns for transactions now includes custom fields
• Web Services: updated user agreement to include public web applications as a restricted use
• Web Services: we now support requesting the transaction columns "problem view" and "problem start time"
• Web Services: column shifting is no longer present in the student-step export

More Information

Sep 13 2011Conference on Socializing Intelligence Through Talk and Dialogue

Titled “Socializing Intelligence through talk and dialogue", this conference will bring together scholars from several different research traditions that have studied the role of talk and dialogue in (classroom) learning, as well as several leading scholars from outside these traditions, to collaboratively explore some of the strongest outcomes of participation in academic talk and to discuss ways in which the unresolved or unexplored issues in this field should be addressed in future research and with new technologies. The conference is primarily sponsored by AERA, with additional support from PSLC, LRDC and the Pitt School of Education.