Loading [MathJax]/jax/input/TeX/config.js

Error Classification in Stoichiometry Tutoring Systems with Different Levels of Scaffolding: Comparing Rule-Based Classification and Machine Learning

Research output: Other contributionOther publicationResearchpeer review

Authors

Research Organisations

Details

Original languageEnglish
Number of pages6
Publication statusE-pub ahead of print - 20 Jun 2025

Abstract

Comparing rule-based and machine learning (ML) approaches to error classification is crucial for advancing adaptive instruction. However, few studies have examined their comparative accuracy for tutoring systems with different levels of scaffolding. The present study addresses this gap by examining the classification of stoichiometry errors using data from 61 science students enrolled at a public German university who interacted with two distinct tutoring systems. We annotated 1,164 error clips from log data and derived an error classification scheme with eight categories covering system-related (e.g., usability) and domain-specific (e.g., unit conversion) categories. We developed decision rules and trained an ML model, comparing automatically classified errors in segments of learner inputs to classifications based on our expert model. Our results indicate that domain-specific errors requiring procedural knowledge are more accurately classified by the rule-based classifier, while concept-based errors are better captured by ML, though only in a lowly scaffolded tutoring system. These findings suggest researchers must carefully choose modeling approaches to address misconceptions in STEM learning.

Cite this

Error Classification in Stoichiometry Tutoring Systems with Different Levels of Scaffolding: Comparing Rule-Based Classification and Machine Learning. / Fleischer, Hendrik; Borchers, Conrad; Schanze, Sascha et al.
6 p. 2025.

Research output: Other contributionOther publicationResearchpeer review

Download
@misc{cc179baf73b64ed28cab7cbfdff3dc85,
title = "Error Classification in Stoichiometry Tutoring Systems with Different Levels of Scaffolding: Comparing Rule-Based Classification and Machine Learning",
abstract = "Comparing rule-based and machine learning (ML) approaches to error classification is crucial for advancing adaptive instruction. However, few studies have examined their comparative accuracy for tutoring systems with different levels of scaffolding. The present study addresses this gap by examining the classification of stoichiometry errors using data from 61 science students enrolled at a public German university who interacted with two distinct tutoring systems. We annotated 1,164 error clips from log data and derived an error classification scheme with eight categories covering system-related (e.g., usability) and domain-specific (e.g., unit conversion) categories. We developed decision rules and trained an ML model, comparing automatically classified errors in segments of learner inputs to classifications based on our expert model. Our results indicate that domain-specific errors requiring procedural knowledge are more accurately classified by the rule-based classifier, while concept-based errors are better captured by ML, though only in a lowly scaffolded tutoring system. These findings suggest researchers must carefully choose modeling approaches to address misconceptions in STEM learning.",
author = "Hendrik Fleischer and Conrad Borchers and Sascha Schanze and Vincent Aleven",
year = "2025",
month = jun,
day = "20",
doi = "10.35542/osf.io/edpgk_v1",
language = "English",
type = "Other",

}

Download

TY - GEN

T1 - Error Classification in Stoichiometry Tutoring Systems with Different Levels of Scaffolding: Comparing Rule-Based Classification and Machine Learning

AU - Fleischer, Hendrik

AU - Borchers, Conrad

AU - Schanze, Sascha

AU - Aleven, Vincent

PY - 2025/6/20

Y1 - 2025/6/20

N2 - Comparing rule-based and machine learning (ML) approaches to error classification is crucial for advancing adaptive instruction. However, few studies have examined their comparative accuracy for tutoring systems with different levels of scaffolding. The present study addresses this gap by examining the classification of stoichiometry errors using data from 61 science students enrolled at a public German university who interacted with two distinct tutoring systems. We annotated 1,164 error clips from log data and derived an error classification scheme with eight categories covering system-related (e.g., usability) and domain-specific (e.g., unit conversion) categories. We developed decision rules and trained an ML model, comparing automatically classified errors in segments of learner inputs to classifications based on our expert model. Our results indicate that domain-specific errors requiring procedural knowledge are more accurately classified by the rule-based classifier, while concept-based errors are better captured by ML, though only in a lowly scaffolded tutoring system. These findings suggest researchers must carefully choose modeling approaches to address misconceptions in STEM learning.

AB - Comparing rule-based and machine learning (ML) approaches to error classification is crucial for advancing adaptive instruction. However, few studies have examined their comparative accuracy for tutoring systems with different levels of scaffolding. The present study addresses this gap by examining the classification of stoichiometry errors using data from 61 science students enrolled at a public German university who interacted with two distinct tutoring systems. We annotated 1,164 error clips from log data and derived an error classification scheme with eight categories covering system-related (e.g., usability) and domain-specific (e.g., unit conversion) categories. We developed decision rules and trained an ML model, comparing automatically classified errors in segments of learner inputs to classifications based on our expert model. Our results indicate that domain-specific errors requiring procedural knowledge are more accurately classified by the rule-based classifier, while concept-based errors are better captured by ML, though only in a lowly scaffolded tutoring system. These findings suggest researchers must carefully choose modeling approaches to address misconceptions in STEM learning.

UR - https://doi.org/10.35542/osf.io/edpgk_v1

U2 - 10.35542/osf.io/edpgk_v1

DO - 10.35542/osf.io/edpgk_v1

M3 - Other publication

ER -

By the same author(s)