Adaptive Learning Curve Analytics with LLM-KC Identifiers for Knowledge Component Refinement

Accurate modeling of student knowledge is essential for delivering timely, targeted feedback in Intelligent Tutoring Systems (ITS). Knowledge Components (KCs)—discrete units of domain knowledge—have traditionally been handcrafted by experts, a process that is both time-consuming and difficult to scale. In this work, we replicate a recent Large Language Model (LLM)-based approach to automate KC extraction called LLM-KC Identifier (LLM-KCI) and extend on it by evaluating the extracted KCs using learning curve analysis. By comparing LLM-generated KCs against expert-annotated counterparts in an introductory programming course, we demonstrate that LLMs not only match experts in capturing core concepts but also bring unique advantages: consistent identification across diverse assignments and scalability. Through static overlap metrics (Jaccard similarity, overlap coefficient) and learning curve analyses, we show that certain LLMs (e.g., GPT-4o, DeepSeekR1) produce error-rate trajectories as smooth or smoother than expert-annotated KC models, effectively isolating student learning trends. Our findings suggest that automated KC extraction can become a mainstream tool for personalized learning analytics, enabling educators to rapidly adapt curriculum and interventions at scale.