Automatic Short Answer Grading at scale using Large Language Models
Project description: The rapid expansion of online and large-scale educational platforms has significantly increased the volume of student assignments, creating a growing demand for efficient and effective grading systems. To address this challenge, our project proposes the implementation of an Automatic Short Answer Grading (ASAG) system, utilizing advancements in Large Language Models (LLMs) to streamline the grading process.
This project enables instructors, irrespective of their familiarity with LLM technology, to utilize the ASAG system for grading free-form textual short answers. The system leverages pretrained models such as OpenAI's GPT variants, as well as specialized locally trained models, to provide accurate, fair, and timely feedback on student submissions. This initiative will not only reduce the grading burden on educators but also enhance the educational experience by delivering immediate and constructive feedback to students, thereby promoting deeper engagement and continuous learning.
Key research questions addressed in this project include the optimization of prompt design, the necessity and structure of training data for LLMs, and the system's ability to function in real-time environments. Additionally, we explore the system's capacity to provide equitable and unbiased feedback across diverse student populations, as well as study human perceptions regarding unreliable systems.
Project codebase: ai-grading.ts, ai-grading.sql