PhD Dissertation Defense - Nabin Maharjan

Deeper Understanding of Student Responses and Tutorial Dialogue Interactions

Nabin Maharjan, PhD Candidate

Friday, Mar. 1, 2019, 11:00 am
Dunn Hall 375 Conference Room

Abstract

Bloom (1984) reported two standard deviation improvement with human tutoring which inspired many types of research and studies towards developing Intelligent Tutoring Systems (ITSs) that are as effective as human tutoring. However, recent studies suggest that the 2-sigma result was misleading and the current ITSs are as good as human tutors. Nevertheless, we can think of 2 standard deviations as the benchmark for tutoring effectiveness of ideal expert tutors which both average human tutors and ITSs aspire to achieve in the long run. In the case of ITSs, there is still the possibility that ITSs could be better than humans. One way to improve the performance of the ITSs would be identifying, understanding, and then successfully implementing effective tutorial strategies that lead to learning gains.

Another step towards improving the effectiveness of ITSs is an accurate assessment of student responses. Incorrectly assessing student responses may confuse the students and decrease their motivation for learning. However, evaluating student answers in tutorial dialogues is challenging. The student answers often refer to the entities in the previous dialogue turns and problem description. Therefore, the student answers should be evaluated by taking dialogue context into account.

Moreover, merely assessing the degree of correctness of the student responses during tutoring is not sufficient. The system should explain which parts of the student answer are correct and which are incorrect. Such explanation capability allows the ITSs to provide targeted feedback to help students reflect upon and correct their knowledge deficits. Furthermore, targeted feedback increases learners' engagement, enabling them to persist in solving the instructional task at hand on their own.

In this dissertation, we describe our approach to discover and understand effective tutorial strategies employed by effective human tutors while interacting with learners. We also present various approaches to automatically assess students' contributions using general methods that we developed for semantic analysis of short texts. We explain our work using generic semantic similarity approaches to evaluate the semantic similarity between individual learner contributions and ideal answers provided by experts for target instructional tasks. We also describe our method to assess the student performance based on tutorial dialogue context, accounting for linguistic phenomena such as ellipsis and pronouns. We then propose an approach to provide an explanatory or interpretable capability for assessing student responses. Finally, we recommend a novel method based on concept maps for jointly evaluating and interpreting the correctness of the student responses.