Repository logo
 

Skill embeddings: artificial neural network representations for pedagogical policy development.


Type

Thesis

Change log

Authors

Moore, Russell 

Abstract

Online education is a growing industry, especially in light of the ongoing (2020-) coro- navirus pandemic, which has restricted physical access to schools, colleges and work- places. Educational service providers typically develop their own content and pedagogical pro- gramming, which - even with modern authoring tools - is a costly and time-consuming exercise. A wealth of less formal material exists in the public domain but such mate- rial cannot simply be brought into service as it is. There is no common standard to describe what skills are taught, practised, or assessed, by these mixed materials: this makes combining them in a common pedagogical framework impracticable in the gen- eral case. This thesis researches methods to gain pedagogical insight from such multi-origin items, and in particular the use of novel neural network techniques to infer the latent content of instructional and assessment events, based on observed human interaction data. Of particular interest are complex tasks requiring a mixed skill set. First, the work looks at inferring composition and difficulty of items, in a static testing scenario, by training an artificial neural network to learn representations - skill embed- dings - of students and items that relate to each other in a common vector space. Second, a practice-based approach to directly infer a pedagogical policy is given. This uses student histories and behavioural cloning of historical teacher decisions to make choices about what homework to set. The third stage combines these two techniques, taking the experiential model from the homework generation task, and combining it with the embedding technique to create longitudinal skill embeddings based on both student-item interaction responses and stu- dent practice patterns in a formative assessment scenario. Several extant and two novel multidimensional interaction models are compared in terms of their predictive skill and their reliability in capturing similarities between objects in the embedding space. The work introduces a benchmark (MPDRC) to measure the latter, and show how the em- beddings can be interpreted with relevance to educational policy design in terms of knowledge, learning and instruction.

Description

Date

2021-01-01

Advisors

Buttery, Paula
Rice, Andrew

Keywords

Neural Networks, Skill Embeddings, AI for Education, Machine Learning

Qualification

Doctor of Philosophy (PhD)

Awarding Institution

University of Cambridge
Sponsorship
Many thanks to Cambridge University Press & Assessment (CUP&A) for providing funding for this work via the Automated Language Teaching & Assessment (ALTA) In- stitute, and also to the Digital & New Product Development (DNPD) team at CUP&A for providing the data set used in Chapter 5. Thanks also to the Isaac Physics team at both the Cavendish Laboratory and Computer Laboratory for providing me with data for the work in Chapter 4.