The limits of annotation in machine learning a documents Hohfeldian legal entities
View / Open Files
Authors
Publication Date
2021-11-15Type
Conference Object
This Version
VoR
Metadata
Show full item recordCitation
Izzidien, A. (2021). The limits of annotation in machine learning a documents Hohfeldian legal entities. https://doi.org/10.33774/coe-2021-dqwvg
Abstract
Natural language processing (NLP) summarisers aim to capture the essential elements of a document. Yet, the ontological character of a summary can be domain specific. In legal analysis, the Hohfeldian matrix is used to summarise principle legal relations between agents, such as individuals and organisations. We test a limit of using machine learning (ML) to detect such agents. Based on training with our 2400 hand labelled annotations, an F1= 80.1 is found. Extrapolating this suggests that over one million annotations are required to capture all the agents mentioned in a document. This questions the feasibility of such an approach, one that is unable to be inclusive of all agents who are party to a legal relation. Such complete capture is an essential criteria of fair ML and accurate legal summaries. An alternative approach based on hypernymy is suggested.
Identifiers
External DOI: https://doi.org/10.33774/coe-2021-dqwvg
This record's URL: https://www.repository.cam.ac.uk/handle/1810/331062
Statistics
Total file downloads (since January 2020). For more information on metrics see the
IRUS guide.