Repository logo
 

The limits of annotation in machine learning a documents Hohfeldian legal entities

Published version
Peer-reviewed

Type

Conference Object

Change log

Authors

Abstract

Natural language processing (NLP) summarisers aim to capture the essential elements of a document. Yet, the ontological character of a summary can be domain specific. In legal analysis, the Hohfeldian matrix is used to summarise principle legal relations between agents, such as individuals and organisations. We test a limit of using machine learning (ML) to detect such agents. Based on training with our 2400 hand labelled annotations, an F1= 80.1 is found. Extrapolating this suggests that over one million annotations are required to capture all the agents mentioned in a document. This questions the feasibility of such an approach, one that is unable to be inclusive of all agents who are party to a legal relation. Such complete capture is an essential criteria of fair ML and accurate legal summaries. An alternative approach based on hypernymy is suggested.

Description

Keywords

Journal Title

Conference Name

Journal ISSN

Volume Title

Publisher