The limits of annotation in machine learning a documents Hohfeldian legal entities
View / Open Files
Authors
Publication Date
2021-11-15Type
Conference Object
This Version
VoR
Metadata
Show full item recordCitation
Izzidien, A. (2021). The limits of annotation in machine learning a documents Hohfeldian legal entities. https://doi.org/10.33774/coe-2021-dqwvg
Abstract
Natural language processing (NLP) summarisers aim to capture the essential elements of a document. Yet, the ontological character of a summary can be domain specific. In legal analysis, the Hohfeldian matrix is used to summarise principle legal relations between agents, such as individuals and organisations. We test a limit of using machine learning (ML) to detect such agents. Based on training with our 2400 hand labelled annotations, an F1= 80.1 is found. Extrapolating this suggests that over one million annotations are required to capture all the agents mentioned in a document. This questions the feasibility of such an approach, one that is unable to be inclusive of all agents who are party to a legal relation. Such complete capture is an essential criteria of fair ML and accurate legal summaries. An alternative approach based on hypernymy is suggested.
Identifiers
External DOI: https://doi.org/10.33774/coe-2021-dqwvg
This record's URL: https://www.repository.cam.ac.uk/handle/1810/331062
Statistics
Total file downloads (since January 2020). For more information on metrics see the
IRUS guide.
Recommended or similar items
The current recommendation prototype on the Apollo Repository will be turned off on 03 February 2023. Although the pilot has been fruitful for both parties, the service provider IKVA is focusing on horizon scanning products and so the recommender service can no longer be supported. We recognise the importance of recommender services in supporting research discovery and are evaluating offerings from other service providers. If you would like to offer feedback on this decision please contact us on: support@repository.cam.ac.uk