Show simple item record

dc.contributor.authorVarol, G
dc.contributor.authorMomeni, L
dc.contributor.authorAlbanie, S
dc.contributor.authorAfouras, T
dc.contributor.authorZisserman, A
dc.date.accessioned2022-05-27T16:08:25Z
dc.date.available2022-05-27T16:08:25Z
dc.date.issued2022-06
dc.date.submitted2021-05-01
dc.identifier.issn0920-5691
dc.identifier.others11263-022-01589-6
dc.identifier.other1589
dc.identifier.urihttps://www.repository.cam.ac.uk/handle/1810/337554
dc.description.abstract<jats:title>Abstract</jats:title><jats:p>The focus of this work is <jats:italic>sign spotting</jats:italic>–given a video of an isolated sign, our task is to identify <jats:italic>whether</jats:italic> and <jats:italic>where</jats:italic> it has been signed in a continuous, co-articulated sign language video. To achieve this sign spotting task, we train a model using multiple types of available supervision by: (1) <jats:italic>watching</jats:italic> existing footage which is sparsely labelled using mouthing cues; (2) <jats:italic>reading</jats:italic> associated subtitles (readily available translations of the signed content) which provide additional <jats:italic>weak-supervision</jats:italic>; (3) <jats:italic>looking up</jats:italic> words (for which no co-articulated labelled examples are available) in visual sign language dictionaries to enable novel sign spotting. These three tasks are integrated into a unified learning framework using the principles of Noise Contrastive Estimation and Multiple Instance Learning. We validate the effectiveness of our approach on low-shot sign spotting benchmarks. In addition, we contribute a machine-readable British Sign Language (BSL) dictionary dataset of isolated signs, <jats:sc>BslDict</jats:sc>, to facilitate study of this task. The dataset, models and code are available at our project page. </jats:p>
dc.languageen
dc.publisherSpringer Science and Business Media LLC
dc.subjectArticle
dc.subjectSpecial issue on Advances in Computer Vision and Applications
dc.subjectSign language recognition
dc.subjectSign spotting
dc.subjectFew-shot learning
dc.titleScaling Up Sign Spotting Through Sign Language Dictionaries
dc.typeArticle
dc.date.updated2022-05-27T16:08:25Z
prism.endingPage1439
prism.issueIdentifier6
prism.publicationNameInternational Journal of Computer Vision
prism.startingPage1416
prism.volume130
dc.identifier.doi10.17863/CAM.84963
dcterms.dateAccepted2022-01-21
rioxxterms.versionofrecord10.1007/s11263-022-01589-6
rioxxterms.versionVoR
rioxxterms.licenseref.urihttp://creativecommons.org/licenses/by/4.0/
dc.contributor.orcidVarol, G [0000-0002-8438-6152]
dc.identifier.eissn1573-1405
cam.issuedOnline2022-04-05


Files in this item

Thumbnail
Thumbnail

This item appears in the following Collection(s)

Show simple item record