Bridging languages through images with deep partial canonical correlation analysis
| dc.contributor.author | Rotman, G | |
| dc.contributor.author | Vulić, I | |
| dc.contributor.author | Reichart, R | |
| dc.date.accessioned | 2018-09-10T22:18:13Z | |
| dc.date.available | 2018-09-10T22:18:13Z | |
| dc.date.issued | 2018 | |
| dc.description.abstract | We present a deep neural network that leverages images to improve bilingual text embeddings. Relying on bilingual image tags and descriptions, our approach conditions text embedding induction on the shared visual information for both languages, producing highly correlated bilingual embeddings. In particular, we propose a novel model based on Partial Canonical Correlation Analysis (PCCA). While the original PCCA finds linear projections of two views in order to maximize their canonical correlation conditioned on a shared third variable, we introduce a non-linear Deep PCCA (DPCCA) model, and develop a new stochastic iterative algorithm for its optimization. We evaluate PCCA and DPCCA on multilingual word similarity and cross-lingual image description retrieval. Our models outperform a large variety of previous methods, despite not having access to any visual signal during test time inference. Our code and data are available at: https://github.com/rotmanguy/DPCCA} | |
| dc.identifier.doi | 10.17863/CAM.27464 | |
| dc.identifier.isbn | 9781948087322 | |
| dc.identifier.uri | https://www.repository.cam.ac.uk/handle/1810/280100 | |
| dc.language.iso | eng | |
| dc.publisher | Association for Computational Linguistics | |
| dc.publisher.url | http://dx.doi.org/10.18653/v1/p18-1084 | |
| dc.title | Bridging languages through images with deep partial canonical correlation analysis | |
| dc.type | Conference Object | |
| dcterms.dateAccepted | 2018-04-21 | |
| prism.endingPage | 921 | |
| prism.publicationDate | 2018 | |
| prism.publicationName | ACL 2018 - 56th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference (Long Papers) | |
| prism.startingPage | 910 | |
| prism.volume | 1 | |
| pubs.conference-finish-date | 2018-07 | |
| pubs.conference-name | Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) | |
| pubs.conference-start-date | 2018-07 | |
| pubs.funder-project-id | European Research Council (648909) | |
| rioxxterms.licenseref.startdate | 2018-01-01 | |
| rioxxterms.licenseref.uri | http://www.rioxx.net/licenses/all-rights-reserved | |
| rioxxterms.type | Conference Paper/Proceeding/Abstract | |
| rioxxterms.version | AM | |
| rioxxterms.versionofrecord | 10.18653/v1/p18-1084 |
Files
Original bundle
1 - 1 of 1
Loading...
- Name:
- bridging-languages-images.pdf
- Size:
- 1.12 MB
- Format:
- Adobe Portable Document Format
- Description:
- Accepted version
- Licence
- http://www.rioxx.net/licenses/all-rights-reserved
License bundle
1 - 1 of 1
No Thumbnail Available
- Name:
- DepositLicenceAgreementv2.1.pdf
- Size:
- 150.9 KB
- Format:
- Adobe Portable Document Format