Repository logo
 

Integrated deep visual and semantic attractor neural networks predict fMRI pattern-information along the ventral object processing pathway.

Simple item page
cam.issuedOnline2018-07-13
dc.contributor.authorTyler, LK
dc.contributor.authorClarke, A
dc.contributor.authorDevereux, B
dc.contributor.orcidTyler, Lorraine [0000-0002-9943-118X]
dc.contributor.orcidClarke, Alex [0000-0001-7768-5229]
dc.date.accessioned2018-10-10T17:30:17Z
dc.date.available2018-10-10T17:30:17Z
dc.date.issued2018-12-01
dc.description.abstractRecognising an object involves rapid visual processing and activation of semantic knowledge about the object, but how visual processing activates and interacts with semantic representations remains unclear. Cognitive neuroscience research has shown that while visual processing involves posterior regions along the ventral stream, object meaning involves more anterior regions, especially perirhinal cortex. Here we investigate visuo-semantic processing by combining a deep neural network model of vision with an attractor network model of semantics, such that visual information maps onto object meanings represented as activation patterns across features. In the combined model, concept activation is driven by visual input and co-occurrence of semantic features, consistent with neurocognitive accounts. We tested the model’s ability to explain fMRI data where participants named objects. Visual layers explained activation patterns in early visual cortex, whereas pattern-information in perirhinal cortex was best explained by later stages of the attractor network, when detailed semantic representations are activated. Posterior ventral temporal cortex was best explained by intermediate stages corresponding to initial semantic processing, when visual information has the greatest influence on the emerging semantic representation. These results provide proof of principle of how a mechanistic model of combined visuo-semantic processing can account for pattern-information in the ventral stream.
dc.description.sponsorshipThis research was funded by an Advanced Investigator grant to LKT from the European Research Council under the European Community's Seventh Framework Programme (FP7/2007-2013)/ ERC Grant agreement n° 249640, and an ERC Advanced Investigator grant to LKT under the Horizon 2020 Research and Innovation Programme (2014-2020 ERC Grant agreement no 669820).
dc.identifier.doi10.17863/CAM.30928
dc.identifier.eissn2045-2322
dc.identifier.issn2045-2322
dc.identifier.urihttps://www.repository.cam.ac.uk/handle/1810/283566
dc.language.isoeng
dc.publisherNature Publishing Group
dc.publisher.urlhttp://dx.doi.org/10.1038/s41598-018-28865-1
dc.rightsAttribution 4.0 International
dc.rights.urihttps://creativecommons.org/licenses/by/4.0/
dc.subjectBrain Mapping
dc.subjectFemale
dc.subjectHumans
dc.subjectImage Processing, Computer-Assisted
dc.subjectMagnetic Resonance Imaging
dc.subjectMale
dc.subjectModels, Neurological
dc.subjectPattern Recognition, Visual
dc.subjectPerirhinal Cortex
dc.subjectSemantics
dc.subjectVisual Cortex
dc.subjectVisual Pathways
dc.titleIntegrated deep visual and semantic attractor neural networks predict fMRI pattern-information along the ventral object processing pathway.
dc.typeArticle
dcterms.dateAccepted2018-06-18
prism.issueIdentifier8
prism.number10636
prism.publicationDate2018
prism.publicationNameScientific Reports
pubs.funder-project-idEuropean Research Council (249640)
pubs.funder-project-idEuropean Research Council (669820)
rioxxterms.licenseref.startdate2018-12-01
rioxxterms.licenseref.urihttp://www.rioxx.net/licenses/all-rights-reserved
rioxxterms.typeJournal Article/Review
rioxxterms.versionVoR
rioxxterms.versionofrecord10.1038/s41598-018-28865-1

Files

Original bundle
Now showing 1 - 1 of 1
Thumbnail Image
Name:
s41598-018-28865-1.pdf
Size:
2.57 MB
Format:
Adobe Portable Document Format
Description:
Published version
Licence
https://creativecommons.org/licenses/by/4.0/
Download
License bundle
Now showing 1 - 1 of 1
No Thumbnail Available
Name:
DepositLicenceAgreement.pdf
Size:
417.78 KB
Format:
Adobe Portable Document Format
Download

Collections

Cambridge University Research Outputs