Show simple item record

dc.contributor.authorJanz, Dave
dc.contributor.authorvan der Westhuizen, Jos
dc.contributor.authorPaige, Brooks
dc.contributor.authorKusner, Matt
dc.contributor.authorHernández-Lobato, José Miguel
dc.date.accessioned2019-01-18T00:30:52Z
dc.date.available2019-01-18T00:30:52Z
dc.date.issued2017-12-05
dc.identifier.urihttps://www.repository.cam.ac.uk/handle/1810/288158
dc.description.abstractDeep generative models have been successfully used to learn representations for high-dimensional discrete spaces by representing discrete objects as sequences and employing powerful sequence-based deep models. Unfortunately, these sequence-based models often produce invalid sequences: sequences which do not represent any underlying discrete structure; invalid sequences hinder the utility of such models. As a step towards solving this problem, we propose to learn a deep recurrent validator model, which can estimate whether a partial sequence can function as the beginning of a full, valid sequence. This validator provides insight as to how individual sequence elements influence the validity of the overall sequence, and can be used to constrain sequence based models to generate valid sequences — and thus faithfully model discrete objects. Our approach is inspired by reinforcement learning, where an oracle which can evaluate validity of complete sequences provides a sparse reward signal. We demonstrate its effectiveness as a generative model of Python 3 source code for mathematical expressions, and in improving the ability of a variational autoencoder trained on SMILES strings to decode valid molecular structures.
dc.titleLearning a Generative Model for Validity in Complex Discrete Structures
dc.typeConference Object
prism.publicationNameICLR Conference
dc.identifier.doi10.17863/CAM.35474
dcterms.dateAccepted2018-02-15
rioxxterms.versionofrecord10.17863/CAM.35474
rioxxterms.versionVoR
rioxxterms.licenseref.urihttp://www.rioxx.net/licenses/all-rights-reserved
rioxxterms.licenseref.startdate2018-02-15
rioxxterms.typeConference Paper/Proceeding/Abstract
pubs.funder-project-idAlan Turing Institute (AT/I00009/16)
cam.issuedOnline2018-02-26
pubs.conference-name6th International Conference on Learning Representations
pubs.conference-start-date2018-04-30
cam.orpheus.counter22
pubs.conference-finish-date2018-05-03


Files in this item

Thumbnail

This item appears in the following Collection(s)

Show simple item record