I am studying David Harwath and Jim Glass's paper from CSAIL, "Deep Multimodal Semantic Embeddings for Speech and Images." I discussed similar work in my 2011 paper "Experiments with Asa H" but rather than speech and images being the sole modalities I allowed for input from a number of Asa's various senses. (The paper is available on my website www.robert-w-jones.com under "book", chapter 1, page 13.)
Such merging of multiple modalities may explain why it has not been possible to discover a formal grammar that covers human language.
No comments:
Post a Comment