Simulation-based language understanding
Together with colleagues in the NTL group, I have been exploring the hypothesis that language exploits many of the same structures used for action, perception, imagination, memory and other neurally grounded processes. This idea brings together previous work in the group on embodied representations (image schemas, x-schemas, etc.) and the assumptions of constructional and cognitive approaches to grammar.
The main structures and processes of our model of language understanding (as described in Bergen and Chang, 2005) are shown above.
- Linguistic knowledge is represented using constructions (form-meaning mappings), which are expressed using the Embodied Construction Grammar formalism.
- The analysis process takes an utterance and draws on linguistic knowledge, world knowledge and the current communicative context to produce a semantic specification. The semspec may also undergo a resolution process to further specify any relevant referents.
- The semantic specification provides parameters for a dynamic simulation using active embodied structures; the meaning of the utterance consists of the simulation and the inferences it produces.
- John Bryant's dissertation work describes a robust, incremental probabilistic construction analyzer that can handle omissions.
- Eva Mok and I have proposed models of the situational and discourse context, necessary for contextual resolution as well as our models of language acquisition.
Simulation semanticsThe simulation-based approach to language understanding provides an attractive means of coping with the open-ended, context-dependent nature of language: constructions need only specify simulation parameters, allowing features of the current context and of richer embodied and world knowledge to influence the result of any particular simulation. A number of phenomena have been addressed:
- Srini Narayanan's work on modeling metaphor and aspect served as a starting point for simulative-based inference; see here for an overview of how x-schemas are used to model aspectual inference and coercion.
- Simulation semantics has been applied to model frame-based representation issues, as exemplified by the Commercial Transaction frame. In collaboration with members of the FrameNet project, we have shown how ECG can be used to bridge the gap between semantically tagged FrameNet corpora and the richer inferential mechanisms provided by simulation.
Representative publicationsSee also language understanding papers and aspect page.
- Bergen and Chang, 2005. Embodied Construction Grammar in Simulation-Based Language Understanding. [ref] ((The foundational ECG chapter, with overview of analysis and simulation.)
- Chang, Feldman and Narayanan, 2004. Structured Connectionist Models of Language, Cognition and Action. [ref] (Brief overview of NTL modeling approach.)
- Chang, Narayanan and Petruck 2002. Putting Frames in Perspective. [ref] (Short paper showing how ECG can link FrameNet frames to simulative inference)
- Chang, Gildea and Narayanan 1998. A Dynamic Model of Aspectual Composition. [ref]