An HPSG approach to synchronous speech and deixis - Alahverdzhieva, Katya; Lascarides, Alex (Proceedings of the 18th International Conference on Head-Driven Phrase Structure Grammar, 2011)
The use of hand gestures to point at objects and individuals, or to navigate through landmarks on avirtually created map is ubiquitous in face-to-face conversation. We take this observation as astarting point, and we demonstrate that deictic gestures can be analysed on a par with speech byusing standard methods from constraint-based grammars such as HPSG. In particular, we use the formof the deictic signal, the form of the speech signal (including its prosodic marking) and theirrelative temporal performance to derive an integrated multimodal tree that maps to an integratedmultimodal meaning. The integration process is constrained via construction rules that rule outill-formed input. These rules are driven from an empirical corporal study which sheds light on theinteraction between speech and deictic gesture.