|
|
 |
|
Adaptive multimodal natural language generation
Speech, prosody & corpus-based methods
Multimodality, cognition and evaluation
Publications
Workshops
Speech, prosody & corpus-based methods
This subproject is devoted to context-dependent speech synthesis and text-to-text natural language generation. The work on speech synthesis concentrates on improved prosody prediction (placement of pitch accent and prosodic boundaries of various kinds). The work on language generation concentrates on sentence fusion (e.g., to combine partial answers, perhaps from different QA engines, to obtain more complete and focused answers).For speech synthesis, the NeXTeNS text-to-speech system for Dutch was integrated in the IMIX demonstrator. NeXTeNS has been improved in many ways, e.g., the text preprocessing was made more robust against ill-formed input, and morphological analysis was added to improve grapheme-to-phoneme conversion. Exploiting the output from the Alpino parser (used for language analysis within IMIX) is in progress. In addition to the original plan, a talking head (RUTH) was ported from English to Dutch, interfaced with NeXTeNS, and added to the IMIX demonstrator. Marsi (2004) reports on evaluation results of prosody prediction for NeXTeNS, and Marsi & van Rooden (2007) present some experiments on the expression of (un)certainty using RUTH.
To facilitate text-to-text generation we have developed a special-purpose graphical annotation tool (Gadget) which enables us to manually align the dependency analyses. With this, we have created a first parallel monolingual (Dutch) corpus. On the basis of this corpus, we have developed a fully functional prototype of a sentence fusion module which takes two sentences as input, performs linguistic analysis, automatically aligns the nodes of the corresponding dependency graphs, classifies the semantic relations between aligned nodes, and generates new variants which are restatements, generalisations or specifications of the orginal input sentences. The tree alignment and semantic classification were evaluated on the corpus using cross-validation; the surface generation output was evaluated by human judges. First results from this research are described in Marsi and Krahmer (2005ab). Since aligning dependency trees is very similar to recognizing equivalence and entailment, we have used this technique to participate in the Second and Third Recognizing Textual Entailment Challenges (Marsi et al. 2006; 2007).
References:
- Bosma, W., E. Marsi, E. Krahmer and M. Theune (2011). Text-to-text generation for question answering. In A. van den Bosch and G. Bouma (eds.), Interactive Multi-modal Question-Answering, Springer Verlag Berlin-Heidelberg, pp. 117-145. © Springer
- Marsi, E. (2004) Optionality in Evaluating Prosody Prediction. Proceedings of 5th ISCA Speech Synthesis Research Workshop, Pittsburgh, USA, pp. 13-18.
- Marsi, E. and E. Krahmer (2005a) Explorations in Sentence Fusion. Proceedings of the 10th European Workshop on Natural Language Generation, 8-10 August 2005, Aberdeen, Scotland, pp. 109-117.
- Marsi, E. and E. Krahmer (2005b) Classification of semantic relations by humans and machines. Proceedings of the ACL 2005 Workshop on Empirical Modeling of Semantic Equivalence and Entailment, 20 June 2005, Ann Arbor, Michigan, pp. 1-6.
- Marsi, E., E. Krahmer, W. Bosma and M. Theune (2006). Normalized alignment of dependency trees for detecting textual entailment. Second PASCAL Recognizing Textual Entailment Challenge, April 10-12, 2006, Venice, Italy, pp. 56-61.
- Marsi, E. and van Rooden, F. (2007). Expressing uncertainty with a talking head in a multimodal question-answering system. In the Proceedings of the Workshop on Multimodal Output Generation (MOG 2007), CTIT Workshop Proceedings WP 07-01, 25-26 January 2007, Aberdeen, Scotland, pages 105-116.
- Marsi, E., E. Krahmer and W. Bosma (2007). Dependency-based paraphrasing for recognizing textual entailment. In the Proceedings of the ACL 2007 Workshop on Textual Entailment and Paraphrasing, 28-29 June 2007, Prague, Czech Republic, pages 83–88.