Towards a formal distributional semantics
IWCS 2013 Workshop

March 19, 2013
IWCS home
Formal and distributional semantics: From romance to relationship
Louise McNally

When I first learned about distributional semantics, I got very excited because I am very concerned with a problem about which "traditional" formal semantics methods have little to say, namely how to make sense of the distinction, assuming one exists, between linguistic content and world knowledge, and how these come together in the resolution of the context-dependent dimension of semantic composition (for example, how we model the effect of adjectival modifiers on nouns). I also confess that I have a hard time buying into the idea of a logic-like "language of thought", and I thought distributional representa- tions, appropriately enriched, might ultimately lead to better approximations of the mental representation of semantic content than what we currently have. Since distributional semantic representations and the operations used to combine them lend themselves naturally to problems like word sense disambiguation, I was willing to set aside all my other concerns about thesere presentations (like how they would deal with function words) in the hopes of making a significant advance in my understanding of context-dependent meaning.
A few years and a few modeling experiences later (Boleda et al., 2012; Boleda et al., 2013), I have to say the romance is over: I don't expect to be abandoning formal semantics any time soon. However, I won't be abandoning distributional semantics, either. In this talk, I'll briefly discuss some of the things that I've learned that have most surprised and interested me, some of the obstacles that I consider most significant, and therefore most urgent, if the distributional semantics community is to attract the collaboration of formal semanticists, something that I think would greatly benefit both communities, and some problems that I think could be addressed now and that formal semanticists could profitably contribute to and learn from.


Boleda, G., E. M. Vecchi, M. Cornudella, and L. McNally (2012). First order vs. higher order modification in distributional semantics. In Proceedings of EMNLP, Jeju Island, Korea, pp. 1223-1233.

Boleda, G., M. Baroni, L. McNally, and N. T. Pham (2013). Intensionality was only alleged: On adjective-noun composition in distributional semantics. In Proceedings of IWCS 2013, Potsdam.

Sentence paraphrase detection: When determiners and word order make the difference
Nghia Pham, Raffaella Bernardi, Yao Zhong Zhang and Marco Baroni

Researchers working on distributional semantics have recently taken up the challenge of going beyond lexical meaning and tackle the issue of compositionality. Several Compositional Distributional Semantics Models (CDSMs) have been developed and promising results have been obtained in evaluations carried out against data sets of small phrases and as well as data sets of sentences. However, we believe there is the need to further develop good evaluation tasks that show whether CDSM truly capture compositionality. To this end, we present an evaluation task that highlights some differences among the CDSMs currently available by challenging them in detecting semantic differences caused by word order switch and by determiner replacements. We take as starting point simple intransitive and transitive sentences describing similar events, that we consider to be paraphrases of each other but not of the foil paraphrases we generate from them. Only the models sensitive to word order and determiner phrase meaning and their role in the sentence composition will not be captured into the foils' trap.

The Curious Case of Metonymic Verbs: A Distributional Characterization
Jason Utt, Alessandro Lenci, Sebastian Padó and Alessandra Zarcone

Logical metonymy combines an event-selecting verb with an entity-denoting noun (e.g., The writer began the novel), triggering a covert event interpretation (e.g., reading, writing). Experimental inves- tigations of logical metonymy must assume a binary distinction between metonymic (i.e. eventselect- ing) verbs and non-metonymic verbs to establish a control condition. However, this binary distinction (whether a verb is metonymic or not) is mostly made on intuitive grounds, which introduces a potential confounding factor. We describe a corpus-based approach which characterizes verbs in terms of their behavior at the syntax-semantics interface. The model assesses the extent to which transitive verbs prefer event-denoting objects over entity-denoting objects. We then test this "eventhood" measure on psycholinguistic datasets, showing that it can distinguish not only metonymic from non-metonymic verbs, but that it can also cap- ture more fine-grained distinctions among different classes of metonymic verbs, putting such distinctions into a new graded perspective.

Semantic transparency: challenges for distributional semantics
Melanie Bell and Martin Schäfer

Using data from Reddy et al. (2011), we present a series of regression models of semantic trans- parency in compound nouns. The results indicate that the frequencies of the compound constituents, the semantic relation between the constituents, and metaphorical shift of a constituent or of the compound as a whole, all contribute to the overall perceived level of transparency. While not proposing an actual dis- tributional model of transparency, we hypothesise that incorporating this information into such a model would improve its success and we suggest some ways this might be possible.

Can distributional approaches improve on Good Old-Fashioned Lexical Semantics?
Ann Copestake

In this position paper, I discuss some linguistic problems that computational work on lexical semantics has attempted to address in the past and the implications for alternative models which incorporate distributional information. I concentrate in particular on phenomena involving count/mass distinctions, where older approaches attempted to use lexical semantics in their models of syntax. I outline methods by which the earlier models allowed the transmission of information between lexical items (regular polysemy and inheritance) and address the possibility that similar techniques could usefully be incorporated into distributional models.

Combining logic-based and distributional representations for inferences over text
Katrin Erk

Distributional models have recently been suggested as an alternative to logic-based sentence semantics because of their ability to model word and phrase similarity and to model gradience in meaning, and because they can be learned automatically from data. For me, an important additional property is their ability to describe the nuances of meaning in context without recourse to sense lists. However, the model that we have proposed for sentence meaning is not a purely distributional one, but a combined distributional and logic-based representation, as we view the two approaches as complementary. Logic-based approaches have well-known mechanisms for representing function words and representing the semantics of long, complex expressions. But they typically have an impoverished representation of the meaning of content words, and cannot typically be used to talk about word and phrase similarity. The strengths and weaknesses of distributional models are exactly the opposite. In this talk, I will describe recent work on using a joint logical and distributional approach to judge sentence similarity within a project geared at deep language understanding. I will also mention first steps towards defining a more systematic connection between the two frameworks. Panel: Compositional distributional semantics Led by Marco Baroni Semantic composition has been the core operation studied by formal semanticists since at least Frege's seminal work, and recently compositionality has attracted much interest among distributional semanticists as well. The topic of how to achieve compositionality with distributional models constitutes thus a natural area of convergence (or clash) between the two fields. In this panel, we will try to assess what is the current state of development in the area of compositional semantics, what are the most important obstacles to be faced, and what are the implications of all this (if any) for the classic formal approach to compositionality.

Neues Palais, Universität Potsdam