Speakers
Description
Studies comparing dictionary entries generated with AI with those of well-established dictionaries edited by lexicographers show that LLMs tend to perform better in some tasks (e.g. writing definitions) than in others (e.g. word-sense disambiguation (e.g. Nichols 2023, Lew 2023, Jakubíček & Rundell 2023, Rees & Lew 2024). One of the problems resulting from the latter is that of “false polysemy” (Jakubíček & Rundell 2023: 525), where the differences between senses listed under a headword are unclear.
Admittedly, the separation of meanings in dictionaries is artificially drawn by lexicographers, and there are often mismatches in sense distinctions across dictionaries. Yet it is still possible for experts to evaluate whether meaning boundaries are sufficiently clear-cut. What is less known is how learners react to false polysemy. Granted that people rarely read dictionary entries in full (Tono 1984, Nuccorini 1994, Bogaards 1998, Dziemianko 2016), and have been reported to stop reading once they find the information they need (Lew, Grzelak & Leszowicz 2013), we wanted to explore whether false polysemy disrupts the consultation process.
This study analysed how 98 L2-English undergraduate students reacted to false polysemy. They took an online quiz consisting of 20 unknown lexical items presented in the context of sentences selected from corpora, some of which were shortened or slightly edited to remove contextual clues. For each vocabulary test item, the participants were given two definitions copied from Reverso, a new English dictionary developed with the assistance of LLMs. Example sentences and sense indicators that could give additional cues about meaning were deliberately omitted. For half of the test items, the pair of definitions provided were indisputably different. For the other half, the definitions were not clearly distinct according to two independent experts (i.e., they were exemplars of AI-generated false polysemy). The test items were shown to the participants in a random order, and each time they were asked to select which of the two definitions (also randomly ordered) was a better fit. They were then asked to judge on a Likert scale how confident they were that they had selected the correct sense. We also recorded the time spent on each test item, the order of the definition selected (first or second), and whether it was correct (when senses were distinct). A sample of the participants was then interviewed to gain further insights into their reactions.
Preliminary results indicate that the participants had little difficulty selecting the correct sense in the true polysemy condition. However, when faced with false polysemy, their confidence dropped and they took longer to decide. Both effects were statistically significant. Our findings suggest that false polysemy can be detrimental to the user experience, and underscore the need for AI-powered systems that acknowledge and address the problem proactively, as recognized by the developers of Reverso, where human expertise, editorial guidelines and built-in feedback loops are key. That said, future user studies on false polysemy require naturalistic observations, as dictionary users may react differently when not explicitly asked to pick one out of two controlled definitions.
