Visual Similarity in Adult Word Extension and Artificial Word Learning
Abstract
Visual similarity measured by a self-supervised visual encoder partially explains word generalization in adults and artificial learners, while the composition of teaching examples contributes additional structure beyond that distance.
When two objects share a name, how much does their visual similarity explain, and what do the teaching examples add? First, we reanalyzed trial-level data from a large adult word-extension study (Lewis & Frank, 2018), computing each test image's distance to the examples shown in the space of a frozen self-supervised visual encoder (DINOv2). Distance was associated with extension (log odds -3.23 per standard deviation; 520 people), and an exploratory post hoc analysis suggested that this association persisted within the taught domain after controlling for taxonomic level and teaching condition. Example composition also mattered: after three examples, adults extended a word to 1.1% of superordinate items when the examples were subordinate and to 77.5% when they were superordinate. Second, in an artificial learner using the same encoder, words joining two categories were learned less accurately the farther apart the categories were (84.7%, 65.4%, and 53.0% naming accuracy for near, mid, and far pairs). The encoder's distances correlated with human similarity and confusability measures from three datasets (rho = .501-.795), but this representational alignment did not translate into conclusive evidence that human-derived distances predict learner difficulty, and a second encoder trained on one child's experience reproduced the tier ordering but not the pair-level effect. The secondary human data were analyzed under an internal plan fixed before the confirmatory analyses (not externally preregistered). Visual similarity accounts for part of how generalization is organized in both systems, while the composition of examples contributes what the distance we used does not capture.
Community
When two objects share a name, how much does their visual similarity explain, and what do the teaching examples add? We reanalyze trial-level data from a large adult word-extension study using distances in a frozen DINOv2 encoder, and test an artificial word learner built on the same encoder. Datasets: https://huggingface.co/collections/MrPatoVera/lexical-consensus-6af31e7f9870b8e83e668c49
This is an automated message from the Librarian Bot. I found the following papers similar to this paper.
The following papers were recommended by the Semantic Scholar API
- Vision Transformers Learn Word Meaning from Images Alone (2026)
- Scaling cross-situational word learning on children's naturalistic experiences (2026)
- Similar Choices, Different Attention: Cross-Modal Associations in Humans and Vision-Language Models (2026)
- GAVAGAI: Referential Alignment for Infant-Scale Vision-Language Models via Balanced Optimal Transport (2026)
- Hierarchical Prompt Learning for Hyperbolic Vision-Language Models (2026)
- Reading Between the Layers: Cross-Modal Alignment and Representational Hierarchy in Vision Transformers (2026)
- Seeing Before Naming: Context-Grounded Learning of Novel Visual Concepts (2026)
Please give a thumbs up to this comment if you found it helpful!
If you want recommendations for any Paper on Hugging Face checkout this Space
You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: @librarian-bot recommend
Models citing this paper 0
No model linking this paper
Datasets citing this paper 2
MrPatoVera/lexical-consensus-data-cifar10h
Spaces citing this paper 0
No Space linking this paper