Research Projects
1. Visual category learning by toddlers provides new principles for teaching rapid generalization
(NSF grant BCS-1842817) with David Crandall
The crux of both human and machine learning is generalization: how can a learning system, biological or artificial, perform well not only on its training examples but also on novel examples and circumstances? One approach, widely used and well supported in both human and machine learning, is experience with many training examples. This solution avoids “overfitting” but is slow and incremental. However, in some cases of human learning, generalization requires minimal experience.
Evidence of rapid learning from few examples, often called “one-shot” or “few-shot,” is particularly well documented in learning visual objects as well as scientific and mathematical concepts. Incremental and one-shot learning have been discussed as distinct mechanisms, but there is growing interest in how one-shot learning might emerge out of prior incremental learning, an idea related to the broader concept of “learning to learn”.
The central idea explaining rapid learning from minimal examples is that deep representational principles allow the learner to represent novel examples for appropriate generalization. Thus, most research on one-shot learning – experimental and computational – focuses on the nature of these representations or on the learning machinery.
But if one-shot learning is learnable, then an additional core question concerns the kinds of experiences that teach an incremental learner to become a one-shot learner. This is our focus. Our main idea is that generalization depends on knowing the allowable and not allowable transformations, for example, the allowable transformations for different views of the same object, for membership in a category, for indicating the same (as in 3-1 and 1+1). We seek to:
characterize the transformations,
in time,
their active generation through behavior,
the underlying learning (and memory) mechanisms.
Designer Martino Gamber, from the 100 chairs
in 100 days project, 2017
2. Infants' self-generated visual statistics support object and category learning
(NIH-NICHD R01HD104624) with Jim Rehg, Chen Yu, and David Crandall
Human visual object recognition is characterized by two remarkable competencies. The first, Recognition, concerns the perception of an individual object as the same thing despite the variability in the 2D image of the object projected to the eye.
The second competency, Categorization, is the recognition of never-seen-before things as members of categories - as dogs, cups, chairs, and flowers. The field does not have a unified understanding of these two competencies nor their developmental origins. Between the period of 18 to 24 months, visual object Recognition and Categorization show marked advances. This project pursues the developmental ties between the two achievements as driven by toddlers’ extended visual experiences with individual instances of a category. Our hypothesis is that category learning (and one-shot learning) in humans derives from extended (and unlabeled) visual experiences with individual objects and not by learning about the many different objects in the same category.
3. The statistics of infant first-person visual experience
(NIH-NEI 1R01EY032897) with T. Rowan Candy, Jason Gold, and David Crandall
The cortical visual system first extracts localized visual features -- contrast, spatial frequency, edge orientations, chromatic content --and then, through a series of transformations, yields meaningful percepts of objects and scenes. We also know that the development of sensitivities to these foundational features develops and depends on visual experience. We ask what the natural statistics of these features are in infant visual experience?
It is frequently assumed that at the massive scale of daily life, the world presents the same visual statistics to everyone. Our work challenges this assumption. The images that fall on the retina depend on the spatial relation of the eyes to the world, a relation that changes with eye, head, postural, and other movements. Over the first year of life, infants’ motor abilities –eye, head, postural positions, and purposeful behaviors --change markedly and systematically.
The extant evidence from laboratory studies of infant preferential looking also indicates developmentally changing visual preferences, but we do not know how those play out in the complicated context of real-world vision.
If infants can control the scenes in front of them, these changing internal biases could also strongly influence the visual statistics. This project seeks to understand how motor development –changes in eye movements, head control, posture, manual actions ,and locomotion creates a developmentally expected curriculum of visual input.
Our empirical approach is to collect in-home, day-long head-camera recordings from infants as they go about their everyday lives, and to also collect in-lab, head-mounted eye-tracking from these same infants from 2 to 25 months of age.
4. Shape in Language & Cognition
(ERC Synergy Grant – ID 101167183)
Shape is a prominent feature of the visual world and determines how we perceive and store information about objects, how we interact with them, and what uses they can be put to.
It is also richly encoded in the languages of the world by organizing object labels in terms of their shape (tall elongated objects, slim objects, round oval objects), as well as actions involving manipulating those objects, which are encoded in verbs. For instance, in Japanese, there is no word for zipping clothes, unlike English, but there are specific verbs for putting on clothes depending on their shape (e.g., pants).
Sign languages are a salient example of how shape organizes the lexicon and grammar of language, with the hands representing visuo-geometric properties of real-world referents. Importantly, around the second year of life, young children start attending to shape as a vital cue for acquiring the label of objects, a phenomenon known as “the shape bias”.
The shape bias develops at the intersection between spatial ability skills and language, especially vocabulary size. The dynamics of this relationship is largely understudied. In addition, children on atypical developmental trajectories tend to have problems in using the shape bias in word learning. Exactly what is causing these problems has not been identified.
The main goals of SHAPE are to map out the relationship between the visual perception of shape and its encoding across languages in the world and to identify the factors that constrain the observed cross-linguistic variation.
We are interested in the two-way relationship between spatial cognition and language across speakers in different cultures, across development, and the factors that conspire to produce difficulties in this domain in children on atypical developmental paths.
SHAPE combines the expertise and builds on the synergy of researchers in vision research, child development, language and cognition, sign language and neuro-diverse populations (autism and developmental language disorder) in 5 countries in Europe and the USA: Mila Vulchanova (NTNU, Norway), Linda Smith (Indiana University, USA), Pamela Perniss (University of Cologne, Germany), Frank Seifart (CNRS, France), Larissa Samuelson (UEA, UK) and Caroline Larson (University of Missouri, USA).
