Research Projects
1. Visual category learning by toddlers provides new principles for teaching rapid generalization
(NSF grant BCS-1842817) with David Crandall
The crux of both human and machine learning is generalization: how can a learning system, biological or artificial, perform well not only on its training examples but also on novel examples and circumstances? One approach, widely used and well supported in both human and machine learning, is experience with many training examples. This solution avoids “overfitting” but is slow and incremental. However, in some cases of human learning, generalization requires minimal experience.
Evidence of rapid learning from few examples, often called “one-shot” or “few-shot,” is particularly well documented in learning visual objects as well as scientific and mathematical concepts. Incremental and one-shot learning have been discussed as distinct mechanisms, but there is growing interest in how one-shot learning might emerge out of prior incremental learning, an idea related to the broader concept of “learning to learn”.
The central idea explaining rapid learning from minimal examples is that deep representational principles allow the learner to represent novel examples for appropriate generalization. Thus, most research on one-shot learning – experimental and computational – focuses on the nature of these representations or on the learning machinery.
But if one-shot learning is learnable, then an additional core question concerns the kinds of experiences that teach an incremental learner to become a one-shot learner. This is our focus. Our main idea is that generalization depends on knowing the allowable and not allowable transformations, for example, the allowable transformations for different views of the same object, for membership in a category, for indicating the same (as in 3-1 and 1+1). We seek to:
characterize the transformations,
in time,
their active generation through behavior,
the underlying learning (and memory) mechanisms.
Designer Martino Gamber, from the 100 chairs
in 100 days project, 2017
2. Infants' self-generated visual statistics support object and category learning
(NIH-NICHD R01HD104624) with Jim Rehg, Chen Yu, and David Crandall
Human visual object recognition is characterized by two remarkable competencies. The first, Recognition, concerns the perception of an individual object as the same thing despite the variability in the 2D image of the object projected to the eye.
The second competency, Categorization, is the recognition of never-seen-before things as members of categories - as dogs, cups, chairs, and flowers. The field does not have a unified understanding of these two competencies nor their developmental origins. Between the period of 18 to 24 months, visual object Recognition and Categorization show marked advances. This project pursues the developmental ties between the two achievements as driven by toddlers’ extended visual experiences with individual instances of a category. Our hypothesis is that category learning (and one-shot learning) in humans derives from extended (and unlabeled) visual experiences with individual objects and not by learning about the many different objects in the same category.
3. The statistics of infant first-person visual experience
(NIH-NEI 1R01EY032897) with T. Rowan Candy, Jason Gold, and David Crandall
The cortical visual system first extracts localized visual features -- contrast, spatial frequency, edge orientations, chromatic content --and then, through a series of transformations, yields meaningful percepts of objects and scenes. We also know that the development of sensitivities to these foundational features develops and depends on visual experience. We ask what the natural statistics of these features are in infant visual experience?
It is frequently assumed that at the massive scale of daily life, the world presents the same visual statistics to everyone. Our work challenges this assumption. The images that fall on the retina depend on the spatial relation of the eyes to the world, a relation that changes with eye, head, postural, and other movements. Over the first year of life, infants’ motor abilities –eye, head, postural positions, and purposeful behaviors --change markedly and systematically.
The extant evidence from laboratory studies of infant preferential looking also indicates developmentally changing visual preferences, but we do not know how those play out in the complicated context of real-world vision.
If infants can control the scenes in front of them, these changing internal biases could also strongly influence the visual statistics. This project seeks to understand how motor development –changes in eye movements, head control, posture, manual actions ,and locomotion creates a developmentally expected curriculum of visual input.
Our empirical approach is to collect in-home, day-long head-camera recordings from infants as they go about their everyday lives, and to also collect in-lab, head-mounted eye-tracking from these same infants from 2 to 25 months of age.
