My research interests fall between machine learning, signal processing, and psychophysics, particularly as they pertain to sound perception. I use the methodology and experimental evidence from these fields to better understand human and machine learning of perceptual data. I make use of physiological and psychophysical data for two ends: to develop predictive computational models of human perception and to inform the design of novel methods for machine perception.
- Current Projects
- A computational model of acquisition and consolidation: 2009-now
- Learning sounds from mixtures of audio: 2007-now
- Past Projects
Current Projects
A computational model of acquisition and consolidation: 2009-now
Human learning goes through a period where information is volatile: learning on one task can be disrupted by a second task if it is practiced shortly after the first (McGaugh 2000, Dudai 1996). This has been interpreted as evidence for a stage of learning called consolidation in which memories move from short to long term memory. Recent studies have suggested that a period called acquisition, occurring prior to consolidation, is functionally distinct from consolidation. In Wright et al. (2009), the tasks interfering with learning during acquisition were mutually exclusive to those interfering during consolidation. This suggests that acquisition is a distinct stage of learning (also see Zach et al. 2005). The goal of this project is to develop a computational model of acquisition and consolidation that can predict these observed interference patterns and to use this model to design future behavioral experiments.
My interest in acquisition and consolidation is grounded in the stability-plasticity dilemma. As learners of many tasks and behaviors, humans and other animals are faced with the challenge of managing all the information that comes their way. How can learning effectively occur in an environment so saturated with information? On the one hand, learners must retain the information they have learned form prior experience (stability), and on the other hand learners must adapt and change in response to new information (plasticity). Addressing this tradeoff between plasticity and stability is a fundamental challenge faced by anything that learns across a lifetime of experience (Grossberg 1988, Abraham and Robins 2005).
My work differs from past computational models of perceptual learning (e.g. Poggio et al. 1992, Petrov et al. 2005, Jacobs 2009) in that these efforts have focused on modeling the mechanisms of a single task, whereas my model necessarily considers constraints across a number of perceptual tasks. I am also unaware of any work but my own modeling auditory perceptual learning tasks.
My goal is for this model to be useful to researchers in several ways: first, since it takes only a short time to run an experiment, the model can be used to search for interesting and/or more effective training regimens, that would take months or years to test on humans; second, such experiments can be used to produce novel testable predictions of the model which can then be confirmed or denied by empirical evidence; and finally, such empirical evidence can be used to further modify the model, allowing results across many experiments to be integrated into a single system, which, in addition to providing a more coherent hypothesis, can in turn lead to novel predictions. The model may also provide benifits to the machine learning community by suggesting effective methods for inductive transfer learning (also called lifelong learning) (e.g. Silver and Bennett 2008).
Related Papers
- D. Little, B. Pardo. Computational Models of Perceptual Learning Across Multiple Auditory Tasks: Modeling Daily Learning Limits as Memory Decay, in Proceedings of the 10th International Conference on Cognitive Modeling, Philadelphia, PA, USA, August 5-8, 2010.
Learning sounds from mixtures of audio: 2007-now
More information about this project can also be found at my lab's website.
When a squirrel sees another squirrel eaten by a hawk, how does it then come to realize that the cry of a hawk that came before is associated with great danger? There are many sounds in the squirrels' environment: the sounds of other squirrels, the sound of passing traffic nearby, the sounds of other birds in the trees. Two thirds of young infants, in an informal survey from Barker and Newman (2004), were frequently in an environment where multiple people were talking, when learning their native language. Yet somehow infants learn to recognize the sounds of their native language, and are more capable of separating out their own name (Newman, 2005) and the voice of their own mother (Barker and Newman, 2004), than other sounds, from a mixture of other sounds. How can these links between a sound to an important label be learned in an environment with many concurrent sounds?
Broadly, my goal in this project is to understand how machine systems can learn to recognize sounds from mixtures of sound, and to understand how particular biological observations might help facilitate this. More specifically, I would like to design a machine system capable of learning to label sounds from mixtures of sounds, using only weak labels. A weak label is much like the label the squirrel or the infant gets: all the learner knows is that something, somewhere around a particular time is important, and the learner must figure out how to appropriately assign that label to future sounds. There may be multiple sounds during learning, which may overlap (in time) with the sound the label is associated with (which I call the target).
For broader society, the ability to learn from mixtures of sounds could be used to teach a system particular sounds and instruments that a music fan enjoys: new songs which the user had not heard that had the same kinds of sounds could then be found. It might be used to train an aid for the deaf, which could identify dangerous sounds in a street-scape, such as a passing truck, the sound of screeching breaks, or a police siren. Such an ability could be used to facilitate monitoring of an environment for particular rare species (Charif, 2009), to track certain individuals (Terry et al., 2005), or to provide a measure of bio-diversity (Chesmore, 2004).
You can find examples of the sounds one version of my system was evaluated with, as described in my ISMIR, 2008 paper, here
Related Papers
- D. Little, B. Pardo, Learning musical instruments from mixtures of audio with weak labels. 9th International Conference on Music Information Retrieval (ISMIR) September 14th-18th, 2008, Philadelphia, USA.
Past Projects
Online learning for Query-by-Humming: 2006-2007
During my first year as a graduate student I worked on a project called VocalSearch, now named Tunebot. VocalSearch allowed a user to search for music by singing melodies into a microphone (also called Query-by-Humming). I showed that the system can improve results for particular users, after it has been deployed, based on feedback from the user. This feedback customizes the parameters of how notes are identified and compared in a person's sung melody.
Related Papers
- D. Little, D. Raffensperger, B. Pardo, A Query By Humming System that Learns from Experience, 8th International Conference on Music Information Retrieval (ISMIR) September 23-27, 2007, Vienna, Austria. (PDF)
- D. Little, D. Raffensperger, B. Pardo, User-specific Training for a Music Search Engine, 4th Joint Workshop on Multimodal Interaction and Related Machine Learning Algorithms, June 28-30, 2007, Brno, Czech Republic. (PDF)
- D. Little, D. Raffensperger, and B. Pardo. Machine Learning and Mutlimodal Interaction: Fourth International Workshop, MLMI 2007, Brno, CZ, June 28-30, 2007, Revised Selected Papers, chapter User specific training of a music search engine. Lecture Notes in Computer Science. Springer, 2007.
Hexagonal Metamorphic Robot Algorithms: 2003-2005
As an undergraduate, I worked for three years with Jennifer Walter at Vassar College developing algorithms for the rearrangement of hexagonal robots. The purpose of this research is to design a set of reconfigurable modular hexagonal robots, which would enable an adaptive system able to rearrange itself for various tasks in hostile environments. The work I did was largely theoretical: I looked at how to move a large collection of these robots efficiently and without error around various kinds of obstacles.
Related Papers
- D. Little and J. Walter, Using Hexagonal Metamorphic Robots to Form Temporary Bridges, in Proc. of the IEEE International Conference on Intelligent Robotic Systems , Aug. 2005, Edmonton, Alberta, Canada, pages 2652-2657.
- J. Walter, M. Brooks, D. Little, and N. Amato, Enveloping Multi-Pocket Obstacles with Hexagonal Metamorphic Robots, in Proc. of the IEEE Intl. Conf. on Robotics and Automation, Apr. 2004, New Orleans, LA, pages 2204-2209.
- J. Walter and D. Little, Bridging Gaps in Traversal Surfaces with Hexagonal Metamorphic Robots, in Proc. of the American Nuclear Society 10th International Conference on Robotics and Remote Systems for Hazardous Environments, 28-31 March, 2004, Gainsville, FL.
References
- Wickliffe C. Abraham and Anthony Robins. Memory retention - the synaptic stability versus plasticity dilemma. Trends in Neurosciences, 28(2):73–78, 2 2005.
- Brittan A. Barker and Rochelle S. Newman. Listen to your mother! the role of talker familiarity in infant streaming. Cognition, 94 (2): B45 - B53, 2004. ISSN 0010-0277. doi: 10.1016/j.cognition.2004.06.001.
- Russ Charif. Ivory-billed woodpecker search, 2009.
- D. Chesmore. Automated bioacoustic identification of species. Anais da Academia Brasileira de Ciências, 76: 436-440, 2004.
- Yadin Dudai. Consolidation: Fragility on the road to the engram. Neuron, 17(3): 367–370, 1996.
- S. Grossberg. Competitive learning: From interactive activation to adaptive resonance. In Connectionist models and their implications: readings from cognitive science. RA Jacobs. Adaptive precision pooling of model neuron activities predicts the efficiency of human visual learning. Journal of Vision, 9(4):22, 2009.
- James L. McGaugh. Memory–a century of consolidation. Science(5451): 248–251, 1 2000.
- R.S. Newman. The cocktail party effect in infants revisited: Listening to one's name in noise. Developmental psychology, 41 (2): 352-362, 2005.
- A.A. Petrov, B.A. Dosher, and Z. Lu. The Dynamics of Perceptual Learning: An Incremental Reweighting Model. Psychological Review, 112(4):715, 2005.
- Tomaso Poggio, Manfred Fahle, and Shimon Edelman. Fast perceptual learning in visual hyperacuity. Science, 256(5059):1018–1021, 1992.
- Daniel Silver and Kristin Bennett. Guest editor’s introduction: special issue on inductive transfer learning. Machine Learning, 73(3):215–220, 12 2008.
- Andrew Terry, Tom Peake, and Peter McGregor. The role of vocal individuality in conservation. Frontiers in Zoology, 2 (1), 2005.
- B. A. Wright, Karen Banai, Andrew T. Sabin, and Yuxuan Zhang. Distinct phases of auditory learning identified by differences in vulnerability to intervening events. Poster Presentation at ARO 32nd MidWinter Meeting, February 14–19 2009.
- N. Zach, N. Kanarek, D. Inbar, Y. Grinvald, T. Milestein, and E. Vaadia. Segregation between acquisition and long-term memory in sensorimotor learning. European Journal of Neuroscience, 22(9):2357, 2005.