Research Projects

In machine learning, as throughout computer science, there is a tradeoff between expressiveness and tractability. On the one hand, we need powerful model classes to capture the richness and complexity of the real world. On the other, we need inference in those models to remain tractable, otherwise their potential for widespread practical use is limited. Deep learning can induce powerful representations, with multiple layers of latent variables, but these models are generally intractable. We are developing new classes of similarly expressive but still tractable models, including sum-product networks and tractable Markov logic. These models capture both class-subclass and part-subpart structure in the domain, and are in some aspects more expressive than traditional graphical models like Bayesian networks and Markov random fields. Research includes designing representations, studying their properties, developing efficient algorithms for learning them, and applications to challenging problems in natural language understanding, vision, and other areas.

Our research focuses on two important aspects of the design of information: our theoretical work investigates principled estimation, and our more applied work explores solutions for color image processing. Estimation problems are common in engineering, from estimating the integrity of a pipeline to reconstructing an object based on transmitted signals. Our work in color image processing combines statistical signal processing with human vision science to design visual information for humans. Please check out our

In many domains, data now arrives faster than we are able to learn from it. To avoid wasting this data, we must switch from the traditional "one-shot" machine learning approach to systems that can mine continuous, high-volume, open-ended data streams as they arrive. We have identified a set of desiderata for such systems and developed an approach to building stream mining algorithms that satisfies all of them. The approach is based on explicitly minimizing the number of examples used in each learning step, while guaranteeing that user-defined targets for predictive performance are met.

How can a computer accumulate a massive body of knowledge? What will web search engines look like in 10 years?

To address these questions, the Open Information Extraction (Open IE) project has been developing a web-scale information extraction system that reads arbitrary text from any domain on the web, extracts meaningful information, and stores it in a unified knowledge base for efficient querying. In contrast to traditional information extraction, the Open IE paradigm attempts to overcome the knowledge acquisition bottleneck by extracting a large number of relations at once.

The Spoken Networks projects studies how real-world, face-to-face social behavior can be measured and modeled in ways that simultaneously protect privacy and provide new insight into the dynamics of human social behavior.

Hierarchical Matching Pursuit uses sparse coding to learn codebooks at each layer in an unsupervised way and then builds hierarchial feature representations from the learned codebooks. It achieves state-of-the-art results on many types of recognition tasks.

In this project we address joint object category, instance, and pose recognition in the context of rapid advances of RGB-D cameras that combine both visual and 3D shape information. The focus is on detection and classification of objects in indoor scenes, such as in domestic environments