Research

I work on two problems: 1) Developing a theory of how the brain works, and 2) building machines that work like the brain. I am fortunate to collaborate with excellent researchers at DeepMind and previously at Vicarious to grapple with these problems. Some of the results from our investigations are in the papers and manuscripts below.

Compositional generative models for perception

A generative vision model that trains with high data-efficiency breaks text based captchas. Science 2017

This generative model emphasized:
  • Top-down attention
  • Lateral & Feedback connections
  • Factorized representation of shape and appearance (contours and surfaces)
  • Border-ownership
  • Iterative inference that has feed-forward, feedback, lateral, and explaining away interactions.

From CAPTCHA to common sense: How brain can teach us about artificial intelligence. Frontiers 2020

Perspective paper discussing:
  • Connections between AGI, evolution and inductive biases.
  • Systematic way to learn relevant insights for AGI from neuroscience and cognitive science.

Learning Attention-controllable border-ownership for objectness and binding. Biorxiv: Introduces a model that tackles the binding problem.

Object-based Generative models for dynamics

Schema networks: Zero-shot transfer with a generative causal model of intuitive physics. ICML 2017

  • Schema representations that abstracient and generalize.
  • Partially based on ideas from Drescher's excellent book.
  • Also related to schemas in the prefrontal cortex.

Cognitive programs & Visual cognitive computer – a cognitive architecture to learn abstract concepts

Zero-shot task transfer on robots by learning concepts as cognitive programs. Science Robotics 2019

  • Brain as a biased computer with generative perception and dynamics, controllable top-down attention, and working memory
  • Concepts and abstractions are programs on this visual cognitive computer.

A Model of Fast Concept Inference with Object-Factorized Cognitive Programs. CogSci 2020

  • Object-factorized, sub-goaling based cognitive programs.

Episodic memory and cognitive maps

Clone-structured graph representations enable flexible learning and vicarious evaluatiton of cognitive maps. Nature Communications 2021

  • Hippocampus as structured higher-order sequence learning.
  • “Spatial” represenations emerge from ordinal sequence learning.
  • Explanations for wide array of hippocampal effects.

Memorize-Generalize: An online algorithm for learning cognitive maps. CCN 2019

  • Shows how to learn cognitive maps using fast episodic memory followed by slower consolidation

Data-efficient hand-eye coordination for visual servoing

DURableVS: Data-efficient Unsupervised Recalibrating Visual Servoing via online learning in a structured generative model. ICRA 2022

Cortical Microcircuits

A detailed mathematical theory of thalamic and cortical microcircuits based on inference in a generative vision model

Cognitive Maps and the hippocampus

Clone-structured graph representations enable flexible learning and vicarious evaluation of cognitive maps