everything.study
9.660

Computational Cognitive Science

3-0-9 units · Graduate

Introduction to computational theories of human cognition. Focuses on principles of inductive learning and inference, and the representation of knowledge. Computational frameworks include Bayesian and hierarchical Bayesian models, probabilistic graphical models, nonparametric statistical models and the Bayesian Occam's razor, sampling algorithms for approximate learning and inference, and probabilistic models defined over structured representations such as first-order logic, grammars, or relational schemas. Applications to understanding core aspects of cognition, such as concept learning and categorization, causal reasoning, theory formation, language acquisition, and social inference. Graduate students complete a final project.

Fall 2004

Topics

  • Introduction
  • Foundations of Inductive Learning
  • Knowledge Representation: Spaces, Trees, Features
  • Knowledge Representation: Language and Logic 1
  • Knowledge Representation: Language and Logic 2
  • Knowledge Representation: Great Debates 1
  • Knowledge Representation: Great Debates 2
  • Basic Bayesian Inference
  • Graphical Models and Bayes Nets
  • Simple Bayesian Learning 1
  • Simple Bayesian Learning 2
  • Probabilistic Models for Concept Learning and Categorization 1
  • Probabilistic Models for Concept Learning and Categorization 2
  • Unsupervised and Semi-supervised Learning
  • Non-parametric Classification: Exemplar Models and Neural Networks 1
  • Non-parametric Classification: Exemplar Models and Neural Networks 2
  • Controlling Complexity and Occam’s Razor 1
  • Controlling Complexity and Occam’s Razor 2
  • Intuitive Biology and the Role of Theories
  • Learning Domain Structures 1
  • Learning Domain Structures 2
  • Causal Learning
  • Causal Theories 1
  • Causal Theories 2
  • Project Presentations

Readings

  • AIMA is short for the recommended course text: Russell, Stuart, J., and Peter Norvig. Artificial Intelligence: A Modern Approach. 2nd ed. Upper Saddle River, NJ: Prentice Hall/Pearson Education, 2003. ISBN: 0137903952.
    Listed reading
  • Goodman, Nelson. “The New Riddle of Induction.” In Fact, Fiction, and Forecast . Cambridge, MA: Harvard University Press, 1983. ISBN: 0674290712.
    Listed reading
  • Fodor, Chomsky. “On the Impossibility of Acquiring more Powerful Structures,” and “The Inductivist Fallacy (including discussion).” In Language and Learning: The Debate between Jean Piaget and Noam Chomsky . Edited by Massimo Piattelli-Palmarini. Cambridge, MA: Harvard University Press, 1984, pp. 142-149. ISBN: 0674509412.
    Listed reading
  • ———. “The Inductivist Fallacy.” In Language and Learning: The Debate between Jean Piaget and Noam Chomsky . Edited by Massimo Piattelli-Palmarini. Cambridge, MA: Harvard University Press, 1984, pp. 259-269, including discussion. ISBN: 0674509412.
    Listed reading
  • Optional Readings
    Listed reading
  • Fodor, J. A., M. F. Garrett, E. C. T. Walker, and C. H. Parkes. “Against Definitions.” Cognition 8 (1980): 263-367.
    Listed reading
  • Laurence, Stephen, and Eric Margolis. “Radical Concept Nativism.” Cognition 86 (2002): 22-55.
    Listed reading
  • Berwick, R. C. “Learning From Positive-only Examples: The Subset Principle and Three Case Studies.” Machine Learning 2 (1986): 625- 645. (The section on phonology can be skipped. Just read the applications to conceptual hierarchies and syntax.)
    Listed reading
  • Bruner, Jerome S., Jacqueline J. Goodnow, and George Austin. A Study in Thinking . Somerset, NJ: Transaction Publishers, 1986. ISBN: 0887386563.
    Listed reading
  • Mitchell, Thomas M. Machine Learning . New York, NY: McGraw-Hill, 1997, chapter 2. ISBN: 0070428077.
    Listed reading
  • Feldman, J. “Minimization of Boolean Complexity in Human Concept Learning.” Nature 407 (2000): 630-633.
    Listed reading
  • Buy at MIT Press Kearns, Michael J., and Umesh V. Vasirani. An Introduction to Computational Learning Theory . Cambridge, MA: MIT Press , 1994, pp. 1-7. ISBN: 0262111934. ↗
    Listed reading
  • Winston, P. H., ed. “Learning Structural Descriptions from Examples.” In The Psychology of Computer Vision . New York, NY: McGaw-Hill, 1975, pp. 157-209. ISBN: 0070710481.
    Listed reading
  • Shepard, R. N. “Multidimensional Scaling, Tree-fitting, and Clustering.” Science 210 (1980): 390-398.
    Listed reading
  • Landaues, T. K., and S. T. Dumais. “A Solution to Plato’s Problem: The Latent Semantic Analysis Theory of the Acquisition, Induction, and Representation of Knowledge.” Psychological Review 104 (1997): 211-240.
    Listed reading
  • Goldstone, R. L, and J. Son. “Similarity.” In Cambridge Handbook of Thinking and Reasoning . Edited by K. Holyoak and R. Morrison. Cambridge, MA: Cambridge University Press, 2005, pp. 13-36.
    Listed reading
  • Chomsky, Noam. Syntactic Structures . Berlin, Germany: Walter De Gruyter, Inc., 1976, pp. 11-48. ISBN: 3110154129.
    Listed reading
  • Markman, Arthur. Knowledge Representation . Mahwah, NJ: Lawrence Erlbaum Associates, 1998, pp. 118-146, and 188-216. ISBN: 0805824413.
    Listed reading
  • Nowak, M. A., N. L. Komarova, and P. Niyogi. “Computational and Evolutionary Aspects of Language.” Nature 417 (2002): 611-617.
    Listed reading
  • Gentner, D., and A. B. Markman. “Structural Alignment in Analogy and Similarity.” American Psychologist 52, no. 1 (1997): 45-56.
    Listed reading
  • 1. Rosch, E. “Principles of Categorization.” In Cognition and Categorisation . Edited by E. Rosch and B. Lloyd. Hillsdale, NJ: Erlbaum, 1978, pp. 27-48.
    Listed reading
  • 2. Armstrong S. L., L. R. Gleitman, and H. Gleitman. “What Some Concepts Might Not Be.” Cognition 13, no. 3 (May 1983): 263-308.
    Listed reading
  • Buy at MIT Press 1. Pinker, S. “Why the Child Holded the Baby Rabbits: A Case Study in Language Acquisition.” In An Invitation to Cognitive Science: Language . Edited by L. Gleitman, and M. Liberman. 2nd ed. Vol. 1. Cambridge, MA: MIT Press , 1995. ISBN: 0262650444. ↗
    Listed reading
  • Buy at MIT Press 2. Rumelhart, D. E., and J. L. McClelland, eds. “On Learning the Past Tenses of English Verbs.” Chapter 18 in Parallel Distributed Processing: Explorations in the Microstructure of Cognition . Vol. 2. Cambridge, MA: MIT Press, 1987. ISBN: 0262631121. ↗
    Listed reading
  • 1. Collins, A. M., and M. R. Quillian. “Retrieval Time from Semantic Memory.” Journal of Verbal Learning and Verbal Behavior 8 (1969): 240-248.
    Listed reading
  • 2. McClelland, and Rogers. “The Parallel Distributed Processing Approach to Semantic Cognition.” Nature Reviews Neuroscience 4 (April 2003): 1-14.
    Listed reading
  • Buy at MIT Press Rumelhart, D. E., and J. L. McClelland, eds. “Schemata and Sequential Thought Processes in PDP Models.” Chapter 14 in Parallel Distributed Processing: Explorations in the Microstructure of Cognition . Vol. 2. Cambridge, MA: MIT Press , 1987. ISBN: 0262631121. ↗
    Listed reading
  • Goldstone, R. L., and A. Kersten. “Concepts and Categories.” In Comprehensive Handbook of Psychology . Edited by A. F. Healy, and R. W. Proctor. Vol. 4: Experimental Psychology. 2003, pp. 591-621.
    Listed reading
  • Paccanaro, A., and G. E. Hinton. “Learning Distributed Representations of Concepts Using Linear Relational Embedding.” Technical Report: GCNU TR 2000-002, March 2000.
    Listed reading
  • AIMA. Chapter 13.
    Listed reading
  • Jeffreys, and Berger. “Bayesian Occam’s Razor.” American Scientist 80 (1992): 64-72.
    Listed reading
  • Tversky, A., and D. Kahneman. “Judgement under Uncertainty: Heuristics and Biases.” Science 185 (1974): 1124-1130.
    Listed reading
  • Sivia. Bayesian Data Analysis . New York, NY: Oxford University Press, 1996, pp. 1-23. ISBN: 0198518897.
    Listed reading
  • Charniak. “Bayesian Networks without Tears.” AI Magazine 12, no. 4 (1991): 50-63.
    Listed reading
  • Buy at MIT Press McClelland, J. L., D. E. Rumelhart, andG. Hinton. “The Appeal of Parallel Distributed Processing.” In Parallel Distributed Processing: Explorations in the Microstructure of Cognition . Vol. 2. Edited by D. E. Rumelhart, and J. L. McClelland. Cambridge, MA: MIT Press , 1987. ISBN: 0262631121. ↗
    Listed reading
  • Johnson-Laird, P. N., and Fabien Savary. “Illusory Inferences about Probabilities.” Acta Psychologica 93 (1996): 69-90.
    Listed reading
  • Gelman, Carlin, Stern, and Rubin. “Hierarchical Models.” Chapter 5 in Bayesian Data Analysis . 2nd ed. London, UK: Chapman & Hall, 1995, pp. 117-131. ISBN: 0412039915.
    Listed reading
  • Griffiths, T. L., and M. Steyvers. “A Probabilistic Approach to Semantic Representation.” In Proceedings of the 24th Annual Conference of the Cognitive Science Society (2002).
    Listed reading
  • Review: Goodman, Nelson. “The New Riddle of Induction.” In Fact, Fiction, and Forecast . Cambridge, MA: Harvard University Press, 1983. ISBN: 0674290712.
    Listed reading
  • Mitchell, Thomas M. “Bayesian Learning.” Chapter 6 in Machine Learning . New York, NY: McGraw-Hill, 1997, sections 6.1-6.3. ISBN: 0070428077.
    Listed reading
  • Buy at MIT Press Tenenbaum, J. B. “Rules and Similarity in Concept Learning.” In Advances in Neural Information Processing Systems 12 . Edited by S. Solla, T. Leen, and K. Muller. Cambridge, MA: MIT Press , 2000, 59-65. ISBN: 0262194503. ↗
    Listed reading
  • Posner, and Keele. “On the Genesis of Abstract Ideas.” Journal of Experimental Psychology 77 (1968): 353-363.
    Listed reading
  • If necessary for background: Bishop, C. M. “Bayesian Classification.” Neural Networks for Pattern Recognition . New York, NY: Oxford University Press, 1995. ISBN: 0198538642.
    Listed reading
  • Fried, and Holyoak. “Induction of Category Distributions: A Framework for Classification Learning.” Journal of Experimental Psychology: Learning, Memory and Cognition 10 (1984): 234-257.
    Listed reading
  • Ghahramani, Z., and M. I. Jordan. “Learning from Incomplete Data.” MIT Center for Biological and Computational Learning Technical Report 108 (1994).
    Listed reading
  • Zhu, Xiaojin, Zoubin Ghahramani, and John Lafferty. “Semi-supervised Learning using Gaussian Fields and Harmonic Functions.” The Twentieth International Conference on Machine Learning ( ICML). Washington, DC: 2003.
    Listed reading
  • Nigam, Kamal, Andrew McCallum, Sebastian Thrun, and Tom Mitchell. “Text Classification from Labeled and Unlabeled Documents using EM.” Machine Learning 39, no. 2/3 (2000): 103-134.
    Listed reading
  • Seeger, M. “Learning with Labeled and Unlabeled Data.” University of Edinburgh Institute for Adaptive and Neural Computation Technical Report (2001).
    Listed reading
  • Nosofsky, R. M. “Optimal Performance and Exemplar Models of Classification.” In Rational Models of Cognition. Edited by M. Oaksford and N. Chater. New York, NY: Oxford University Press, 1998, pp. 219-247.
    Listed reading
  • Kruschke, J. K. “Human Category Learning: Implications for Backpropagation Models.” Connection Science 5, no. 1 (1993): 3-37.
    Listed reading
  • B. W. Silverman. “Maximum Penalized Likelihood Estimators.” In Density Estimation . London, UK: Chapman and Hall, 1986, pp. 110-119.
    Listed reading
  • Hinton, G. E., P. Dayan, B. J. Frey, and R. M. Neal. “The Wake-sleep Algorithm for Unsupervised Neural Networks.” Science 268 (1995): 1158-1160.
    Listed reading
  • Anderson, J. R. “The Adaptive Nature of Human Categorization.” Psychological Review 98, no. 3 (1991): 409-429.
    Listed reading
  • Smyth, P. “Model Selection for Probabilistic Clustering using Cross-validated Likelihood.” Statistics and Computing 10, no. 1 (2000): 63-72.
    Listed reading
  • MacKay, D. J. C. “Probable Networks and Plausible Predictions - A Review of Practical Bayesian Methods for Supervised Neural Networks.” Network: Comput. Neural Syst. 6 (1995): 469-505.
    Listed reading
  • Buy at MIT Press Murphy, Gregory L., and Douglas L. Medin. “The Role of Theories in Conceptual Coherence.” Chapter 19 in Concepts: Core Readings . Edited by E. Margolis, and S. Laurence. Cambridge, MA: MIT Press, July 1999, pp. 425-458. ISBN: 9780262631938. ↗
    Listed reading
  • Gelman, Susan A. The Essential Child . New York, NY: Oxford University Press, March 1, 2003, chapter 1, and 3, pp. 3-18, and 60-88. ISBN: 0195154061.
    Listed reading
  • Courville, Aaron C., Nathaniel D. Daw and David S. Touretzky. “Similarity and Discrimination in Classical Conditioning: A Latent Variable Account.” Neural Information Processing Systems Conference (2004).
    Listed reading
  • Rehder, Bob. “Essentialism as a Generative Theory of Classification.” To appear in Gopnik, A., and L. Schulz (Eds.) Causal learning: Psychology, philosophy, and computation . New York, NY: Oxford University Press.
    Listed reading
  • Pearl, Judea. Causality: Models, Reasoning, and Inference . Cambridge, MA: Cambridge University Press, 2000, chapter 1. ISBN: 0521773628.
    Listed reading
  • Buy at MIT Press Gregory F. Cooper. “An Overview of the Representation and Discovery of Causal Relationships using Bayesian Networks.” In Computation, Causation, and Discovery . Edited by Clark Glymour and Gregory F. Cooper. MIT Press , June 1999, pp. 3-62. ISBN: 0262571242. ↗
    Listed reading
  • Gopnik, Alison, and Laura Schulz. “Mechanisms of Theory Formation in Young Children.” Trends in Cognitive Sciences 8, no. 8 (August 2004): 371-377.
    Listed reading
  • Wellman, Henry M., and Susan A. Gelman. “Cognitive Development: Foundational Theories of Core Domains.” Annu Rev Psychol 43 (1992): 337-75.
    Listed reading
  • Tenenbaum J. B., and Griffiths. “The Place of Intuitive Theories in Rational Causal Inference.” To appear in Gopnik, A., and L. Schulz (Eds.) Causal learning: Psychology, philosophy, and computation . New York, NY: Oxford University Press.
    Listed reading
  • Charles Kemp, Thomas L. Griffiths, and Joshua B. Tenenbaum. Discovering Latent Classes in Relational Data. MIT Computer Science and Artificial Intelligence Laboratory. AI Memo 2004-019, September 2004.
    Listed reading
  • Rehder, Bob. “Essentialism as a Generative Theory of Classification_.”_ To appear in Gopnik, A., and L. Schulz (Eds.) Causal learning: Psychology, philosophy, and computation . New York, NY: Oxford University Press.
    Listed reading
  • Milch, Brian, Bhaskara Marthi, and Stuart Russell. “BLOG: Relational Modeling with Unknown Objects.” Workshop on Statistical Relational Learning and Its Connections to Other Fields. The Twenty-First International Conference on Machine Learning (ICML). Banff, Alberta: July 2004.
    Listed reading
  • Osherson, Daniel N., O. Wilkie, E. E. Smith, A. Lopez, and E. Shafir. “Category-Based Induction.” Psychological Review 97, no. 2 (1990): 185-200.
    Listed reading
  • Buy at MIT Press Atran, Scott. “Classifying Nature Across Cultures.” In Thinking: An Invitation to Cognitive Science . Edited by Edward E. Smith, Daniel N. Osherson, et. al. 2nd ed. Vol. 3. Cambridge, MA: MIT Press , 1995, pp. 131-174. ISBN: 0262150433. ↗
    Listed reading
  • Kemp, C., and J. B. Tenenbaum. “Theory-based Induction.” In Proceedings of the 25th Annual Conference of the Cognitive Science Society (2003).
    Listed reading
  • Optional Reading
    Listed reading
  • Kemp, C., T. L. Griffiths, S. Stromsten, and J. B. Tenenbaum. “Semi-supervised Learning with Trees.” In Advances in Neural Information Processing Systems (2003).
    Listed reading
  • Keil, Frank C. “Contraints on Knowledge and Cognitive Development.” Psychological Review 88, no. 3 (May 1981): 197-227.
    Listed reading
  • McClelland, and Rogers. “The Parallel Distributed Processing Approach to Semantic Cognition.” Nature Reviews Neuroscience 4 (April 2003): 1-14.
    Listed reading
  • Kemp, Charles, Amy Perfors, and Joshua B. Tenenbaum. Learning Domain Structures . Department of Brain and Cognitive Sciences, MIT. Internal Memo.
    Listed reading
  • Geman, Stuart, and E. Bienenstock. “Neural Networks and the Bias/Variance Dilemma.” Neural Computation 4 (1992): 1-58. [Especially pp. 46-48]
    Listed reading
  • Scholl, Brian J.,and Patrice D. “Perceptual Causality and Animacy.” Tremoulet Trends in Cognitive Sciences 4, no. 8 (August 2000): 299-309.
    Listed reading
  • Feldman, Jacob, and Patrice D. Tremoulet. The Computation of Intention . (Forthcoming)
    Listed reading
  • There is no single required text for this class. Russell, Stuart J., and Peter Norvig. Artificial Intelligence: A Modern Approach. 2nd ed. Upper Saddle River, N.J.: Prentice Hall/Pearson Education, 2003. ISBN: 0137903952, is strongly recommended as background reading on relevant formal models. Readings will consist of papers from the cognitive literature and background material from AIMA and several texts and tutorials in machine learning.
    Listed reading

Prerequisites & course information

Subject meets with 6.4120J, 9.66[J] Prereq: Permission of instructor G (Fall) 3-0-9 units

Sources