192.151 Introduction to Deep Learning

Contents

• Neural Networks
  • Association
  • Associationism
  • Connectionism
  • Bain’s Ideas
  • Connectionist Machine
  • McCulloch–Pitts Model
  • Neurons that fire together wire together
  • Hebbian Learning
  • Rosenblatt Perceptron
  • Perceptron
  • Multi-Layer Perceptron
    • MLPs are universal functions Machines
    • MLPs are universal classifiers
    • MLPs are universal function approximators
  • Karnaugh Map
• Learning the Network
  • Empirical Error
  • The Geometry of the Perceptron Update
  • Hidden Neurons Have No Teacher
  • Binary Error is Flat
  • Perceptrons Draw Hyperplanes
  • Neural Layers Do Not Draw One Boundary
  • Sigmoid Turns a Score into a Probability
  • Divergence Function
    • L2 Divergence
    • Kullback–Leibler Divergence
  • A Smooth Proxy Can Miss a Separator
  • One Learning Rate Has to Please Every Direction
    • A Hessian Eigenvalue Is Curvature in One Direction
  • Optimisation Method
    • First-order Optimisation Method
    • Second-order Method

Lectures

• Lecture 20.03.2026
• Lecture 27.03.2026
• Lecture 17.04.2026
• Lecture 24.04.2026
• Lecture 08.05.2026
• Lecture 22.05.2026
• Lecture 29.05.2026
• Lecture 05.06.2026
• Lecture 12.06.2026 (Part 1)
• Lecture 12.06.2026 (Part 2)
• Lecture 19.06.2026