Machine Learning
Looking for a Function
Speech RecognitionHandwritten RecongnitionWeather ForecastPlay video games
Machine Learning Framework
Models: Hypothesis Function SetTraining: Pick the best functionTesting: test the best function
Deep Learning
Production line: a very complex functionDeep Model: All function are learned from dataA Single Neuron: A Single Simple Function
Why Speed Matters
Traning time: Big data increases the training timeInference time: not patient to wait
Why Deeper is Better
More parametersUniversality Theorem
How to Apply Deep Learning
Input domain: Aggregate InformationOutput domain: Classification
Layer Output Relation
Neural Network Formulation
Chain Rule
we can get this:
Backpropagation
Backward Pass And Forward Pass
Momentum
考虑之前数据的惯性作用，避免无法突破 local minimum
