ch1 slides

Prévia do material em texto

DEEP LEARNING IN PYTHON
Introduction to 
deep learning
Deep Learning in Python
Imagine you work for a bank
● You need to predict how many transactions each customer 
will make next year
Deep Learning in Python
Example as seen by linear regression
Age
Bank Balance
Retirement Status
…
Number of 
Transactions
Deep Learning in Python
Example as seen by linear regression
Predicted 
Transactions
Bank Balance
Retired
Not Retired
Model with no interactions
Predicted 
Transactions
Bank Balance
Retired
Not Retired
Model with interactions
Deep Learning in Python
Interactions
● Neural networks account for interactions really well 
● Deep learning uses especially powerful neural networks 
● Text 
● Images 
● Videos 
● Audio 
● Source code
Deep Learning in Python
Course structure
● First two chapters focus on conceptual knowledge 
● Debug and tune deep learning models on conventional 
prediction problems 
● Lay the foundation for progressing towards modern 
applications 
● This will pay off in the third and fourth chapters
Deep Learning in Python
Build deep learning models with keras
In [1]: import numpy as np 
In [2]: from keras.layers import Dense 
In [3]: from keras.models import Sequential 
In [4]: predictors = np.loadtxt('predictors_data.csv', delimiter=',') 
In [5]: n_cols = predictors.shape[1] 
In [6]: model = Sequential() 
 
In [7]: model.add(Dense(100, activation='relu', input_shape = (n_cols,))) 
In [8]: model.add(Dense(100, activation='relu') 
In [9]: model.add(Dense(1))
Deep Learning in Python
Deep learning models capture interactions
Age
Bank Balance
Retirement Status
…
Number of 
Transactions
Deep Learning in Python
Interactions in neural network
Age
Number of 
Transactions
Income
# Accounts
Input Layer
Hidden Layer
Output Layer
DEEP LEARNING IN PYTHON
Let’s practice!
DEEP LEARNING IN PYTHON
Forward 
propagation
Course Title
Bank transactions example
● Make predictions based on: 
● Number of children 
● Number of existing accounts
Deep Learning in Python
Forward propagation
3
2
1
1
-1
1
2
-1
Input Hidden Layer
Output
5
1
9 # Transactions
# Accounts
# Children
Deep Learning in Python
Forward propagation
3
2
1
1
-1
1
2
-1
Input Hidden Layer
Output
5
1
9
# Children
# Accounts
# Transactions
Deep Learning in Python
Forward propagation
3
2
1
1
-1
1
2
-1
Input Hidden Layer
Output
5
1
9
# Children
# Accounts
# Transactions
Deep Learning in Python
Forward propagation
3
2
1
1
-1
1
2
-1
Input Hidden Layer
Output
5
1
9
# Children
# Accounts
# Transactions
Course Title
Forward propagation
● Multiply - add process 
● Dot product 
● Forward propagation for one data point at a time 
● Output is the prediction for that data point
Deep Learning in Python
In [1]: import numpy as np 
In [2]: input_data = np.array([2, 3]) 
In [3]: weights = {'node_0': np.array([1, 1]), 
 ...: 'node_1': np.array([-1, 1]), 
 ...: 'output': np.array([2, -1])} 
In [4]: node_0_value = (input_data * weights['node_0']).sum() 
In [5]: node_1_value = (input_data * weights['node_1']).sum()
Forward propagation code
2
-1
Input Hidden Layer Output
3 1
5
1 2
-11
1
Deep Learning in Python
In [6]: hidden_layer_values = np.array([node_0_value, node_1_value]) 
In [7]: print(hidden_layer_values) 
[5, 1] 
In [8]: output = (hidden_layer_values * weights['output']).sum() 
In [9]: print(output) 
9
Forward propagation code
2
-1
Input Hidden Layer Output
3 1
5
9
1 2
-11
1
DEEP LEARNING IN PYTHON
Let’s practice!
DEEP LEARNING IN PYTHON
Activation 
functions
Deep Learning in Python
Linear vs Nonlinear Functions 
Linear Functions Nonlinear Functions
Deep Learning in Python
Activation functions
● Applied to node inputs to produce node output
Deep Learning in Python
Improving our neural network
3
2
1
1
-1
1
2
-1
Input Hidden Layer
Output
5
1
9
Deep Learning in Python
Activation functions
3
2
1
1
-1
1
2
-1
Input Hidden Layer
Output
tanh(2+3)
9
tanh(-2+3)
Deep Learning in Python
ReLU (Rectified Linear Activation)
Rectifier
Deep Learning in Python
Activation functions
In [1]: import numpy as np 
In [2]: input_data = np.array([-1, 2]) 
In [3]: weights = {'node_0': np.array([3, 3]), 
 ...: 'node_1': np.array([1, 5]), 
 ...: 'output': np.array([2, -1])} 
In [4]: node_0_input = (input_data * weights['node_0']).sum() 
In [5]: node_0_output = np.tanh(node_0_input) 
In [6]: node_1_input = (input_data * weights['node_1']).sum() 
In [7]: node_1_output = np.tanh(node_1_input) 
In [8]: hidden_layer_outputs = np.array([node_0_output, node_1_output]) 
In [9]: output = (hidden_layer_output * weights['output']).sum() 
In [10]: print(output) 
1.2382242525694254
DEEP LEARNING IN PYTHON
Let’s practice!
DEEP LEARNING IN PYTHON
Deeper networks
Deep Learning in Python
Multiple hidden layers
Age
5
3
2
4
4
-5
-1
1
2
2
-3
7
Calculate with ReLU Activation Function
5
Deep Learning in Python
Multiple hidden layers
Age
5
3
2
4
4
-5
-1
1
2
2
-3
7
Calculate with ReLU Activation Function
5
Deep Learning in Python
Multiple hidden layers
Age
5
3
2
4
4
-5
-1
1
2
2
-3
7
Calculate with ReLU Activation Function
5
Deep Learning in Python
Multiple hidden layers
Age
5
3
2
4
4
-5
-1
1
2
2
-3
7
Calculate with ReLU Activation Function
5
Deep Learning in Python
Multiple hidden layers
Age
5
3
2
4
4
-5
-1
1
2
2
-3
7
Calculate with ReLU Activation Function
5
Deep Learning in Python
Multiple hidden layers
Age
5
3
2
4
Calculate with ReLU Activation Function
5
Deep Learning in Python
Multiple hidden layers
Age
5
3
2
4
Calculate with ReLU Activation Function
5
Deep Learning in Python
Multiple hidden layers
Age
5
3
2
4
Calculate with ReLU Activation Function
5
Deep Learning in Python
Multiple hidden layers
Age
5
3
2
4
Calculate with ReLU Activation Function
5
26
Deep Learning in Python
Multiple hidden layers
Age
5
3
Calculate with ReLU Activation Function
5
26
4
-5
Deep Learning in Python
Multiple hidden layers
Age
5
3
Calculate with ReLU Activation Function
5
26
4
-5
Deep Learning in Python
Multiple hidden layers
Age
5
3
Calculate with ReLU Activation Function
5
26
4
-5
Deep Learning in Python
Multiple hidden layers
Age
5
3
Calculate with ReLU Activation Function
5
26
4
-5
0
2
4
-1
1
2
2
7
-3
Deep Learning in Python
Multiple hidden layers
Age
5
3
Calculate with ReLU Activation Function
5
26
4
-5
0
2
4
-1
1
2
2
7
-3
0
52
364
Deep Learning in Python
Representation learning
● Deep networks internally build representations of pa"erns in 
the data 
● Partially replace the need for feature engineering 
● Subsequent layers build increasingly sophisticatedrepresentations of raw data
Deep Learning in Python
Representation learning
Deep Learning in Python
Deep learning
● Modeler doesn’t need to specify the interactions 
● When you train the model, the neural network gets weights 
that find the relevant pa"erns to make be"er predictions
DEEP LEARNING IN PYTHON
Let’s practice!