Difference between a GRU and LSTM. Explaining with an example.
The key difference between GRU and LSTM is that GRU's bag has two gates that are reset and update while LSTM has three gates that are input, output, forget. GRU is less complex than LSTM because it has less number of gates.
If the dataset is small then GRU is preferred otherwise LSTM for the larger dataset.
GRU exposes the complete memory and hidden layers but LSTM doesn't.
import keras
from keras.models import Sequential
from keras.layers import GRU, LSTM
import numpy as np
We will define two different models and Add a GRU layer in one model and an LSTM layer in the other model.
# define model where GRU is also output layer
model_1 = Sequential()
model_1.add(GRU(1, input_shape=(20,1)))
model_1.compile(optimizer='adam', loss='mse')
# define model where LSTM is also output layer
model_2 = Sequential()
model_2.add(LSTM(1, input_shape=(50,1)))
model_2.compile(optimizer='adam', loss='mse')
We will define a sample array to run in both models.
# input time steps
y = np.array([[1, 2, 3, 4, 5, 6, 7, 8, 9, 10],
[9, 8, 7, 6, 5, 4, 3, 2, 1, 0],
[11, 12, 13, 14, 15, 16, 17, 18, 19, 20],
[21, 22, 23, 24, 25, 26, 27, 28, 29, 30],
[31, 32, 33, 34, 35, 36, 37, 38, 39, 40]]).reshape((5,10,1))
# make and show prediction
print(model_1.predict(y))
[[6.1044526e-01] [4.0416101e-01] [1.4171210e-02] [1.2617696e-04] [8.3446486e-07]]
# input time steps
y = np.array([[1, 2, 3, 4, 5, 6, 7, 8, 9, 10],
[9, 8, 7, 6, 5, 4, 3, 2, 1, 0],
[11, 12, 13, 14, 15, 16, 17, 18, 19, 20],
[21, 22, 23, 24, 25, 26, 27, 28, 29, 30],
[31, 32, 33, 34, 35, 36, 37, 38, 39, 40]]).reshape((5,10,1))
# make and show prediction
print(model_2.predict(y))
[[-1.9881524e-02] [-5.2695298e-01] [-3.5639611e-04] [-3.7144428e-06] [-2.5736982e-08]]