RNN Example with Keras SimpleRNN in Python

   Recurrent Neural Network models can be easily built in a Keras API.  In this tutorial, we'll learn how to build an RNN model with a keras SimpleRNN() layer. For more information about it, please refer to this link.
   The post covers:

• Generating sample dataset
• Preparing data (reshaping)
• Building a model with SimpleRNN
• Predicting and plotting results

We'll start by loading the required libraries.

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
from keras.models import Sequential
from keras.layers import Dense, SimpleRNN

Generating sample dataset

For this tutorial, we'll generate simple sequence data.

N = 1000    
Tp = 800    

t=np.arange(0,N)
x=np.sin(0.02*t)+2*np.random.rand(N)
df = pd.DataFrame(x)
df.head()

        0
0  0.224671
1  0.490856
2  0.567755
3  0.974433
4  0.805705 

plt.plot(df)
plt.show() 

Next, we'll split 'df' dataset into a training and test parts.

values=df.values
train,test = values[0:Tp,:], values[Tp:N,:]

Preparing data (reshaping)

RNN model requires a step value that contains n number of elements as an input sequence. Here, we define it as a 'step'. This is an important part of RNN so let's see an example:
x has the following sequence data.
x = [1,2,3,4,5,6,7,8,9,10]
for step=1, x input  and its y prediction become:
x  y
1  2
2  3
3  4
4  5
..
9  10
for step=3, x and y contain:
x         y
1,2,3   4
2,3,4   5
3,4,5   6
4,5,6   7
...
7,8,9   10

As you have noticed the sizes of x input and y output become different. We'll fix it by adding step size into the training and test data.

step = 4

# add step elements into train and test
test = np.append(test,np.repeat(test[-1,],step))
train = np.append(train,np.repeat(train[-1,],step))

Next, we'll convert test and train data into the matrix with step value as it has shown above example.

# convert into dataset matrix
def convertToMatrix(data, step):
 X, Y =[], []
 for i in range(len(data)-step):
  d=i+step  
  X.append(data[i:d,])
  Y.append(data[d,])
 return np.array(X), np.array(Y)

trainX,trainY =convertToMatrix(train,step)
testX,testY =convertToMatrix(test,step)

Finally, we'll reshape trainX and testX to fit with the Keras model. RNN model requires three-dimensional input data. You can see the shape of testX below.

trainX = np.reshape(trainX, (trainX.shape[0], 1, trainX.shape[1]))
testX = np.reshape(testX, (testX.shape[0], 1, testX.shape[1]))

trainX.shape
(800, 1, 4) 

Building a model with SimpleRNN

Next, we create the keras Sequential model.

# SimpleRNN model
model = Sequential()
model.add(SimpleRNN(units=32, input_shape=(1,step), activation="relu"))
model.add(Dense(8, activation="relu")) 
model.add(Dense(1))
model.compile(loss='mean_squared_error', optimizer='rmsprop')
model.summary()
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
simple_rnn_26 (SimpleRNN)    (None, 32)                1184      
_________________________________________________________________
dense_50 (Dense)             (None, 8)                 264       
_________________________________________________________________
dense_51 (Dense)             (None, 1)                 9         
=================================================================
Total params: 1,457
Trainable params: 1,457
Non-trainable params: 0
_________________________________________________________________

Predicting and plotting the result

We'll fit model with trainX data and predict testX data.

model.fit(trainX,trainY, epochs=100, batch_size=16, verbose=2)
trainPredict = model.predict(trainX)
testPredict= model.predict(testX)
predicted=np.concatenate((trainPredict,testPredict),axis=0)

Next, we'll check the loss

trainScore = model.evaluate(trainX, trainY, verbose=0)
print(trainScore)

0.35334232926368714 

Finally, we check the result in a plot. A vertical line in a plot identifies a splitting point between the training and the test part.

index = df.index.values
plt.plot(index,df)
plt.plot(index,predicted)
plt.axvline(df.index[Tp], c="r")
plt.show() 

    In this post, we've learned how to fit and predict sequence data with the keras SimpleRNN model. The full source code is listed below.
   Thank you for reading!

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
from keras.models import Sequential
from keras.layers import Dense, SimpleRNN

# convert into dataset matrix
def convertToMatrix(data, step):
 X, Y =[], []
 for i in range(len(data)-step):
  d=i+step  
  X.append(data[i:d,])
  Y.append(data[d,])
 return np.array(X), np.array(Y)

step = 4
N = 1000    
Tp = 800    

t=np.arange(0,N)
x=np.sin(0.02*t)+2*np.random.rand(N)
df = pd.DataFrame(x)
df.head()

plt.plot(df)
plt.show()

values=df.values
train,test = values[0:Tp,:], values[Tp:N,:]

# add step elements into train and test
test = np.append(test,np.repeat(test[-1,],step))
train = np.append(train,np.repeat(train[-1,],step))
 
trainX,trainY =convertToMatrix(train,step)
testX,testY =convertToMatrix(test,step)
trainX = np.reshape(trainX, (trainX.shape[0], 1, trainX.shape[1]))
testX = np.reshape(testX, (testX.shape[0], 1, testX.shape[1]))

model = Sequential()
model.add(SimpleRNN(units=32, input_shape=(1,step), activation="relu"))
model.add(Dense(8, activation="relu")) 
model.add(Dense(1))
model.compile(loss='mean_squared_error', optimizer='rmsprop')
model.summary()

model.fit(trainX,trainY, epochs=100, batch_size=16, verbose=2)
trainPredict = model.predict(trainX)
testPredict= model.predict(testX)
predicted=np.concatenate((trainPredict,testPredict),axis=0)

trainScore = model.evaluate(trainX, trainY, verbose=0)
print(trainScore)

index = df.index.values
plt.plot(index,df)
plt.plot(index,predicted)
plt.axvline(df.index[Tp], c="r")
plt.show()