Fully-connected RNN with Keras layer_simple_rnn in R

  Fully-connected RNN can be implemented with layer_simple_rnn function in R. In keras documentation, the layer_simple_rnn function is explained as "fully-connected RNN where the output is to be fed back to input." In this tutorial, we'll learn how to use layer_simple_rnn in regression problem in R.
   This tutorial covers:

• Generating sample data
• Reshaping input data
• Building the model
• Predicting and plotting a result

   We'll start with loading the 'keras' library for R.

library(keras)

 Generating sample data

   We'll create sample dataset for this tutorial. Here, we'll create 'a' vector with the length of N as a regression data.

N <- 400
set.seed(123)
n <- seq(1:N)
a <- n/10+4*sin(n/10)+sample(-1:6,N,replace=T)+rnorm(N)

Reshaping input data

   Next, we'll create 'x' and 'y' training sequence data. Here, we apply window method with the size of 'step' value. The result (y value) comes after the sequence of window elements (x values), then the window shifts to the next element x and y values are collected and so on.

step <- 4   # step is a window size

To cover all elements in a vector, we'll add a 'step' into the last part of  'a' vector by replicating the last element.

a <- c(a, replicate(step, tail(a, 1)))

Creating x - input, and y - output data.

x <- NULL
y <- NULL

for(i in 1:N)
{
  s <- i-1+step
  x <- rbind(x,a[i:s])
  y <- rbind(y,a[s+1])
}

Input data should array type, so we'll reshape it.

X <- array(x, dim=c(N, step,1))

Building the model

Next, we'll create Keras sequential model, add layer_simple_rnn layers, and compile it with defined metrics.

model<- keras_model_sequential() %>% 
   layer_simple_rnn(units=128, input_shape=c(step, 1), activation="relu") %>% 
   layer_dense(units = 32) %>% 
   layer_dense(units = 1, activation = "linear")
 
model %>% compile(loss = 'mse',
                  optimizer = 'adam',
                  metrics = list("mean_absolute_error")
)
 
model %>% summary()
____________________________________________________________________________
Layer (type)                      Output Shape                  Param #     
============================================================================
simple_rnn_71 (SimpleRNN)         (None, 128)                   16640       
____________________________________________________________________________
dense_155 (Dense)                 (None, 32)                    4128        
____________________________________________________________________________
dense_156 (Dense)                 (None, 1)                     33          
============================================================================
Total params: 20,801
Trainable params: 20,801
Non-trainable params: 0
____________________________________________________________________________

Predicting and plotting the result

Next, we'll train the model with X and y input data, predict X data, and check the errors.

model %>% fit(X,y, epochs=50, batch_size=32, verbose=0)
 
y_pred <- model %>% predict(X)
 
scores <- model %>% evaluate(X, y, verbose = 0)
print(scores)
$loss
[1] 7.443566

$mean_absolute_error
[1] 2.189379

 Finally, we'll plot the results.

x_axes <- seq(1:length(y_pred)) plot(x_axes, y, type = "l", col = "red", lwd = 1) lines(x_axes, y_pred, col = "blue", lwd = 2) legend("topleft", legend = c("y-original", "y-predicted"), col = c("red", "blue"), lty = 1, cex = 0.8)

   In this tutorial, we've learned how to use layer_simple_rnn to predict regression data in R. Thank you for reading! The full source code is listed below.

library(keras)

N <- 400
set.seed(123)
n <- seq(1:N)
a <- n/10+4*sin(n/10)+sample(-1:6,N,replace=T)+rnorm(N)

step <- 4   # step is a window size

a <- c(a, replicate(step, tail(a, 1)))

x <- NULL
y <- NULL

for(i in 1:N)
{
  s <- i-1+step
  x <- rbind(x,a[i:s])
  y <- rbind(y,a[s+1])
}

X <- array(x, dim=c(N, step,1))

model<- keras_model_sequential() %>% 
   layer_simple_rnn(units=128, input_shape=c(step, 1), activation="relu") %>% 
   layer_dense(units = 32) %>% 
   layer_dense(units = 1, activation = "linear")
 
model %>% compile(loss = 'mse',
                  optimizer = 'adam',
                  metrics = list("mean_absolute_error")
)
 
model %>% summary()
 

model %>% fit(X,y, epochs=50, batch_size=32, verbose=0)
 
y_pred <- model %>% predict(X)
 
scores <- model %>% evaluate(X, y, verbose = 0)
print(scores)

x_axes <- seq(1:length(y_pred))
plot(x_axes, y, type = "l", col = "red", lwd = 1)
lines(x_axes, y_pred, col = "blue", lwd = 2)
legend("topleft", legend = c("y-original", "y-predicted"),
        col = c("red", "blue"), lty = 1, cex = 0.8)