Regression Example with Keras in Python

   We can easily fit the regression data with Keras sequential model and predict the test data.  In this post, we'll briefly learn how to fit regression data with the Keras neural network API in Python. We'll check the model in both methods KerasRegressor wrapper and the sequential model itself. The tutorial covers:

• Preparing data
• Defining the model
• Fitting with KerasRegressor (accuracy check and visualizing the results)
• Fitting with the sequential model (accuracy check and visualizing the results)

We'll start by loading the required modules.

import random
import numpy as np
import matplotlib.pyplot as plt
from keras.models import Sequential
from keras.layers import Dense
from keras.wrappers.scikit_learn import KerasRegressor
from sklearn.metrics import mean_squared_error

Preparing data

First, we'll create a sample regression dataset for this tutorial.

random.seed(123)
def CreateDataset(N):
 a,b,c,y = [],[],[],[]
 for i in range(N):    
  aa = i/10+random.uniform(-4,3)
  bb = i/30+random.uniform(-4,4)
  cc = i/40+random.uniform(-3,3)-5
  yy = (aa+bb+cc/2)/3
  a.append([aa])
  b.append([bb])
  c.append([cc])
  y.append([yy])
 return np.hstack([a,b,c]), np.array(y)

N = 200
x,y = CreateDataset(N)
x_ax = range(N)
plt.plot(x_ax, x, 'o', label="feature", markersize=3)
plt.plot(x_ax, y, lw=1.5, color="red", label="y")
plt.legend()
plt.show()

Red-line is y output, and the remaining dots are features for x input.


Defining the model

Next, we'll build a keras sequential model to use in KerasRegressor wrapper.

def BuildModel():
 model = Sequential()
 model.add(Dense(128, input_dim=3,activation='relu')) 
 model.add(Dense(32, activation='relu')) 
 model.add(Dense(8,activation='relu')) 
 model.add(Dense(1,activation='linear'))
 model.compile(loss="mean_squared_error", optimizer="adam")   
 return model

BuildModel().summary()

_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
dense_13 (Dense)             (None, 128)               512       
_________________________________________________________________
dense_14 (Dense)             (None, 32)                4128      
_________________________________________________________________
dense_15 (Dense)             (None, 8)                 264       
_________________________________________________________________
dense_16 (Dense)             (None, 1)                 9         
=================================================================
Total params: 4,913
Trainable params: 4,913
Non-trainable params: 0
_________________________________________________________________

Fitting with KerasRegressor

We include the above model into KerasRegressor and fit model with x and y data. Then, we can predict x data.

regressor = KerasRegressor(build_fn=BuildModel,nb_epoch=100,batch_size=3)
regressor.fit(x,y) 

y_pred = regressor.predict(x)

We'll check a mean squared error rate

mse_krr = mean_squared_error(y, y_pred)
print(mse_krr)

0.15731915715966263 

Finally, we'll plot the results.

plt.plot(y, label="y-original")
plt.plot(y_pred, label="y-predicted")
plt.legend()
plt.show()

Fitting with keras sequential model

This time, we'll fit the model without a wrapper.

model = BuildModel()
model.fit(x, y, nb_epoch=100, verbose=False, shuffle=False)
y_krm = model.predict(x) 

We'll check a mean squared error rate.

mse_krm=mean_squared_error(y, y_krm)
print(mse_krm)

0.03334813391197139 

Finally, we'll plot the results.

plt.plot(y, label="y-original")
plt.plot(y_krm, label="y-predicted")
plt.legend()
plt.show()

   In this tutorial, we've briefly learned how to fit and predict regression data with Keras neural networks model in Python. Thank you for reading! The full source code is listed below.

import random
import numpy as np
import matplotlib.pyplot as plt
from keras.models import Sequential
from keras.layers import Dense
from keras.wrappers.scikit_learn import KerasRegressor
from sklearn.metrics import mean_squared_error

random.seed(123)
def CreateDataset(N):
 a,b,c,y = [],[],[],[]
 for i in range(N):    
  aa = i/10+random.uniform(-4,3)
  bb = i/30+random.uniform(-4,4)
  cc = i/40+random.uniform(-3,3)-5
  yy = (aa+bb+cc/2)/3
  a.append([aa])
  b.append([bb])
  c.append([cc])
  y.append([yy])
 return np.hstack([a,b,c]), np.array(y)

N = 200
x,y = CreateDataset(N)
x_ax = range(N)
plt.plot(x_ax, x, 'o', label="feature", markersize=3)
plt.plot(x_ax, y, lw=1.5, color="red", label="y")
plt.legend()
plt.show()

def BuildModel():
 model = Sequential()
 model.add(Dense(128, input_dim=3,activation='relu')) 
 model.add(Dense(32, activation='relu')) 
 model.add(Dense(8,activation='relu')) 
 model.add(Dense(1,activation='linear'))
 model.compile(loss="mean_squared_error", optimizer="adam")   
 return model

BuildModel().summary()
regressor = KerasRegressor(build_fn=BuildModel,nb_epoch=100,batch_size=3)
regressor.fit(x,y) 

y_pred = regressor.predict(x)
mse_krr = mean_squared_error(y, y_pred)
print(mse_krr)

plt.plot(y, label="y-original")
plt.plot(y_pred, label="y-predicted")
plt.legend()
plt.show()

model = BuildModel()
model.fit(x, y, nb_epoch=100, verbose=False, shuffle=False)
y_krm = model.predict(x)
mse_krm=mean_squared_error(y, y_krm)
print(mse_krm)

plt.plot(y, label="y-original")
plt.plot(y_krm, label="y-predicted")
plt.legend()
plt.show()