Running Keras directly on TensorFlow

Keras is a high-level library/API for neural network, a.k.a. deep learning. You can write shorter, simpler code using Keras. At the time of writing, Keras can use one of TensorFlow, Theano, and CNTK as a backend of deep learning process.

From TensorFlow 1.4, Keras API became one of core APIs of TensorFlow. Therefore, you don’t need to install both Keras and TensorFlow if you have a plan to use only TensorFlow backend in Keras. In other words, you can run Keras in simple way with full GPU support if you have got nvidia-docker environment which is mentioned in my last blog post, “TensorFlow over docker with GPU support“

In this post, I’ll show you how to modify original Keras code to run on TensorFlow directly.

How to run Keras code in TensorFlow

Modify import

Summary: Replace keras to tensorflow.python.keras at every import directive.

Since the Keras module in TensorFlow is tf.keras, some of you may try to convert the following code:

1
2
from keras.models import Sequential
from keras.layers import Dense

to

1
2
3
import tensorflow as tf
from tf.keras.models import Sequential
from tf.keras.layers import Dense

But, this would not work. You may come across the following error:

1
ImportError: No module named tf.keras.models

Instead, the following works:

1
2
from tensorflow.python.keras.models import Sequential
from tensorflow.python.keras.layers import Dense

The reason is that the module location of tf.keras is tensorflow/python/keras/__init__.py as documented at https://www.tensorflow.org/api_docs/python/tf/keras. As the directory hierarchy, you should import tensorflow.python.keras, not tf.keras.

tf.keras may be used if you use Keras API with TensorFlow style such as follows:

1
2
3
import tensorflow as tf
model = tf.keras.models.Sequential()
model.add(tf.keras.layers.Dense(8, input_dim=11, activation='relu'))

Migrate Keras API

You may skip this section if you already have the program written in Keras 2 API.

Summary: Change nb_epoch to epochs in model.fit() method. And consult new API reference for APIs that cause Keyword argument not understood error, and fix the argument to new one.

Some of early Keras documents, tutorials and books are written in Keras 1 API which is currently deprecated. Since the TensorFlow implementation of Keras API only supports Keras API version 2, you should change old API to the new one if your code is written in Keras 1.

The summary of changes from Keras 1 to Keras 2 is mentioned in “Introducing Keras 2“. According to the article, while The APIs are significantly changed, Keras gives warning or error message to help migrate to Keras 2. It is true for original Keras, but false for TensorFlow implementation. For example, note the following code which is wrtten in Keras 1 API:

1
model.add(Dense(12, input_dim=11, init='uniform', activation='relu'))

The (original) Keras gives the following warning to help migrating to Keras 2:

1
UserWarning: Update your `Dense` call to the Keras 2 API: `Dense(12, activation="relu", kernel_initializer="uniform", input_dim=11)`

But, the Keras API of TensorFlow 1.6.0 gives the following error message:

1
TypeError: ('Keyword argument not understood:', 'init')

The most critical argument is nb_epoch in model.fit() since TensorFlow does not give any error messages. You should change nb_epoch to epochs. Without this, TensorFlow just run 1 epoch only.

Migration Example

Sample Keras 1 code

“A Gentle Introduction to Deep Learning using Keras” is a great introductory lecture about Keras. It is one hour lecture and easy to understand with explanatory example code in Jupyter notebook. You can get the iPython notebook and sample data from Section 1, Lecture 9.

The Keras code in the Lecture is written in Keras 1. So, you need to modify some python code to run it on TensorFlow directly.

Run TensorFlow usnig docker image

If you are not interested in docker, you can skip this section.

You can run TensorFlow instantly without installing Python, pip, pandas, TensorFlow, CUDA, cuDNN if you are using docker.

After download and unzip KerasDownload.zip, you can get Jupyter environment by running official TensorFlow docker image:

1
2
3
sudo docker run -it --rm \
  -v "$PWD":/notebooks -p 8888:8888 \
  gcr.io/tensorflow/tensorflow:latest-py3

If you want to use your nVidia graphics card and have installed nvidia-docker, you can also get full support by GPU by running docker:

1
2
3
sudo nvidia-docker run -it --rm \
  -v "$PWD":/notebooks -p 8888:8888 \
  gcr.io/tensorflow/tensorflow:latest-gpu-py3

Modify import

In “Step 1. Import our modules”, there is import directive to import Keras classes.

1
2
3
from keras.models import Sequential
from keras.layers import Dense
import numpy

You need to change this to:

1
2
3
from tensorflow.python.keras.models import Sequential
from tensorflow.python.keras.layers import Dense
import numpy

Modify Dense and fit

You may get an error in “Step 4. Build the Model” due to the argument init:

1
2
3
4
model = Sequential()
model.add(Dense(12, input_dim=11, init='uniform', activation='relu'))
model.add(Dense(8, init='uniform', activation='relu'))
model.add(Dense(1, init='uniform', activation='sigmoid'))

From Keras 2, init argument1 of Dense class is changed to kernel_initializer. (More precisely, init and weights are arranged to kernel_initializer and bias_initializer.) Therefore, you need to change the above code to:

1
2
3
4
model = Sequential()
model.add(Dense(12, input_dim=11, kernel_initializer='uniform', activation='relu'))
model.add(Dense(8, kernel_initializer='uniform', activation='relu'))
model.add(Dense(1, kernel_initializer='uniform', activation='sigmoid'))

In “Step 5. Fit the Model”, you should fix the following code:

1
model.fit(X, Y, nb_epoch=200, batch_size=30)

In Keras 2, this code works, but not correctly. This code runs just 1 epoch which is the default value of epochs. If you want to run 200 epochs, you should fix it as follows:

1
model.fit(X, Y, epochs=200, batch_size=30)

Conclusion

You don’t need to install Keras in addtion to TensorFlow if you decided to use TensorFlow only as a backend since TensorFlow provides Keras API with small modification. In this post, I have showed how to run Keras on TensorFlow without install Keras package. I have also showed to migrate Keras 1 to Keras 2 API.

References