3 Keras Design Patterns Every ML Engineer Should Know



Original Source Here

Keras is a Machine Learning (ML) library built on top of TensorFlow, making it extremely easy to create and fit Deep Learning model architectures. As they like to say, “Keras API is designed for human beings, not machines.”

And that’s true! If you want to quickly sketch up an idea, Keras is the tool for the job. However, Keras can be very flexible and powerful; you can gradually peel the layers of complexity and dive deeper to customize almost every little detail.

Moreover, Keras can take advantage of multi-GPU configurations or TPU training because of its TensorFlow-backed backend.

In this story, we examine the three main Keras design patterns. We will first build a simple sequential model. Then we will expand our implementation to support non-sequential use-cases. Finally, we will dive deeper to create and execute our own training loop by subclassing the Keras Model class. Let’s start!

Learning Rate is a newsletter for those who are curious about the world of AI and MLOps. You’ll hear from me every Friday with updates and thoughts on the latest AI news and articles. Subscribe here!

Straight As an Arrow

To showcase the power of Keras, we need a running example. To this end, we will use the “Hello world” example of computer vision, the MNIST dataset. I know, I know… the MNIST problem is pretty boring nowadays, but we try to keep things simple here so we can focus on what matters: the code.

First, we will solve MNIST using a sequential model. This is as simple as it gets. The best way to follow this document from now on is to launch a Google Colab Notebook and change the runtime to a GPU-enabled one.

Inside the first code cell, provide the import statements:

Then, let’s load and prepare the data:

Now, we are ready to build our model. The Keras sequential API makes it effortless to build a Deep Learning model that works much like an open highway; we feed the data through a straight sequence of layers:

Finally, all you have to do is compile the model, fit it and evaluate it on the test dataset. Compile broadly means provide the loss function, the optimizer, and the metrics you care about. After this step, you are ready to start training:

That’s all! After 15 epochs, I get an accuracy of 0.9912 on the test dataset. Not bad, but I’m sure you are not surprised. After all, the MNIST task is considered to be a solved one.

The Functional API

But what if you cannot solve your problem using a straight sequence of layers? What if you need branches that perform separate calculations and the model merges their outcomes down the line? Then, it’s time to use the Keras functional API.

So we will take our input, feed it through two different convolutional layers with different characteristics, and merge their outputs down the line. No, for the MNIST problem, that’s not really necessary, but we will pretend it is.

To achieve all that, we only need to update the model creation step:

After 15 epochs, I get an accuracy of 0.9919 on the test dataset. However, you cannot really say that this model is better or worse. This is just a toy example!

Subclassing

Now it’s time to bring out the big guns. Some engineers need to know what is going on inside the fit function and have control of everything. Keras offers the subclassing API, with which you can create your own model classes and fully customize the training step.

First, we need to create our datasets:

Then, create our model. This is the same model we used during the sequential API section:

Next, we need to create our training and test steps:

Finally, let’s bring everything together and build our training loop:

After 15 epochs, I get an accuracy of 0.9910 on the test dataset. It’s a wrap!

Conclusion

Keras is a Machine Learning (ML) library built on top of TensorFlow, making it extremely easy to create and fit Deep Learning model architectures. As they like to say, “Keras API is designed for human beings, not machines.”

Keras provides three main ways to solve your ML tasks:

  • The sequential API
  • The functional API
  • The Keras subclassing method

With each technique, we get one step deeper into low-level TensorFlow. Usually, the sequential and functional APIs solve 99% of our problems, but a good ML engineer should be aware of every design pattern.

About the Author

My name is Dimitris Poulopoulos, and I’m a machine learning engineer working for Arrikto. I have designed and implemented AI and software solutions for major clients such as the European Commission, Eurostat, IMF, the European Central Bank, OECD, and IKEA.

If you are interested in reading more posts about Machine Learning, Deep Learning, Data Science, and DataOps, follow me on Medium, LinkedIn, or @james2pl on Twitter.

Opinions expressed are solely my own and do not express the views or opinions of my employer.

AI/ML

Trending AI/ML Article Identified & Digested via Granola by Ramsey Elbasheer; a Machine-Driven RSS Bot

%d bloggers like this: