ML with Intel OpenVINO Toolkit — Image Inpainting [Part 1]

Original Source Here

ML with Intel OpenVINO Toolkit — Image Inpainting [Part 1]

Learn to perform image inpainting in the first installment of a three-part series on the Intel OpenVINO toolkit

Photo by Christian Englmeier on Unsplash

Photos play an essential role in how we interact with others in our day-to-day lives. But sometimes photos may become damaged; maybe they’ve deteriorated with time or maybe they’ve been edited poorly. Maybe you’re looking to digitize an old photo album, but some of the photos have physical damage, likes spots or scratches.

This is where image inpainting comes in. Inpainting is the process of reconstructing lost or deteriorated parts of images and videos. Traditionally, inpainting has been a lengthly process, undertaken exclusively by professional restorers. However, with modern deep neural networks achieving state-of-the-art results in inpainting tasks, this process can now be automated and is available to anyone with access to a computer and a digital image they want to restore.

Applications of Image Inpainting

Image inpainting has a lot of use-cases, from restoration of historical artwork, to applications in the digital photography universe. Here’s a couple of applications in the real world:

  • Museums use image inpainting to restore old photographs, paintings, or other pieces of art that may be damaged or worn.
  • The photography and cinema industry use image inpainting to restore old film reels, or replace the lost blocks in the coding and transmission of images.
  • Creative agencies use image inpainting in the photo/video editing processes and speed up delivery times.

What is OpenVINO?

OpenVINO is an open-source toolkit developed and maintained by Intel that facilitates the optimization and deployment of a deep learning model using an inference engine onto Intel hardware. It allows developers to use models trained with popular frameworks like TensorFlow, PyTorch, Keras, and more, while boosting deep learning performance in computer vision, automatic speech recognition, natural language processing, and other common tasks. Check out their official documentation for more information about OpenVINO and how it works.

OpenVINO has an open_model_zoo repository that contains optimized, pre-trained deep learning models that developers can use easily. For the inpainting task, we are going to use a generative multi-column convolution neural network (GMCNN) that was trained on the Places2 dataset with free-form masks. It aims to estimate suitable pixel information to fill holes in images. The model zoo has an implementation of such a model using TensorFlow and that’s what we are going to use.

The GMCNN implementation takes an image input of shape [1x3x512x680] that represents the batch size, number of channels, image height, and width. The expected color order is BGR. The output is the restored image of the shape [1, 512, 680, 3], or one 512 x 680 BGR image.


Make sure you have Python 3.7, and CMAKE installed in your development environment, then install OpenVINO. Refer to this tutorial to get started.

NB: You will use python version 3.7 because the model used in this tutorial leverages TensorFlow version 1.4 (or later versions, excluding 2.x) with NVIDIA GPU or CPU. Also, note the directory where OpenVINO is installed.

Download the model

Use OpenVINO’s model downloader to download the required models and convert the downloaded model into OpenVINO IR models with Model Converter.

(Linux) python3 $INTEL_OPENVINO_DIR/deployment_tools/tools/model_downloader/ — list models.lstpython3 $INTEL_OPENVINO_DIR/deployment_tools/tools/model_downloader/ — list models.lst(Win10) python “%INTEL_OPENVINO_DIR%\deployment_tools\tools\model_downloader\” — list models.lstpython “%INTEL_OPENVINO_DIR%\deployment_tools\tools\model_downloader\” — list models.lst

Run the demo app

You can run the model on your command line using the script and passing the image you would like restored. Below is the command to achieve this, as well as the results:

(Linux) python3 $INTEL_OPENVINO_DIR/deployment_tools/open_model_zoo/demos/image_inpainting_demo/python/ --input <input_image_file> --output <output_image_file>(Win10) python “%INTEL_OPENVINO_DIR%\deployment_tools\open_model_zoo\demos\image_inpainting_demo\python\” --input <input_image_file> --output <output_image_file>
Model input and output


Image inpainting is a useful tool in image editing and restoration, and the use of deep learning technologies can accelerate these tasks. Deep learning models designed for this particular task allow users to inpaint photos on their own personal computer, but this process can also scale to operate on huge data sets of images. Additionally, developers can integrate this technology with Flask to create AI-powered APIs for their multimedia apps.

Editor’s Note: Heartbeat is a contributor-driven online publication and community dedicated to providing premier educational resources for data science, machine learning, and deep learning practitioners. We’re committed to supporting and inspiring developers and engineers from all walks of life.

Editorially independent, Heartbeat is sponsored and published by Comet, an MLOps platform that enables data scientists & ML teams to track, compare, explain, & optimize their experiments. We pay our contributors, and we don’t sell ads.

If you’d like to contribute, head on over to our call for contributors. You can also sign up to receive our weekly newsletters (Deep Learning Weekly and the Comet Newsletter), join us on Slack, and follow Comet on Twitter and LinkedIn for resources, events, and much more that will help you build better ML models, faster.


Trending AI/ML Article Identified & Digested via Granola by Ramsey Elbasheer; a Machine-Driven RSS Bot

%d bloggers like this: