If you have old photos that are badly damaged, they could get a makeover using the new AI-based tool put together by Microsoft and a team of academics. It uses deep learning to bring degraded photos to life, including a triplet domain translation network using real photos as well as pairs of massive synthetic images. The authors have posted on Colab, Google’s machine learning knowledge-sharing tool, a benchmark implementation to get you started.

The photo editing tool was designed by a team of researchers from Microsoft Research Asia, Microsoft Cloud Artificial Intelligence, USTC (University of Science and Technology of China) and CityU (City University of Hong Kong). She first explained that unlike conventional restoration operations that can be solved using supervised learning, the degradation of real photos is complex and the domain gap between synthetic images and real old photos means that the network fails to generalize.

For this reason, she offers a new technique of photo restoration based on deep learning. It is a new triplet domain translation network that uses real photos as well as pairs of massive synthetic images. In other words, the team trained two variational autocoders (VAEs) to transform old photos and clean photos respectively into two latent spaces. And the translation between these two latent spaces is learned using matched synthetic data.

According to the team, this translation generalizes well to real photos, because the domain gap is closed in the compact latent space. Also, to effectively deal with multiple degradations mixed in an old photo, it keeps a global branch with a non-local partial block that targets structured defects, such as scratches and dust spots, and a local branch targeting non-local defects. structured, such as noise and blur. Then the two branches are merged in the latent space, which improves the ability to restore old photos from multiple defects.

According to the team, the method they proposed outperforms leading-edge visual quality techniques for restoring old photos. Apart from eliminating the degradations present on a photo, the tool also allows you to eliminate the scratches it contains. Finally, it can also enhance the face (s) in the photo. For this, he uses a progressive generator to refine the face areas. The tool codes and the preformed template are available on GitHub under the MIT license.

Source: Report of the study