Deep Photo Archive: Recognition and Retrieval of Historical Photographs

Overview

Photos Machine Learning Tools for data enrichment Linked Data

Historical photographs are full of evidence that tells us the story of a particular snapshot in time. One just needs to pay attention to the subtle cues that are found in different objects that appear in the scene: the clothes that people wear, their haircut styles, the overall environment, the tools and machinery, the natural landscape, etc. All of these visual cues are important to find semantic features and relations between photos (illustrating a particular place over several years, finding thematic clusters, estimating the creation date…).

On another hand, cross-linking photographs to other modalities of source data, like language-based resources, allows to extract richer interpretations and to augment the photo collections with complementary metadata. Examples of this are automatic captioning of photos after cross-modal representation learning of visual and language data, or using some domain-specific documents (e.g. historical tax and census records) to label photos of streets or family portraits in terms of the level of wealth.

Discovering semantic attributes and relations for historical photos can provide solutions for automatic classification of historical collections in archives, in particular estimating the creation date, or taxonomical annotation. On another hand, the implementation of advanced search and visualization tools to access to historical assets.

Project partners

With financial support by