Automatic Creation of Large Scale Object Databases from Web Resources: A Case Study in Robot Vision

A fundamental ingredient in the success of deep learning for computer and robot vision is the availability of very large-scale annotated databases. ImageNet, with its 1000 object classes and 1.2 million images, tends to be the dominant data collection for creating pre-trained deep architectures. A less investigated avenue is how the possibility to create task-specific data collections on demand, with limited or without manual effort, would affect the performance of convolutional architectures. This would be useful for all those cases where contextual information about the deployment of the deep net is available, and it would be particularly relevant for robot vision applications, where such knowledge is usually available. The goal of this work is to present a protocol for the automated creation of task specific datasets starting from a pre-defined list of object classes, exploiting the Web as a source of information in an automated fashion. Our pipeline consists of (a) an algorithm for automatic Web crawling that searches for “image class seeds”, i.e., informative images of object classes of interest, (b) algorithms for figure-ground segmentation of the object of interest and pasting of the segmented item in contextual images close to where the agent is going to work, and (c) a tailored data augmentation routine for maximizing the informative content of the generated images. A thorough set of experiments on a public benchmark, as well as deployment to a robot platform, prove the value of the proposed approach