Computer vision based reuse detection in digital artworks
Image reuse refers to the use of visual elements of existing images in order to
create new ones. In this thesis, we study the automatic image reuse detection problem
in digital artworks, which is a relatively under-studied problem of image retrieval. We
introduce two novel image reuse datasets: an artificial dataset that simulates different
types of reuse systematically, and an annotated natural dataset that includes a set
of digital artworks that are crawled from the web. Based on the natural dataset, we
propose a taxonomy which identifies the primary types of reuse and manipulations.
Then, for image reuse detection, we evaluate different feature extraction and classification
methods that are commonly used for image copy detection, content-based image
retrieval, and computer analysis of artworks. The features we use include, color histograms,
Histogram of Oriented Gradient (HOG) descriptors, and the Scale Invariant
Feature Transform (SIFT) descriptor and its color-based variants. We use the bag-of-
visual-words (BoW) approach with the SIFT descriptors. We also present a novel
image description algorithm, called the Affine Invariant Salient Patch (AISP) descriptor,
which provides a foreground sensitive description of images by fitting concentric
ellipses to the most salient region in an image and extracting features from each track.
Our results show that the AISP method can be suitable for reuse detection with its
compactness and good retrieval accuracy, especially in images with prominent foreground
objects. On the other hand, the use of the SIFT descriptors in a BoW model
can be more advisable in a more natural setting and for cluttered scenes.