Intra-category sketch-based image retrieval by matching deformable part models

Yi Li, Tim Hospedales, Yi-Zhe Song and Shaogang Gong

In Proceedings British Machine Vision Conference 2014
http://dx.doi.org/10.5244/C.28.115

Abstract

An important characteristic of sketches, compared with text, rests with their ability of intrinsically capturing structure and appearance detail of objects. Nonetheless, akin to traditional text-based image retrieval, conventional sketch-based image retrieval (SBIR) principally focuses on retrieving photos of the same category, neglecting the fine-grained characteristics of sketches. In this paper, we further advocate the expressiveness of sketches and examine their efficacy under a novel intra-category SBIR framework. In particular, we study how sketches can be adopted to permit pose-specific retrieval within object categories. Key challenge to this problem is introducing a mid-level sketch representation that not only captures object pose, but also possess the ability to traverse sketch and photo domains. More specifically, we learn deformable part-based model (DPM) as a mid-level representation to discover and encode the various poses and parts in sketch and image domains independently, after which graph matching is utilized to establish component and part-level correspondences across the two domains. We further propose an SBIR dataset that covers the unique aspects of fine-grained SBIR. Through in-depth experiments, we demonstrate the superior performance of our proposed SBIR framework, and showcase its unique ability in pose-specific retrieval.

Session

Poster Session

Files

Extended Abstract (PDF, 1 page, 1.0M)

Paper (PDF, 12 pages, 2.7M)

Bibtex File

Citation

Yi Li, Tim Hospedales, Yi-Zhe Song, and Shaogang Gong. Intra-category sketch-based image retrieval by matching deformable part models. Proceedings of the British Machine Vision Conference. BMVA Press, September 2014.

BibTex

@inproceedings{BMVC.28.115
	title = {Intra-category sketch-based image retrieval by matching deformable part models},
	author = {Li, Yi and Hospedales, Tim and Song, Yi-Zhe and Gong, Shaogang},
	year = {2014},
	booktitle = {Proceedings of the British Machine Vision Conference},
	publisher = {BMVA Press},
	editors = {Valstar, Michel and French, Andrew and Pridmore, Tony}
	doi = { http://dx.doi.org/10.5244/C.28.115 }
}