Mix and Match: Joint Model for Clothing and Attribute Recognition

Kota Yamaguchi, Takayuki Okatani, Kyoko Sudo, Kazuhiko Murasaki and Yukinobu Taniguchi

Abstract

This paper studies clothing and attribute recognition in the fashion domain. Specifically, in this paper, we turn our attention to the compatibility of clothing items and attributes. For example, people do not wear a skirt and a dress at the same time, yet a jacket and a shirt are a preferred combination. We consider such inter-object or inter-attribute compatibility in the recognition problem, and formulate a Conditional Random Field (CRF) that seeks the most probable combination in the given picture. The model takes into account the location-specific appearance with respect to a human body and the semantic correlation between clothing items and attributes, which we learn using the max-margin framework. We evaluate our model using two datasets that resemble realistic application scenarios: on-line social networks and shopping sites. The empirical evaluation shows that our model effectively improves the recognition performance over baselines including the state-of-the-art feature designed exclusively for clothing recognition. The results also suggest that our model generalizes well to different fashion-related applications.

Session

Poster 1

Files

Extended Abstract (PDF, 379K)
Paper (PDF, 5M)

DOI

10.5244/C.29.51
https://dx.doi.org/10.5244/C.29.51

Citation

Bibtex

@inproceedings{BMVC2015_51,
	title={Mix and Match: Joint Model for Clothing and Attribute Recognition},
	author={Kota Yamaguchi and Takayuki Okatani and Kyoko Sudo and Kazuhiko Murasaki and Yukinobu Taniguchi},
	year={2015},
	month={September},
	pages={51.1-51.12},
	articleno={51},
	numpages={12},
	booktitle={Proceedings of the British Machine Vision Conference (BMVC)},
	publisher={BMVA Press},
	editor={Xianghua Xie, Mark W. Jones, and Gary K. L. Tam},
	doi={10.5244/C.29.51},
	isbn={1-901725-53-7},
	url={https://dx.doi.org/10.5244/C.29.51}
}