Saliency Prediction with Active Semantic Segmentation

Ming Jiang, Xavier Boix, Juan Xu, Gemma Roig, Luc Van Gool and Qi Zhao

Abstract

Semantic-level features have been shown to provide a strong cue for predicting eye fixations. They are usually implemented by evaluating object classifiers everywhere in the image. As a result, extracting the semantic-level features may become a computational bottleneck that may limit the applicability of saliency prediction in real-time applications. In this paper, to reduce the computational cost at the semantic level, we introduce a saliency prediction model based on active semantic segmentation, where a set of new features are extracted during the progressive extraction of the semantic labeling. We recorded eye fixations on all the images of the popular MSRC-21 and VOC07 datasets. Experiments in this new dataset demonstrate that the semantic-level features extracted from active semantic segmentation improve the saliency prediction from low- and regional-level features, and it allows controlling the computational overhead of adding semantics to the saliency predictor.

Session

Poster 1

Files

Extended Abstract (PDF, 879K)
Paper (PDF, 4M)

DOI

10.5244/C.29.15
https://dx.doi.org/10.5244/C.29.15

Citation

Bibtex

@inproceedings{BMVC2015_15,
	title={Saliency Prediction with Active Semantic Segmentation},
	author={Ming Jiang and Xavier Boix and Juan Xu and Gemma Roig and Luc  Van Gool and Qi Zhao},
	year={2015},
	month={September},
	pages={15.1-15.13},
	articleno={15},
	numpages={13},
	booktitle={Proceedings of the British Machine Vision Conference (BMVC)},
	publisher={BMVA Press},
	editor={Xianghua Xie, Mark W. Jones, and Gary K. L. Tam},
	doi={10.5244/C.29.15},
	isbn={1-901725-53-7},
	url={https://dx.doi.org/10.5244/C.29.15}
}