LSTM for Image Annotation with Relative Visual Importance

Geng Yan, Yang Wang and Zicheng Liao

Abstract

We consider the problem of image annotations that takes into account of the relative importance of tags. Previous work usually consider the tags associated with an image as an unordered list of object names. In contrast, we exploit the implicit cues about the relative importance of objects mentioned by the tags. For example, important objects tend to be mentioned first in a list of tags. Our proposed a recurrent neural network with long-short term memory for this problem. Given an image, our model can produce a ranked list of tags, where tags for important objects appear earlier in the list. Experimental results demonstrate that our model achieves better performance on several benchmark datasets.

Session

Posters 2

Files

PDF iconExtended Abstract (PDF, 336K)
PDF iconPaper (PDF, 1M)

DOI

10.5244/C.30.78
https://dx.doi.org/10.5244/C.30.78

Citation

Geng Yan, Yang Wang and Zicheng Liao. LSTM for Image Annotation with Relative Visual Importance. In Richard C. Wilson, Edwin R. Hancock and William A. P. Smith, editors, Proceedings of the British Machine Vision Conference (BMVC), pages 78.1-78.11. BMVA Press, September 2016.

Bibtex

        @inproceedings{BMVC2016_78,
        	title={LSTM for Image Annotation with Relative Visual Importance},
        	author={Geng Yan, Yang Wang and Zicheng Liao},
        	year={2016},
        	month={September},
        	pages={78.1-78.11},
        	articleno={78},
        	numpages={11},
        	booktitle={Proceedings of the British Machine Vision Conference (BMVC)},
        	publisher={BMVA Press},
        	editor={Richard C. Wilson, Edwin R. Hancock and William A. P. Smith},
        	doi={10.5244/C.30.78},
        	isbn={1-901725-59-6},
        	url={https://dx.doi.org/10.5244/C.30.78}
        }