Subpixel Semantic Flow

Berk Sevilmis and Benjamin Kimia

Abstract

Dense semantic correspondence is usually cast as a variational optimization problem. Current methods generally focus on obtaining more discriminative features (to improve the data/correspondence term), and adopt a message-passing algorithm for inference, which is generally a variant of loopy belief propagation. One drawback of such optimization is that the flow vectors are constrained to be discrete variables resulting in pixel resolution, “blocky” flow fields. The main hinderance to formulating the problem in continuous space, and hence solving the problem at subpixel resolution, is the use of histogram based descriptors such as SIFT, HOG, etc. Such sparse feature descriptors are distinctive but linearize poorly. In this paper, we revisit a classic dense descriptor, namely Geometric Blur, which is, in contrast, extracted from a linear filter (spatially varying Gaussian) response that can be linearized and therefore interpolated at subpixel values. In addition to the data and smoothness terms used in variational models, we also add a term promoting bidirectional flow consistency. As there is no longer a finite set of values a flow vector can take, we use gradient descent based continuous optimization. We present promising results encouraging the use of gradient based continuous optimiza- tion in establishing dense semantic correspondences.

Session

Posters

Files

Paper (PDF)
Supplementary (PDF)

DOI

10.5244/C.31.172
https://dx.doi.org/10.5244/C.31.172

Citation

Bibtex

            @inproceedings{BMVC2017_172,
                title={Subpixel Semantic Flow},
                author={Berk Sevilmis and Benjamin Kimia},
                year={2017},
                month={September},
                pages={172.1-172.11},
                articleno={172},
                numpages={11},
                booktitle={Proceedings of the British Machine Vision Conference (BMVC)},
                publisher={BMVA Press},
                editor={Tae-Kyun Kim, Stefanos Zafeiriou, Gabriel Brostow and Krystian Mikolajczyk},
                doi={10.5244/C.31.172},
                isbn={1-901725-60-X},
                url={https://dx.doi.org/10.5244/C.31.172}
            }