Fine-Pruning: Joint Fine-Tuning and Compression of a Convolutional Network with Bayesian Optimization

Frederick Tung, Srikanth Muralidharan and Greg Mori

Abstract

When approaching a novel visual recognition problem in a specialized image domain, a common strategy is to start with a pre-trained deep neural network and fine-tune it to the specialized domain. If the target domain covers a smaller visual space than the source domain used for pre-training (e.g. ImageNet), the fine-tuned network is likely to be over-parameterized. However, applying network pruning as a post-processing step to reduce the memory requirements has drawbacks: fine-tuning and pruning are performed independently; pruning parameters are set once and cannot adapt over time; and the highly parameterized nature of state-of-the-art pruning methods make it prohibitive to manually search the pruning parameter space for deep networks, leading to coarse approximations. We propose a principled method for jointly fine-tuning and compressing a pre-trained convolutional network that overcomes these limitations.

Session

Posters

Files

Paper (PDF)

DOI

10.5244/C.31.115
https://dx.doi.org/10.5244/C.31.115

Citation

Bibtex

            @inproceedings{BMVC2017_115,
                title={Fine-Pruning: Joint Fine-Tuning and Compression of a Convolutional Network with Bayesian Optimization},
                author={Frederick Tung, Srikanth Muralidharan and Greg Mori},
                year={2017},
                month={September},
                pages={115.1-115.12},
                articleno={115},
                numpages={12},
                booktitle={Proceedings of the British Machine Vision Conference (BMVC)},
                publisher={BMVA Press},
                editor={Tae-Kyun Kim, Stefanos Zafeiriou, Gabriel Brostow and Krystian Mikolajczyk},
                doi={10.5244/C.31.115},
                isbn={1-901725-60-X},
                url={https://dx.doi.org/10.5244/C.31.115}
            }