Randomized Support Vector Forest

Xutao Lv, Tony Han, Zicheng Liu and Zhihai He

In Proceedings British Machine Vision Conference 2014
http://dx.doi.org/10.5244/C.28.61

Abstract

Based on the structural risk minimization principle, the linear SVM aiming at find increasing popularity in the vision community due to its generalizability, efficiency and acceptable performance. However, rarely training data are evenly distributed in the input space [1], which leads to a high global VC confidence [4], downgrading the performance of the linear SVM classifier. Partitioning the input space in tandem with local learning may alleviate the unevenly data distribution problem. However, the extra model com- plexity introduced by partitioning frequently leads to overfitting. To solve this problem, we proposed a new supervised learning algorithm, Randomized Support Vector Forest (RSVF): Many partitions of the input space are constructed with partitioning regions amenable to the corresponding linear SVMs. The randomness of the partitions is injected through random feature selection and bagging. This partition randomness prevents the overfitting introduced by the over-complicated partitioning. We extensively evaluate the performance of the RSVF on several benchmark datasets, originated from various vision applications, including the four UCI datasets, the letter dataset, the KTH and the UCF sports dataset, and the Scene-15 dataset. The proposed RSVF outperforms linear SVM, kernel SVM, Random Forests (RF), and a local learning algorithm, SVM-KNN, on all of the evaluated datasets. The classification speed of the RSVF is comparable to linear SVM.

Session

Poster Session

Files

Extended Abstract (PDF, 1 page, 115K)

Paper (PDF, 12 pages, 339K)

Bibtex File

Citation

Xutao Lv, Tony Han, Zicheng Liu and Zhihai He. Randomized Support Vector Forest. Proceedings of the British Machine Vision Conference. BMVA Press, September 2014.

BibTex

@inproceedings{BMVC.28.61
	title = {Randomized Support Vector Forest},
	author = {Lv, Xutao and Han, Tony and Liu, Zicheng and He, Zhihai},
	year = {2014},
	booktitle = {Proceedings of the British Machine Vision Conference},
	publisher = {BMVA Press},
	editors = {Valstar, Michel and French, Andrew and Pridmore, Tony}
	doi = { http://dx.doi.org/10.5244/C.28.61 }
}