Языки
Modest Adaboost
Introduction
This project was devoted to investigation of modern classification techniques and their possible application to computer vision tasks.
Novel Weak Classifier Boosting Algorithm
This project was devoted to investigation of modern classification techniques and their possible application to computer vision tasks.
During the research we have devised a new classification algorithm, based on weak classifier boosting approach. Our method, called "Modest AdaBoost", was implemented in MatLab environment and compared to well known Gentle AdaBoost scheme. Our experiments, conducted on UCI Machine Learning Repository database sets using 5-fold cross validation, show that our algorithm provides:
- Better generalization capabilities;
- Higher robustness to overfitting;
- High robustness to noise;
- Natural Stopping criterion;
Algorithm
Main difference of Modest AdaBoost from other boosting methods is in the way, how the weights for each weak hypothesis are calculated. We tried to make a boosting procedure less greedy, by penalizing the hypothesis, that are too much correlated with the performance of the classifier built on previous steps. While providing additional beneficial properties, algorithm does not become much more complex - in words, it's you dont have to tune any parameters, or implement some complex prcedures. It's as easy and convinient as basic boosting methods. For more information, see the publication section.
Experiments
To evaluate the performance of our algorithm we ran 5-fold cross-validation on some datasets from UCI Repository for 20 times and have averaged the results. Some of the results you can see above.
Future Work
We plan to further improve our algorithm, and investigate it's characteristics. Now we are working on more experimental analyzes.
Download
Implementation is avaliable in our AdaBoost Matlab Toolbox . Feel free to download and try it out.
Download GML AdaBoost Matlab Toolbox 0.3
Publications
Alexander Vezhnevets, Vladimir Vezhnevets "'Modest AdaBoost' - Teaching AdaBoost to Generalize Better". Graphicon-2005, Novosibirsk Akademgorodok, Russia, 2005.
.pdf(107 KB)
The project team
Undergraduate students: Alexander Vezhnevets (graduated June 2006)
Faculty advisors: Dr. Vladimir Vezhnevets
Contacts
Vezhnevets Alexander
avezhnevets@graphics.cs.msu.ru