AdaBoost Module

A AdaBoost model.

The doc is here : http://scikit-learn.org/dev/modules/generated/sklearn.ensemble.AdaBoostClassifier.html

Overfitting

Je fais une vérification de l’overfitting via la fonction Score, qui permet de savoir combien des passengers dans l’ensemble d’apprentissage sont classés dans la bonne catégorie par le classifieur.

Ce score devrait être < 95%, sinon c’est “louche”.

Ce paragraphe s’applique à tous les autres classifieurs présentés après, mais il n’est pas répété.

---

Sortie du script

$ python AdaBoost.py
Opening the file 'train.csv' and 'test.csv'...
Find the best value for the meta parameter n_estimators, with 5 run for each...
Searching in the range : [1, 5, 10, 20, 30, 40, 50, 60, 75, 82, 100]...
Using the first part (67.00%, 596 passengers) of the training dataset as training, 
and the second part (33.00%, 295 passengers) as testing !
For 1 random tree(s), learning from the first part of the dataset...
... this value of n_estimators seems to have a (mean) quality = 78.86%...
For 5 random tree(s), learning from the first part of the dataset...
... this value of n_estimators seems to have a (mean) quality = 81.54%...
For 10 random tree(s), learning from the first part of the dataset...
... this value of n_estimators seems to have a (mean) quality = 79.53%...
For 20 random tree(s), learning from the first part of the dataset...
... this value of n_estimators seems to have a (mean) quality = 81.04%...
For 30 random tree(s), learning from the first part of the dataset...
... this value of n_estimators seems to have a (mean) quality = 82.55%...
For 40 random tree(s), learning from the first part of the dataset...
... this value of n_estimators seems to have a (mean) quality = 81.71%...
For 50 random tree(s), learning from the first part of the dataset...
... this value of n_estimators seems to have a (mean) quality = 81.54%...
For 60 random tree(s), learning from the first part of the dataset...
... this value of n_estimators seems to have a (mean) quality = 82.05%...
For 75 random tree(s), learning from the first part of the dataset...
... this value of n_estimators seems to have a (mean) quality = 81.71%...
For 82 random tree(s), learning from the first part of the dataset...
... this value of n_estimators seems to have a (mean) quality = 82.21%...
For 100 random tree(s), learning from the first part of the dataset...
... this value of n_estimators seems to have a (mean) quality = 81.88%...
With trying each of the following n_estimators ([1, 5, 10, 20, 30, 40, 50, 60, 75, 82, 100]), each 5 times, the best one is 30. (for a quality = 82.55%)
Creating the adaboost classifier (of 100 estimators)...
Learning...
 Proportion of perfect fitting for the training dataset = 82.60%
Predicting for the testing dataset
Prediction: wrote in the file csv/AdaBoost_best.csv.

Résultats

La soumission du résultat à Kaggle donne 75.12%.

---

AdaBoost.list_n_estimators = [1, 5, 10, 20, 30, 40, 50, 60, 75, 82, 100]

Espace de recherche

AdaBoost.Number_try = 5

Nombre de tests utilisés pour méta-apprendre

AdaBoost.proportion_train = 0.67

Proportion d’individus utilisés pour méta-apprendre.

AdaBoost.best_n_estimators = 100

La valeur optimale trouvée pour le paramètre n_estimators

AdaBoost.score = 84.062850729517393

The score for this classifier.