SVM Module

A SVM model.

The doc is here : http://scikit-learn.org/dev/modules/svm.html#svm and there : http://scikit-learn.org/dev/modules/generated/sklearn.svm.SVC.html#sklearn.svm.SVC

Méta-apprentissage

Pour régler les méta-paramètres des méthodes de classification, dans chaque script, je sépare les données de train.csv en apprentissage et test (selon une proportion de 55% à 85%).

À faire

Utiliser sklearn.cross_validation.train_test_split pour séparer les données de ‘train.csv’ en train et test.

Pour les SVM, je règle la constante C de régularisation.

Il y a d’autres méta-paramètres, que je n’ai pas encore cherché à modifier.


Sortie du script

$ python SVM.py
Opening the file 'train.csv' and 'test.csv'...
Find the best value for the meta parameter C, with 5 run for each...
Searching in the range : [1e-05, 0.0001, 0.001, 0.01, 0.1, 1, 10, 100, 1000, 10000, 100000, 5e-06, 5e-05, 0.0005, 0.005, 0.05, 0.5, 5.0, 50.0, 500.0, 5000.0, 50000.0]...
Using the first part (67.00%, 596 passengers) of the training dataset as training, 
and the second part (33.00%, 295 passengers) as testing !
For C=1e-05, learning from the first part of the dataset...
... this value of C seems to have a (mean) quality = 61.91%...
For C=0.0001, learning from the first part of the dataset...
... this value of C seems to have a (mean) quality = 61.91%...
For C=0.001, learning from the first part of the dataset...
... this value of C seems to have a (mean) quality = 61.91%...
For C=0.01, learning from the first part of the dataset...
... this value of C seems to have a (mean) quality = 61.91%...
For C=0.1, learning from the first part of the dataset...
... this value of C seems to have a (mean) quality = 80.03%...
For C=1, learning from the first part of the dataset...
... this value of C seems to have a (mean) quality = 80.03%...
For C=10, learning from the first part of the dataset...
... this value of C seems to have a (mean) quality = 81.38%...
For C=100, learning from the first part of the dataset...
... this value of C seems to have a (mean) quality = 81.21%...
For C=1000, learning from the first part of the dataset...
... this value of C seems to have a (mean) quality = 81.38%...
For C=10000, learning from the first part of the dataset...
... this value of C seems to have a (mean) quality = 82.21%...
For C=100000, learning from the first part of the dataset...
... this value of C seems to have a (mean) quality = 81.04%...
For C=5e-06, learning from the first part of the dataset...
... this value of C seems to have a (mean) quality = 61.91%...
For C=5e-05, learning from the first part of the dataset...
... this value of C seems to have a (mean) quality = 61.91%...
For C=0.0005, learning from the first part of the dataset...
... this value of C seems to have a (mean) quality = 61.91%...
For C=0.005, learning from the first part of the dataset...
... this value of C seems to have a (mean) quality = 61.91%...
For C=0.05, learning from the first part of the dataset...
... this value of C seems to have a (mean) quality = 78.19%...
For C=0.5, learning from the first part of the dataset...
... this value of C seems to have a (mean) quality = 79.70%...
For C=5.0, learning from the first part of the dataset...
... this value of C seems to have a (mean) quality = 81.04%...
For C=50.0, learning from the first part of the dataset...
... this value of C seems to have a (mean) quality = 80.87%...
For C=500.0, learning from the first part of the dataset...
... this value of C seems to have a (mean) quality = 81.38%...
For C=5000.0, learning from the first part of the dataset...
... this value of C seems to have a (mean) quality = 81.71%...
For C=50000.0, learning from the first part of the dataset...
... this value of C seems to have a (mean) quality = 80.87%...
With trying each of the following C ([1e-05, 0.0001, 0.001, 0.01, 0.1, 1, 10, 100, 1000, 10000, 100000, 5e-06, 5e-05, 0.0005, 0.005, 0.05, 0.5, 5.0, 50.0, 500.0, 5000.0, 50000.0]), each 5 times, the best one is 10000. (for a quality = 82.21%)
Creating the classifier with the optimal value of C.
Learning...
 Proportion of perfect fitting for the training dataset = 87.21%
Predicting for the testing dataset
Prediction: wrote in the file csv/SVM_best.csv.

Résultats

La soumission du résultat à Kaggle donne 65.07%.


SVM.list_C = [1e-05, 0.0001, 0.001, 0.01, 0.1, 1, 10, 100, 1000, 10000, 100000, 5e-06, 5e-05, 0.0005, 0.005, 0.05, 0.5, 5.0, 50.0, 500.0, 5000.0, 50000.0]

Espace de recherche

SVM.Number_try = 5

Nombre de tests utilisés pour méta-apprendre

SVM.proportion_train = 0.67

Proportion d’individus utilisés pour méta-apprendre.

SVM.best_C = 500.0

La valeur optimale trouvée pour le paramètre n_estimators

SVM.score = 85.746352413019082

The score for this classifier.

Table des matières

Cette page en .txt et en .pdf

Sujet précédent

ExtraTrees Module

Sujet suivant

AdaBoost Module