I've improved my text classification to topic module, from simple word2vec to piped tfidf and OneVsRestClassifier (using sklearn). It does improve the classification but with word2vec I was able to calculate the match percentage for each topic and with OneVsRestClassifier i get a match or no match to a specific topic. Is there a way to see with OneVsRestClassifier what was the percentage of the classification?P.S.I am not talking about evaluating the performance of the training but the actual real time matching percentage.
How to detect the match precision of OneVsRestClassifier
classification;nlp;scikit learn;multiclass classification
Yes, of course. Assuming that you have used sklearn's OneVsRestClassifier and so you have a decision function for example a Support Vector Classifier with say linear kernel. Use set_params to change probability key to True, default is False. Use this in the OneVsRestClassifier classifier and then go with the inbuilt function predict_proba likefrom sklearn.multiclass import OneVsRestClassifierfrom sklearn.svm import SVCmod = OneVsRestClassifier(SVC(kernel='linear').set_params(probability=True)).fit(samples,classes)print mod.predict_proba(np.array([your_sample_vector]).reshape(1,-1))Edit:You can use your old LinearSVC with decision_function to find the distance from the hyperplane and convert them to probabilities likemod = OneVsRestClassifier(LinearSVC()).fit(sample,clas)proba = (1./(1.+np.exp(-mod.decision_function(np.array(your_test_array).reshape(1,-1)))))proba /= proba.sum(axis=1).reshape((proba.shape[0], -1))\print probaNow you don't need tuning the parameters, I guess. :)