Wekalist Digest, Vol 60, Issue 24 Send Wekalist mailing list submissions to 	iuxxdkqk@list.scms.waikato.ac.nz To subscribe or unsubscribe via the World Wide Web, visit 	https://list.scms.waikato.ac.nz/mailman/listinfo/wekalist or, via email, send a message with subject or body 'help' to 	ohwwlejs-pbfotbp@list.scms.waikato.ac.nz You can reach the person managing the list at 	pobypuhs-hvhzj@list.scms.waikato.ac.nz When replying, please edit your Subject line so it is more specific than "Re: Contents of Wekalist digest..." Today's Topics:    1. Re: Questions about the experimenter (Peter Reutemann)    2. Re: question on feature selection (Harri M.T. Saarikoski)    3. Weka models into PMML (Carmelo Saffioti)    4. Memory independent algorithms (Carmelo Saffioti)    5. Ignoring  attribute (rafiqul haque)    6. Re: Memory independent algorithms (Peter Reutemann)    7. Re: Ignoring  attribute (Peter Reutemann)    8. Re: Weka models into PMML (Peter Reutemann)    9. Re: Knowledge Flow interface. Clustering. Can variables	that       are not input to the clustering algorithm be kept or	reassociated       with the result of clustering? (Mark Hall) ---------------------------------------------------------------------- >From dlnirllc@cs.waikato.ac.nz  Sat Apr 26 15:52:36 2008 Message: 1 Date: Sat, 16 Feb 2008 14:44:50 +1300 (NZDT) From: "Peter Reutemann"  Subject: Re: [Wekalist] Questions about the experimenter To: "Weka machine learning workbench list." 	 Message-ID: 	<44445.408.84.01.039.1444454088.sujmdnjk@webmail.scms.waikato.ac.nz> Content-Type: text/plain;charset=iso-8859-1 > What I am not sure about: is there a way to have seperate experiment > results > in one > database? I was not able to achieve this (everything seems to go into the > Results0 table > automatically) and had to use a different mysql database for each > experiment > that i want > to keep seperated. The Experiment setup, i.e., cross-validation/percentage-split (randomized/fixed) and their parameters, defines a single experiment. Basically, as soon as you can't compare it anymore (you can't compare cross-validation and percentage-split results, or 2-fold and 10-fold CV), the results will be directed into another table. If you always use the same setup, you will need to create different databases for it (like you create different output files). >> 3) When one shows the summary or Ranking, a lot of pairwise significance >> > tests are used. >> > Are these tests Bonferroni adjusted? (does the "corrected" shown with >> paired >> > t-test indicate >> > Bonferroni correction for multiple significance tests or is this a >> different >> > kind of correction? >> >> Copy/paste from the weka.experiment.PairedCorrectedTTester Javadoc: >> >> Behaves the same as PairedTTester, only it uses the corrected resampled >> t-test statistic. >> >> For more information see: >> >> Claude Nadeau, Yoshua Bengio (2001). Inference for the Generalization >> Error. Machine Learning.. >> > > I dug that up and as far as my little understanding of statistics goes, > the > corrected > t-test corrects for the under-estimate of variance when one does repeated > CVs. > > So it seems that there is no (automatic) Bonferroni adjustment -- at least > it is nowhere > mentioned in Weka. Sorry, but my statistics knowledge is even less... HTH Cheers, Peter --  Peter Reutemann, Dept. of Computer Science, University of Waikato, NZ http://www.cs.waikato.ac.nz/~fracpete/           Ph. +64 (7) 858-5174 ------------------------------ >From grelj.zktpquktdd@helsinki.fi  Sat Apr 26 15:52:36 2008 Message: 2 Date: Sat, 16 Feb 2008 11:40:53 +0200 From: "Harri M.T. Saarikoski"  Subject: Re: [Wekalist] question on feature selection To: "Weka machine learning workbench list." 	 Message-ID: <1203154853.47b6afa57b1c4@webmail.helsinki.fi> Content-Type: text/plain; charset=us-ascii Lainaus FANG WANG : > Hello All: >  > I am recently doing feature selection and classification. >  > For example, I have a gene list, each gene contains 15 features, With > Weka, by using filter, I get 5 features among the initial 15 fetures, > with > these 5 features, using supervised classifer, I could seperate these > genes > into cancer-related and noncancer-related, Now I want to use this 5 > features to scan > another group of genes, and want to try to find out which is cancer > related and which is not. >  > In a brief sentence, my question is: > 1.a list of genes (each gene has 15 features, I already know which is > cancer related and which is  not) > 2.a list of genes to be investigate on. (each gene have the same feature > as above genes, but no idea of which is cancer related genes) Hi.  If you are able to reduce dimensions to five features and only want to  classify cancer/no (and not the extent to which each of these features  predicts that for instance), I think any sensible classifier (Naive Bayes,  Decision Trees, instance-based classifier Ibk) will deliver optimal  accuracy (95+%). At least this is what I have found. If it is not possible to mark the classes for these new gene instances  (test set), you should use cross-validation (10-fold CV in Explorer or more  recommendedly 10 times 10-fold CV in Experimenter). The only possible  unexpected problem can come with iterative classifiers (support vector  machines SMO or LibSVM, error-minimizing neural network  MultiLayerPerceptron). With these it is possible (when their key parameters  are set too 'tight') that even though CV error is (close to) zero, test  error may not be. This is because they tend to make more generalizations  about the carsinogeny of the value ranges of your features:   - With SVM the problem comes if the support vectors (instances that mark  the class border) cannot be defined clearly (i.e. some instances are highly  ambiguous). Basically you could consider using SVM if there are  several 'local maxima' per feature, i.e. that ranges 0..10 and 80..100 (but  not 10..80) are correlated with cancer, since SVM does not evaluate  features on their own but as part of whole instances. - With neural networks the problem is when to stop the iteration of feature  weight changes (hard to estimate and can lead to overfitting, i.e.  incorrect feature weights). Anyway, set learning rate parameter high  because your dataset does not require many iterations. This (low generalization accuracy) does not tend to happen with  more 'stable' classifiers such as Naive Bayes, instance-based classifier  lazy.Ibk (e.g. J48 with confidence parameter set at maximum 0.50, which  performs no pruning of the tree) or basically any decision tree (trees  package). Therefore, these would be a safer choice. If you expect that some of the five features work together to predict  cancer, consider BayesNet in place of Naive Bayes, since it assumes  features are correlated. Harri >  > Any suggestion and idea on this question will be so appreciated. >  > Thanks! >  > Fang Wang >  > _______________________________________________ > Wekalist mailing list > iuxxdkqk@list.scms.waikato.ac.nz > https://list.scms.waikato.ac.nz/mailman/listinfo/wekalist >  >  ------------------ yst. terv | Best wishes Harri Saarikoski ------------------------------