We began by building classification models using only general features obtained by observing the
word counts, word lengths, and character properties in individual messages.
We compared the use of different models and performed
feature scoring to rank the perceived usefulness of each feature. All model
evaluations were performed using 10 repetitions of 10-fold cross-validation.
Explicit comparisons between different methods were made using the Bayesian correlated
t-test which can be used to compute probabilities of one method being better than the other and
avoids some common pitfalls associated with the more typical frequentist
approaches [1]}. Subsequently, the next batch of features was added to the feature
extraction/engineering process and the evaluation process repeated. for the initial feature subset
evaluations, we focused exclusively on the book relevance prediction objective.
Using the full feature set, we evaluate the best scoring models on all prediction objectives.
