A year ago I bought from Amazone bookstore "Statistical and Neural Classifiers: An integrated approach to design."
Springer, 2001 by Raudys S.
I began to read it, found it interesting and useful, however, it was rather difficult to study since it required certain tension and a general knowledge in the field of statistical pattern recognition.
So I kept the book in my bookshelf for several months until September 2002 when I had to evaluate how many vectors I needed to design decision tree classifier for the problem related to ecology.
I asked my colleague, also PhD student, for an advice. He looked around and noticed Raudys S. book.
He opened it and pointed to Table 3.7 where generalization error for Multinomial classifier was tabulated.
I applied this result to the binary decision tree and solved my problem.
After this event, I got a stimulus to read the book more attentively. Soon reading became much easier and I was able to discover many useful results for me.
For example, I found that simple Equation based on binomial distribution for variance of error counting error estimate
stand_dev=sqrt(Perror x (1-Perror)/sample size, Section 6.3.1) can be used
a) for a rough provisional evaluation of the accuracy of classification error estimate and
b) for approximate evaluation of the optimistic bias when we select the best features or the best classifier's model. I learned, roughly speaking, that the bias of adaptation to
the test data is approximately equal to standard deviation mentioned in Section 6.5.3.
I understood that number of vectors required to train Euclidean distance classifier (EDC) can be small (Table 3.1 in p. 87).
I become acquainted with the fact that if we move the sample mean vector into the origin of the coordinates, train the perceptron starting from zero initial weight vector in the batch mode, we get EDC just after the first iteration.
If the data is decorrelated and scaled, then EDC (some people cal l it the nearest means classifier) is not so bad classifier.
Thus, small training set can be sufficient to train perceptron per few iterations.
To train longer we need more training vectors, since more complex classifiers (regularized Fisher, standard Fisher, robust Fisher, .. , support vector) are obtained.
Since September Raudys S. book became my Handbook.
