Data mining could be considered to be "Artificial Intelligence Lite", since it deals with many of the same issues in learning, classification, and analysis as they occur in the field of artificial intelligence but does not have as its goal the construction of "thinking machines." Instead, the emphasis is on practical problems that are important in business and industry, even though the solutions of many of these problems makes use of techniques that a thinking machine should be expected to have. Data mining has become an enormous industry, and has even been the subject of political and legal concerns due to the efforts of some governments to mine data on its citizens. This book gives a general overview of data mining with emphasis on classification and associative analysis. Anyone who is interested in data mining could read the book, but some rather sophisticated background in mathematics will be needed to read some of the sections. Pseudocode is given throughout the book to illustrate the different data mining algorithms. There are also exercises at the end of each chapter, but noticeably missing in the book is the inclusion of real case studies in data mining. The inclusion of these case studies would alert the reader to the fact that data mining is of great interest from the standpoint of business and industry, and would lessen the belief that data mining is just another academic field or just another branch of statistics.
Speaking somewhat loosely, the goal of data mining is to find interesting patterns in massive amounts of data or the classification of such patterns. This entails of course that one have a notion of what is "interesting" and one of the main problems in data mining is to find suitable `interestingness measures'. And since one is typically dealing with large amounts of data, one must use various statistical sampling and preprocessing techniques to massage the data and obtain a `representative' sample of the original data. In addition, one must be able to handle data that is `anomalous', i.e. data that has characteristics that are markedly different from most of the other data, or that has attributes that are unusual if compared with typical values for those attributes. These issues and techniques are discussed in detail in the first three chapters of the book, where the authors outline some of the bread-and-butter topics needed for effective manipulation of data.
The real substance and power of data mining comes from its role in classification and for discovering interesting patterns in huge data sets. The authors, in chapters 4 - 7, discuss various powerful techniques for data classification and association analysis. Association analysis in particular has been used quite extensively in recent years, due to the use of market basket transactions in on-line purchasing and the goal of marketers to learn the purchasing behavior of their customers. Association analysis uncovers relationships in the marketing data in the form of `association rules'. For disjoint itemsets X and Y, an association rule is a logical implication expression between these itemsets that has a certain `strength' that is measured by its `support' and `confidence.' The support measures how often a rule is applicable to a given data set, while the confidence measures how frequently the items in Y appear in X. The support reflects the ability of the rule to be not due to chance alone, while the confidence measures the reliability of the rule inference. The collection of all association rules that can be formed from a data set is too large to be practical and so strategies must be developed to prune the number of rules. The authors discuss in detail various methods for dealing with this computational drawback, such as `frequent itemset generation' and `rule generation.'
The detection of anomalies consists of the identification of `outliers', which as the name implies are data objects that lie "far away" from the other data objects. It remains of course to quantity what it means to be "far away" and for this reason this branch of data mining, as the author points out, is sometimes called `deviation detection' or `exception mining'. The omission of outliers is sometimes justified, since they are merely artifacts that only serve to alter the statistics of a particular data set. However, sometimes their presence signals important information, if not a major scientific discovery. Data mining therefore must contain tools that detect anomalies intelligently and efficiently. The authors discuss anomaly detection in fair detail, emphasizing the statistical techniques that are available to do it. They classify the techniques for anomaly detection as being `unsupervised', `supervised', and `semi-supervised'. As the name implies, supervised anomaly detection requires the existence of a training set with both anomalous and "normal" data with each class being labeled as such. When these labels are unavailable, one has to perform unsupervised anomaly detection, and for this approach to work the anomalies must be distinct from one another. If the normal data is labeled but the anomalies are not, one must do semi-supervised anomaly detection. The only weakness in the authors' discussion is that they do not include real-world case studies that illustrate the different techniques, such as clustering and density methods.