Automated Software Testing: Introduction, Management and Performance [Paperback]

Some of the areas which might be considered improvements for a future edition include any/all of the following:

Without exception, there is no mention or discussion on incorporating 'disk imaging' into the QA life cycle. How can you test incremental builds efficiently, with confidence, and stay within budget if the QA or IS department spends half of EACH day rebuilding a PC client and/or server with a 'virgin' OS, network and printing drivers, QA automation applications and preferred settings, and a 'clean' registry? See [webpage] for specifics of "Ghost" software.

A very useful code coverage analysis tool with is completely integration with Microsoft's Developer Studio was missed. The vendor is NuMega DevPartner and the related internet link is [online].

Consider using Visio 2000...for illustrations/diagrams (e.g. pps. 9, 11, 15, 29, 31, 69, 415). This software package has won many product awards and has been adopted by millions of users in all aspects of society including authoring and printing.

Issues stated by page number:

Page 11 - Reduce the table content and synchornize the various section numbers to the table: Section 1: 1.1, 1.2, 2.1, 2.2, 2.3 as section 1 Section 2: 1.3 (and budget/skills), 2.4 (and implementation) Section 3: 3.1, 3.2, 4.1, 4.2 Section 4: 4.3, 4.4, 5.1 Section 5: 5.2 - 5.4 Section 6: 5.5, 6.1 Section 7: Test reporting (audience, criteria, frequency, and methodology) Input from MKT, MGT, DEV, DOCS, QA, and CUSTOMERS (e.g. Microsoft Office)

Page 23 - Consider adding an internet site list, and in the chapter where career paths are discussed [online].

Page 25 - Within the Team Lead section on Table 1.4, the use of "for" should be "four".

Page 50 - has a good table, but bad reference declaration (difficult to follow). Same thing on Page 53. N is defined, but the meaning of the definition appears hidden on page 52 (e.g. "N").

Page 90 - good explanation of test tool evaluation, but re-prioritize factors such as: Pricing, Ease of Use, Tool Functionality, and Reporting Capability

For example, if one can't afford the tool, who cares if it's easy to use or has particular functionality? Similarly, if reports are not available or too cumbersome to obtain, how will you convince management that the investment in the tool was worthwhile?

Page 97 - I thoroughly disagree with the Guidelines for an Unsuitable Pilot Project discussion.

Page 99 - Excellent remarks on how to report on the evaluation of test tools.

Page 129 - An important issue with a naïve conclusion (anyone with any experience in QA knows about the implicitly defined "collapsed release cycle") is provided. This is what happens when development is late in achieving milestones related to GUI freeze, function complete, code complete, and bug fixes. Typically, QA is forced into overtime for the purpose of locating newly placed software defects and verifying bug fixes given a reduced time allotment (typically less than two weeks for setup, on-line documentation, and regression testing).

Page 134 - The test tool diagram content is excellent. Have you considered using Visio 2000 for diagrams?

Page 138 - A limited view of when to introduce an automated test tool is utilized. Have you considered the norm whereby an evolving application GUI and/or API causes QA test development efforts to be scrapped because of development redesign of or marketing influence to change product scope and content.

Page 143 - Good conclusion on when to utilize manual testing.

Pages 172-187 - Good section on how, who, why, where to recruit QA personnel

Page 213 - Project and/or QA reports via intranet has been overlooked. This is a major oversight and a de facto standard for large, small, business, scientific, UNIX, and NT specific software companies on the eastern and western coasts of the United States.

Page 217 - The test plan has merit, but is generic (even if database or client/server is the intended audience). At a minimum, a sample software project with QA tasks and milestones should be provided.

Page 241 - The term 'trouble report' is not common on the west coast. I've heard the term used in the midwest, northeast, and southeast regions of the United States who use this term frequently. Similar terms with a more implicit recognition are "bug reports" and "software defect issues".

Page 244 - Suggest adding a "suggestion" to search the web for white box testing and/or black box testing. For example...yahoo...has a great search mechanism for finding specific concepts pertaining to QA.

Page 252 - Again, there's no mention of disk imaging (see early comment)

Page 326 - A minor oversight but BOOLEAN has been omitted from the list of variable types.

Page 329 - Why not use "exit for" in an if-else-endif error/validity check inside the FOR loop discussion?

Page 364 - A single page dedicated to test output/logging? What good are automated tests without concrete examples of effective reports, on the intranet, and updated daily, weekly, or at major milestones such as baseline, GUI freeze, function complete, alpha release, beta release, and etc.

Page 378 - Good section on cost analysis of automation tools. Surprisingly, few QA managers know about this tool for persuading upper/executive management that "you've got to spend money to make money" is a valid concern.

Page 418 - As stated earlier, there's no mention of a great code coverage tools TrueCoverage (VB and VC++) BoundsChecker (works with Rational Visual Test, Microsoft Visual Basic, and Microsoft C++) by NuMega.

And finally, a 'quality' issue surfaces when attempting to access the ATLM.pdf file within the ATLM_Graphics directory of the enclosed CD results in the following:

Disclaimer: No monetary sum has been paid for review of this book (ISBN 0-201-43287-0), and all remarks/illustrations in this document are to be construed as opinions of Keith Wilson. You are free to distribute this document as allowed by applicable law.