Top critical review
5 people found this helpful
A short guide to running website experiments
on 6 September 2012
Please note - this is a Revised Review (see my note below).
If you're unfamiliar with this type of material then you will learn something. It's a good starting point on how to run quality, meaningful experiments on your website(s).
My recommendation: Buy it if you're new to this field. It's a useful (short) introduction. It's also useful if you need to strengthen your case for 'proper' experimentation in your team or organisation.
When I read the title of this book, I pre-ordered it immediately. Considering what I do (digital marketing) the book synopsis sounded great. And I've waited, what seems like, an age for the book to be published.
You can imagine my delight when the mighty Amazon informed me that the Kindle ebook was ready for download! I couldn't wait to start reading it and absorbing the content.
This didn't take long. Absorbing that is.
I think I misunderstood the aim of this book. I *thought* it was a 'How to Guide' to Website Conversion Rate Optimisation (CRO).
It is. And it isn't.
It's actually MORE about how you run effective experiments including A/B and Multivariate tests. Obviously, these topics are intertwined. But the book wasn't what I thought it was going to be.
The book is short. I think it's about 157 or so pages (including introductions, index etc). You don't, therefore, get a huge amount of material. Material that is, as I see it, flimsy and less than useful. Indeed, on many of the topics covered (A/B Testing, Multivariate Testing, Google Website Optimizer and so on) I've read much more in-depth blog posts.
Now, I know the book isn't a 'How to Guide' for individual testing tools. For me, at least, I thought the book oscillated too much between topics as diverse as UX Design to 'Bravery and Responsibility'. And, whilst I acknowledge it's not meant as 'a bible', nothing is covered in sufficient depth. Nothing is explained in enough detail and - if you have any experience in this field - nothing is said that adds huge value.
In essence, I scanned through this 'book' (it feels like a rather long essay) and, unusually for me, highlighted nothing and took no notes. Which is a real shame because the newly emerging field of CRO needs more solid resources for people to use. Unfortunately, whilst this book will be a great starting point for many, it left me disappointed.
Finally, I'd suggest you'll still need other books to complete your CRO Library e.g. Always Be Testing: The Complete Guide to Google Website Optimizer by Bryan Eisenberg and John Quarto-vonTivadar.
Why I Revised My Review:
My initial of this review was more critical. However, via LinkedIn and for various reasons, I'd made contact with the Author (Colin McFarland).
He took considerable time to respond to my input, here on Amazon.co.uk, and made several valuable comments. I greatly respect and appreciate his engagement with his readers. I now feel clearer about the purpose of the book and have, therefore, tried to clarify my review of it.
Nevertheless, I still feel the book was too short and didn't provide enough (for me at least) substantial material or alternative perspectives. Hence my 3-star rating.