Customer Review

4.0 out of 5 stars Data appears reasonable, 3 Feb 2013
By 
Verified Purchase(What is this?)
This review is from: Zeo Sleep Manager Pro+ Mobile - Watch, Monitor and Track Sleep, with SmartWake Alarm Clock (Electronics)
This seems to be the only device of its kind other than a proper medical version costing 100s of times more. However it's still a very expensive gadget if it doesn't work! I would guess this isn't the sort of thing that would appeal to a hypochondriac. Some of the reviews for this and its sister product (with a base unit) appear, like me, to start from a sceptical viewpoint; not written by the sort of people who would buy a cream that would ensure they would never age despite the claims of clinical trials to the contrary! I have some technical knowledge which I hope to be able to add to the debate.

I will begin briefly with why I purchased the device. I don't generally have trouble in getting off to sleep (I have developed an almost foolproof technique) but I do sometimes get woken by low-level noise, heartburn or other minor distractions and then can find it difficult to get back to sleep. This is especially true when I have something on my mind. This can be something pleasant or something worrying or stressful. Either way, if my mind becomes active after awaking, I can have trouble getting back off. I also like an alcoholic nightcap to go to bed with (note, this is not my foolproof way of getting to sleep)! I'm aware this is claimed to help you get into a deep sleep can disrupt sleep patterns later in the night. I was interested to know if this was true in my case. I would be able to experiment with and without a nightcap and see if there was a difference in my sleep pattern. Why give up something you enjoy if it's not a problem?

Before purchasing, I looked at the Zeo website where there was a fair amount of information from various sources. There were two abstracts from `clinical trials'. While they gave the appearance of being scientifically written (for example quoting means and standard deviations) one was co-authored and supported by the device's manufacturer and neither claimed to have been peer reviewed, that is submitted to a respected journal after being reviewed by other scientists in the field. Following publication they are then subject to informed criticism and quoted by other scientists. These papers attempted to correlate the data from a Zeo device against clinical methods of sleep evaluation; however the sample size in both these papers was worryingly small. A third paper did appear to be from a journal but the link to it was broken so I was unable to check. So far not totally convincing but not off-putting on its own.

There was also a blog by someone from the company about how the Zeo works. This blog discusses Fast Fourier Transform (FFT) filters and artificial intelligence in the form of neural networks (apparently the device's `secret ingredient'). Now as luck would have it, I have used these techniques in my job as head of forecasting hour by hour demand by customers of the energy company for which I work. I therefore felt competent to review how appropriate these techniques really are to their task here. If these terms were used as flannel to bamboozle the reader, it wasn't going to work with me.

So let's start with the first problem, that of getting brain signals into the device. The device works with a headband which has electrical contacts to pick up electrical activity from the brain. There are two difficulties. The first is that the signals are very weak and need a massive amount of amplification. The second is that the signals are `noisy' (there's lots of other electrical activity both inside and outside our bodies) and the amplification amplifies the noise as well. Think of a large party of, say 200 people in a room, all chatting. You wish to hear what someone on the far side of the room is saying. This person has a slightly distinctive voice however it's clearly impossible to hear what they say at that distance with the high level of competing noise. Imagine you had a directional microphone and some electronics that could filter out all sounds other than those that were `distinctive'. Might it then be possible? Well that's what the device does, using the FFT to filter the noise and try and get a useable `signal'. Let's assume that bit works; it's probably not rocket science with fast processors and the approach is entirely reasonable. The next stage makes some assumptions from sleep science 'experts'. It uses data that claims certain `frequencies' of electrical impulses are suggestive of the stages of sleep. I have to take this as gospel as I've not researched the literature to find out if this is true but it does seem to make sense. The manufacture's website contains some links so the reader could check this aspect should they wish.

The next stage is to look for patterns in these key frequencies and the Zeo claims to use neural network technology to achieve this. I'm not clear why the frequency data extracted above can't be used on its own to determine the sleep state; there appears to be a direct link between sleep state and brain-wave frequency and these are available without the need of a further process. In my professional life I have looked at neural networks at performing different tasks. They are very good at some (like clustering) or pattern recognition. They also need human experts to `train' them and their efficiency at doing their job is directly related to the skill of the person doing the training. My biggest concern is that the neural network may be blending the filtered data discussed above with `typical' sleep patterns in a way which could lose information from the individual being monitored while making the result look plausible. In summary, I'm prepared to accept that the neural network may be adding something useful to the process compensating for the fact that the filtering output is still very noisy. On the other hand, it would have the potential to make the output data look `sensible' even while being inaccurate.

To summarise so far, the theory and statistical tools employed are not totally bunkum but without additional compelling evidence of accuracy, would not on their own convince me of the device's competence.

My next task therefore was to try and test its accuracy. Now I said above that the papers claiming the device works had small sample sizes in my view. Well my tests have a smaller sample still; one. I can defend this as I'm attempting to prove whether the device works for me personally and not the population in general. I have thought about a few ways of testing the device. I could only think of three that I was competent to perform and one of these had to be ruled out for practical reasons. In summary my three tests were:

1. Test if I am awake when the device thinks I'm asleep (and if it gets my `time before falling asleep' correct).
2. A qualitative test to correlate whether my personal view of `did I get a good night's sleep' is similar to what the Zeo reports about my sleep quality. This test has a practical flaw for me though. To be impartial, I'd need to carry out the test without checking the data from the device for a reasonable period. If I looked at the data each morning, there's a chance I could `calibrate' my thoughts in line with what the machine thinks in the same way that a placebo can be seen to cure some ills with no active ingredients; the mind can be powerful and its owner not always in full control! If I was to carry out the test, I'd have to avoid looking at the output until the test was complete. As a confirmed gadget freak, I certainly don't have the patience to do this and my curiosity to examine the results after my first night ensured this test would need to be scrapped.
3. I would be able to perform the `nightcap' test by evaluating my sleep patterns with and without an alcoholic nightcap over many nights (with controls to ensure other aspects of each night's sleep were taken account of as far as possible). I have the statistical skills to do this if I can collect enough data.

So test 2 is ruled void and test 3 is underway but will take time to evaluate.

So starting for now with test 1,did it accurately record my wake time? It might at first appear easy to determine when you are awake but assuming the device was making its assessment only from brain wave patterns, it's reasonable to suppose (although not certain) that this is no less easy than determining other sleep states (e.g. deep, light, REM). So, starting with my first night I checked my `time before I was asleep' statistic. Now, I was listening to an audio book as I went to sleep on a Squeezebox player. In the morning I was able to check the elapsed time to the last point in the story I could remember; I calculated I fell asleep after 16 minutes (plus or minus a minute); the device had be asleep after 7 minutes. I also remember waking twice, once for an hour or more, and looked at my alarm clock both times with a view to check these times in the morning against the machine. The device significantly under-recorded the time it thought I was awake (it did get it right when I went to the bathroom). Having got used to the headband by the second night, I tightened it somewhat in case the contact with my skin was poor. The data on this occasion and for the following few nights has shown a far better agreement with my own observations - in fact to a quite impressive degree.

The Android version of the App works well but as previous reviews have noted, does not allow you to update your diary, one of the reasons why I'll not award 4 stars.

After a few nights use and with lots more trials to conduct, my intitial reaction is that the data certainly appears reasonable. I'm warming to this gadget!

Regarding the headband, be the second night (after my wife's hysterical laughter had almost abated) I was far less aware I had it on and by the third night, it was almost completely un-noticed. I do feel the replacements are very expensive, another reason for awarding only 4 stars.
Help other customers find the most helpful reviews 
Was this review helpful to you? Yes No

Be the first person to comment on this review.

[Add comment]
Post a comment
To insert a product link use the format: [[ASIN:ASIN product-title]] (What's this?)
Amazon will display this name with all your submissions, including reviews and discussion posts. (Learn more)
Name:
Badge:
This badge will be assigned to you and will appear along with your name.
There was an error. Please try again.
Please see the full guidelines ">here.

Official Comment

As a representative of this product you can post one Official Comment on this review. It will appear immediately below the review wherever it is displayed.   Learn more
The following name and badge will be shown with this comment:
 (edit name)
After clicking on the Post button you will be asked to create your public name, which will be shown with all your contributions.

Is this your product?

If you are the author, artist, manufacturer or an official representative of this product, you can post an Official Comment on this review. It will appear immediately below the review wherever it is displayed.  Learn more
Otherwise, you can still post a regular comment on this review.

Is this your product?

If you are the author, artist, manufacturer or an official representative of this product, you can post an Official Comment on this review. It will appear immediately below the review wherever it is displayed.   Learn more
 
System timed out

We were unable to verify whether you represent the product. Please try again later, or retry now. Otherwise you can post a regular comment.

Since you previously posted an Official Comment, this comment will appear in the comment section below. You also have the option to edit your Official Comment.   Learn more
The maximum number of Official Comments have been posted. This comment will appear in the comment section below.   Learn more
Prompts for sign-in
 


Review Details

Item

Reviewer


Top Reviewer Ranking: 18,594