Glendening's Icelandic is a staple for a serious learner of Icelandic. It appears at first to be very formal and academic, though on closer inspection it's not an unfriendly text.
It does, however, suffer from its age. The book has not been updated to reflect the abolition of the letter 'z' from the Icelandic language, and the usage notes describing formal patterns of Icelandic are a bit outdated.
This book demands to be read carefully: important concepts aren't always emphasised and are on occasion even only noted in a single sentence. It's also fairly heavy going at first, so strong motivation is required, though as concepts start falling into place it becomes easier.
Furthermore, the language it teaches is more scholarly than conversational. This isn't too much of a problem, though, since the grammar is the same in both cases. You'll need another source for conversational vocabulary, though: I recommend the (free) Icelandic Online course offered by Háskoli Íslands and its companion texts (including dictionary) at the University of Wisconsin's digital collection (which is available online to anyone).
This unfortunately can not be appropriately called a 'teach yourself' book, however much it tries, but if you're looking for an affordable grammar reference and don't mind compensating for its age (which isn't too difficult: mainly change 'z's into 's's and be less formal than it suggests), it's a good book. The score I've given it reflects this use, rather than its apparently intended use.