Not quite as daunting as a big box expansion like Dunwich Horror, the small box packs offer a lot of variety and a few extra components to integrate into the core set.
King in Yellow gives you lots of options, most of which make the game harder. You get investigator items and extra Mythos cards, which you can just shuffle into the main decks. But you also get a few interesting aspects that change the game dynamics.
Most significantly, the Yellow Sign tokens are added to the board each time the terror level increases. This subsequently further increases either the doom track or the terror level, with the possibility of 'blights' appearing in Arkham. These blights hamper your team's progress in a number of ways, such as increasing the price of items or altering investigator stats.
There are other optional changes to impose on your game, such as the Act cards, which add a sort of time limit, and, of course, the King in Yellow herald. Heralds affect the game throughout, and give games an extra layer of theme.
If you've played a few games of Arkham and you're looking to spice things up a bit, but don't fancy the commitment of investing in a big box expansion, then King in Yellow is the place to start. I also recommend looking into Lurker at the Threshold, which adds gate bursts and relationship cards.