This was my life at the beginning, I wanted a GD Horse for crying out loud!!!! So I wandered around the castle in Whiterun many times taking anything that wasn't nailed down, got my horse. Mission Accomplished.
So back when I first started playing, I became a member of The Companions, and realized I could take all the stuff out of the main hall. Realizing this, I proceeded to take everything I could, and then walked over to the shop down the way. Realizing it was just slowing me down and wasn't worth shit, I proceeded to dump it all in front of the shop. To this day there's just a giant pile of clothes, dishes, and cheese wheels in front of the shop.
Gold ~600. ---Steal everything edible for me and my mercenary follower whenever I defeat bandits or other baddies.
Why? because all our money goes toward arms and armor and healing potions.
Level 72. Gold ~200,000 ---I note that I am still stealing everything edible to provide for myself and the same mercenary follower who is still with me...AND an human adopted daughter...AND an adopted orphaned Khajit cub (thank you, mod makers!)...AND my housecarl...AND my steward...AND my bard...oh yes, AND my Argonian wife.
Why? Because the cost high-level arms and armor followed my income, Grand soul gems for top-level enchanting make my wallet cry, my household has quadrupled and supermarkets haven't been invented yet!
Not just applicable to Skyrim. Lots of games with theft mechanics follow this. I for one am fond of playing Spelunky and absolutely cleaning out the first few merchant shops I come across and ignoring the rest.
... On that subject, I have a dumb little theory about why the hell the Dovahkiin is such a klepto. The character is the Dragonborn, right? And dragons are known to hoard treasure. Well, what do general goods traders call the random nick-nacks you pawn to them? That's right, treasures~
Sometimes I think it's harder *not* to find yourself in Breezehome at some point of the game. Most advanced of my characters had four homes, Breezehome one of them. Stashed there Cicero's motley, Savos Aren's amulet and countless of other things. From time to time he has to tour all around Skyrim, visiting his abodes and looking for some piece, most desired at the moment. Wabbajack, however, is still nowhere to be found.
xD The only reason I decided not to be an Argonian in Skyrim (though I was one for a very short time) was because I figured he/she'd be freezing their tail off. Otherwise, you basically get to be a tiny, bi-pedaled, wingless-dragon that can breathe underwater. What's there not to love?
On my first playthrough, I was an Orc. Now, I'm a Bosmer.