3.15: So does Booth move at some point?

In Season 3 Episode 15, Bones bursts into Booth’s bathroom and yells at him, and he asks “how’d you get into my house?” And she mentions the rock by his front door. Which implies that he lives in an actual house, as it would be kind of silly to use a rock as a hide-a-key in the hallway of an apartment building.

But later, every time we see where he lives, it’s an apartment. Does he downsize at some point? Why would he do that, assuming he keeps the same salary the whole time (or more likely, gets raises commensurate with inflation and performance) and assuming he doesn’t start gambling again offscreen? Season 3 started airing in 2007, was he a victim of the 2008 recession?