I got the loaded up limited because I know that I would want the doo-dads that come on it and grew weary over the years of buying the "cheaper" model and trying to then upgrade it to what was essentially the higher trim level over time.
Buying it loaded saves both time and money IMO. Just peruse these boards, and boards on other vehicles, and notice all the posts about "adding" this or that...wheels, spoilers, fog lamps, upgraded radios, nav systems, even steering wheels. I say save the trouble and buy it loaded to begin with *IF* that is where you want to end up. Then you don't have to waste time and money tracking down the parts and installing them or worse, paying to have them installed. Yes, you waste money buying parts after the fact, on things like inflated aftermaket prices, shipping and sales tax - and in my case (self-employed) actual time, which is worth a lot to me. IMHO, you cannot buy an SR5 or Sport and upgrade it to a Limited level for less than you can just buy a Limited to begin with.
Best advice I can give: Decide up front what you would like to have and what you have to have. Find the best compromise of those features, and let that drive your trim level decision.
For example, I had to have: leather, heated seats, sunroof, V8, 4x4 and the nav system. I found one with all those that *also had* XREAS, 18" wheels, etc and it wasn't that much more. It happened to be a Limited, I bought it. Had I found a Sport or SR5 loaded like I wanted at a reasonable price, I would have considered them as well - even without XREAS and the big wheels. In fact, I wanted 17s...easier to find tires for those IMO...but I am not a die-hard off-roader and I won't be trying to lift my ride to stuff big tires under it, so that was my 'compromise'. I made the right choice. We've just had a couple of snows and the all-purpose 18" tires that came on my limited have done just fine.