For me, the biggest reason would be the Xreas and full time 4WD. The other stuff, while nice, is more or less fluff that comes down to personal preference, IMHO. The biggest reason I see to go with the nav system is the larger screen for the back up camera vs. the small screen integrated into the mirror. However, the differences in the drivetrain and suspension are worth the upgrade. Like many people, I am looking to reduce the amount of nose dive and body roll of the SR5 through upgrades (so the dollars may end up being spent anyway) but, unlikely many people, I do not want any kind of lift through suspension modifications.
If I had thought things out better and had a chance at a better, more thorough test drive, I would likely had gone with a Limited. However, this is the most expensive vehicle I have ever purchased (by $12K) so I wasn't mentally ready to spend an additional $6K at the time of my purchase. Second guessing myself now.