Pathfinder?
Let me throw this out there: Since Nissan products seem to be on the table, have you considered an R51 Pathfinder? This would be the '05 - '11 or '12 model, I believe, which was the last Pathy to have a true body-on-frame construction (the new Pathy is a unibody crossover.)
I think the Pathy is an underrated vehicle for the kind of use you're talking about - mild off roading with true 4wd and low range. Now the Pathy has an independent rear suspension, which makes lifting both more expensive and more difficult, but if you're not planning to lift it, that shouldn't be an issue.
Complaints about the Xterra's "dated" interior are misplaced IMO. An Xterra isn't supposed to be a luxury vehicle, it's supposed to be a Spartan and rugged off roader. The Pathfinder is a better comparison to the 4th gen 4runner.
The R51 is a bit bigger than the 4runner and has 3 rows of seats. It has a V6 that is more powerful than the 4.0 in the 4runner, although less powerful than the V8. There was also a limited model of the V8 Pathfinder with full-time 4wd, although those are somewhat hard to find.
As for the comments regarding build quality, just remember, this is a Toyota board.

: Wife drives a Nissan and it seems to be as well built as anything else these days.
One of the biggest advantages of the 4th gen (2003-2009) 4runner, IMO, is the V6 with the multi-mode 4wd system. It gives you the option of keeping the vehicle in 4wd when you are on intermittently icy or snowy roads without having to worry about damaging the drivetrain. No Nissan offers this, to my knowledge.
To be honest, I was close to getting an R51 Pathy but it was the multi-mode that finally convinced me to stay with the 4runner.
On the 4runner with the V8 you get full time 4wd rather than multi-mode, but this will cost you in terms of MPG, which may or may not be a factor for you (lots of folks here will say "get the V8, the mileage difference is insiginifcant and then sure as hell, 6 months later they're on here complaining about the crappy MPG they're getting.

)
If you're bound and determined to stay below 70k in miles and below $20k in price, you're going to have a tough time finding a 4runner that fits into those requirements.