Handling and cornering with the T4R is very good considering what it is (a high CG mid-sized SUV). I think you will find the power of even the 6 cylinder version a noticable improvement over the RAV4. I personally own a V8 4WD, which in actuality is a 40%/60% (Front/Rear) all wheel drive, there is no 2WD mode in the US-spec V8 4WD T4R models. This AWD (full time 4WD is more accurate) combined with the V8 makes it a hoot to drive, and again, for its height and weight, it corners quite well. I got to drive a V8 2WD model while waiting for mine to come in to the dealership and found it to be every bit as nice a driver, it rode a tad better (most 2WDs do), but in the seat of the pants, the 'less weight' didn't not appear to make it feel any more "powerful".
When you take a T4R offroad, it is very solid. I've had mine off-road several times since my Dec 08 purchase and it has not let me down. Took it out in a foot of snow back in December and never even had to alter the default F/T 4WD settings...never got stuck, never even kicked in the VSC.
Having owned a similar Hummer (H3 Alpha AWD w/5.3L V8) and several other 4WDs and AWDs, I can honestly say that about the only *other* off-road vehicles/SUV I'd compare to it are my friend's LandCruiser, and an old CJ Jeep I had back in the day with a nasty 360cid V8 in it.
If on-road sportiness is your primary goal, there are better options out there...but will cost you more. Like the BMW X5. That's a great onroad SUV, and one that I'd never take off-road - or certainly never with its stock tires and ground clearance.
I read a recommendation for the Acura MDX - I cannot recomment that one. Very good friend of mine had one for about 6 months and hated it, as did I. Decent build quality, ride and handling but they are grossly underpowered. Don't go by the horsepower ratings, look at the torque. MDXs have highish stated horsepower numbers but are wimps in the Tq department IMO - especially low end (low RPM) torque, which is most useful in the case of heavier vehicles like SUVs...not to say that my friend didn't ditch his MDX for another Acura, he did, he just got out of SUVs entirely and went to the TSX...my V8 T4R has him wanting back into the club though. I told him to hurry, the current US administration is trying to kill any vehicle that doesn't get 100mpg.
Go test drive a T4R, drive your RAV4 to the dealership so you can get right out of it and into a Runner - then any drive/ride/power differences will jump right out at you.