You can tow that trailer, with a few things to consider. 1st, it is likely that you only have the standard V6 hitch receiver, which is NOT rated for any type of weight distribution towing system. I can't say for certain with the '11 models, but the earlier 4th gen's had different hitch receivers for the V6 and V8 packages. If the owners manual rates your TM as max 5k towing and max 500 lbs tongue weight, then it is probably the lighter duty V6 version. In order to use any type of weight distribution towing system, you will need to upgrade the hitch receiver to a frame mounted model. Doing this will allow you to use weight distribution towing systems, and may increase the overall tow ratings (depending on the specific receiver hitch you get). You mentioned installing an aux trans cooler, and that is a very good idea.
All that being said, the 4Runner is a short wheel base SUV, and isn't the ideal tow vehicle for heavier loads. As you approach the max ratings, you can expect the V6 to work, expecially on hills, to pull the load. Trailer sway can be an issue, and can be made easier to control with the right type of sway control device. Last, but certainly not least, is towing experience. The more experience you have towing heavy loads, the easier it to manage towing heavy loads with the 4Runner. Not to say it can't be done (as my sig indicates, I occasionally tow a 26ft, 5,600 lb camper trailer), but I have done the necessary upgrades, and take it easy when towing it.