First you need to make sure it's indeed from the exhaust. I have isolated the sound to be coming from the manifold area. For my V6, I can only hear it with windows down. If you can hear it even with all windows up, it might be an tiny exhaust leak. There's an interesting theory coming from a 120 Prado owner sharing the same V6 with us:
Welcome to Land Cruiser Owners On Line
Basically, the headers on our 1GR is a simple 90 degree bend out from the head, and it can cause exhaust pulses interfering with each other, hence making that noise that almost sounds like an exhaust leak.