I will put mine down as often as possible. I'm lucky enough to be able to take my dog to work with me so that was a huge factor in purchasing my 4runner. He loves sticking his head out the window and letting the fresh air come through. Plus its funny to watch people wave at him and try to take pictures while they are driving.
I always put my other windows down if I have the back open. Like was previously stated the exhaust tends to recirculate back inside so I don't want to end up poisoning myself and my dog. I would like to find out if anyone has had luck with a turn down exhaust fixing this problem. I have been debating adjusting the exhaust exit point to see if it fixes it.