That's exactly what happens. As an object moves through the air it displaces it and a vacuum is created behind. The air that was displaced has to come rushing back in behind. That's why a wing tapers to a point at the rear, that's why trash always gets sucked behind in the wake of a semi, that's why a boat tapers at the rear (water and air are both "fluids" in this case)...a million examples are possible.
You guys don't smell it? It's really bad with the windows up and the rear hatch open - how could you not smell it? If you're driving with the rear window open you should have all of the rest of the windows open. That way air can flow through the vehicle and fill the vacuum and keep most of it out. If I'm going slow off road I'll sometimes run with ALL of the windows open including the rear so the dogs can hang their heads out the back but not at anything above 20mph and certainly not if I had a small child in the back.
Re-aiming the exhaust out the side won't really help at all, the fumes will still get sucked in. A pipe with a downward turn in the tip that points to the ground is most effective at reducing it but won't eliminate it entirely.