sounds like you've done a pretty good job of diagnosing it as a door or window sealing issue....
you could test the door seal by putting baby powder on it and then shutting the door, open the door back up and see where the powder isn't sort of thing..
window seals are a different beast... 2003-2009 corollas a prone to faulty window seals that were too small and cause massive amounts of interior noise. I know because I own a corolla in this range... the solution there from toyota was a recall on the window seals. something to think about... this thread talks about the corolla issue:
Need to replace my rubber window channels - Toyota Nation Forum : Toyota Car and Truck Forums
good luck