I have this problem on my 2003. I purchased it used not too long ago so I don't know how the previous owner took care of it. But it is quite serious for me. The metal around the rubber seal is all rusty and is flaking off. The locking mechanism and motor that pulls the door shut failed and needed to be replaced, these parts are many hundreds of dollars. This fact was known before I bought the truck and the place I purchased the truck from sent me to Toyota and another mechanic shop to get it repaired for me on their charge. So that was good...But now that I noticed all the rust under the rubber seal, I bet that water had gotten all the way inside and also rusted these components out.......Because.....Another issue I have now is that the metal clamps that hold up the rear window and hang on to when it raises and lowers is also rusting. I did not know this, I didn't even know my rear window lowered when I bought the truck. Until..... My left side pretty much rusted right off and the left side began to droop down. So now I have 2 make shift hook contraptions that I fashioned to hold the window up semi-permanent and the window can not be lowered.
I recommend to take a close look at your rear tail gate windows, pull back the rubber seals and check for rust. Also remove the inside plastic cover and inspect for rust. Remove the bolt where the pull down strap is attached and the rest is just clips you can pull off. Then you will be able to check if any of the internal components are rusting too. Spary all the metal parts with rust proofing/protection oil just in case anyway.
I didn't notice it when I bought the truck, and so the place I bought won't do anything about this problem. So I'm going to have to fashion a more permanent solution to keep the window permanently raised up, to avoid about 500-1000$ in repairs. The component is about 200$. there are 2 of them, add labour 2-3 hrs and possibly the motor too, as per an estimate i got.