Exact same problem
Your description s sounds exactly like what is happening to my '05 SR5 - started about a month ago and has gotten noticeably worse. As with you, it doesn't happen when I accelerate quickly.
I have a good friend that is a mechanic for a toyota dealer and he and a coworker...