I had the same problem with a 3.4 V6 manual Tacoma I owned and it was indeed the IAC valve. I obtained a new one under warranty and changed it myself (I have a connection at a Toyota dealer). It's located on the bottom of the throttle body so that needs to be removed to replace it. When I removed my old one, I found one of the passages full of oil that I suspect was causing my problem and I figure it must have come from my K&N air filter?
With that being said you could remove your's and try cleaning it.