I had a similar problem on a different vehicle, it was the tires. I replaced just one tire and it resolved the problem (on that vehicle.)
A shimmy like yours when stopping (only when stopping) is usually warped rotors. Get them turned or replaced.
Idler arm: Good possability. Mine is bad, I need to get one. My truck does not shimmy or sway at all, but I DO get a clunk when steering under load or off chamber.
Where to find your idler arm:
Look under the front of your truck. On the driver's side, there is a shaft that comes from the steering box. This attatches to the rod that goes between both front wheels. The rod is the tie rod. On the passenger side, there is a connection that looks like the one on the driver's side, but just bolts to frame and pivots; THAT is your idler arm. If the bushings inside it are worn, or the pivots where the tie rod attaches are bad, that could likely be your problem. Easy test: jack up the front of your rig, both wheels off the ground. Support it very well with jackstands. Wiggle that rod and other parts. If they are bad, you will be able to tell easily. Looks for rust, a tel-tale sign of a shot joint.
Idler arms can be rebuildable, depending on what year and kind they are. Mine is just shot where it connects to the tie rod, but that joint is not replaceable. Shop around for the replacement part, Toyota wants almost $200 for one, NAPA wants 90 or so, Checker has one for $45 bucks. Also check Car Quest if you have one, NAPA and they use the same suppliers, same parts, even a lot of the same part numbers, but Car Quest is much cheaper.
One thing tho, it is an easy job (relatively) to replace the arm yourself, but you still want an alignment done afterwards, so it may be worthwhile to just have it replaced...plus a warranty is always a good thing to have!