I had this issue for years!
Thought it was suspension related, but replaced that and still had it. Checked my calipers and they were torqued to spec. Checked the slash guard underneath and that was set tight.
In the end it was my brake pad. About 3 years ago I replaced my calipers and pads all around. On the front left I install 2 of the little springs to hold the brake pad, and for some reason only 1 on the right side. I think they are only suppose to need one, but it turns out that with out a second one the pads can slide up and down in the caliper just enough to make a "clank" sound when going over bumps.
I tested this out by hitting the same bumps with my foot on and off the brake. I only heard the clank when my foot was off the brakes. Then a few weekends ago, when working on the Runner, I had the wheels off and was able to grab a pad with my fingers, while still in place, and slide it up and down. Sure enough it produce the same clank sound.
I just installed
both of these on the front right pads. The guy at Autozone said the one I had was needing replacement, so maybe just one fresh one would be enough, but it seems fine with both, and my left side has had 2 for years.