ahhhh I only removed the fender flare; not the entire fender (sorry.. my bad). Check the shop manual (it's posted here on the site) for the step by step instructions to remove the entire exterior panel.
EDIT: http://www.toyota-4runner.org/5th-gen-t4rs/66572-2010-4runner-collision-repair-manual.html
Okki