swap
the short answer. No. Nothing is different. as long as you run a 6 bolt wheel
the longer answer, and one you might find through a search is that one member found through a set of very nerdy measurements... no offense, they were impressive (they utilized drop lines and t squares im certain a micrometer was probably in there somewhere) whatever, anyways, this guy found that the offset was different and while the measurement was found to be more shallow the 20" wheel actually stuck out further. he was concerned about undue stress on the hub/bearing assembly. the general concensus was that no there was nothing that would cause damage. its no different than running spydertrax on your setup
as far as performance, nothing changes. a slight weight difference, so maybe nominal gas savings. but the tires are the same for both wheels. more rubber on 17s help with offroad and longevity.
all that being said. if you want to trade your 20's for my 17" limited then im all for it. i have 9 month old bridgestones with 5k miles on em. just throwing that out there.