Almost every body panel, except the glass, doors, tailgate and roof, is different on the 06 and up. They may not look too different, but they are, most notably the shape of the fenders and wheel wells. The fenders wouldn't match up, to say the least.
Its basically not possible.
But thats okay because, like me, you have the 03-05 body style which is far superior, and slightly reminiscent of the old 85-89 4Runners