I had this also, and in my case it was a plastic screw spacer thing. Basically that handle unit is attached by one screw going from the inside and through the door. It goes through a threaded plastic piece that acts as the female and a spacer...that plastic piece was broken. I swapped mine out with one from a junkyard. Pretty easy switch, just remember to swap out your lock (it's pretty simple when you look at it). I believe the screw is behind a round sticker when you take the inner panel off. I don't remember taking 2 bolts out from the outside.Inside or outside? Inside is one screw and a clip that flips off. Outside is held by two bolts, You have to remove the inside door panel to access. Most likely your bolts have backed out