Both mine were loose, I took the passenger side apart a replaced the foam around the inside of the housing, between the mirror and mount. It ended up being a ton of work stripped a screw... Blech. Cost maybe $20. That's why mine was loose, that weather strip type foam breaks down. No amount of tightning those screws will fix when that's what's up.
Driver side, I tied a slip knot to make a loop with some poly string about as thick as butcher twine, tied it tight as hell in the gap between the mirror and mount.. shoved it in snug. Took 2 minutes and $.02.
Now both are snug. Fwiw.