I had a very similar issue a while back! Thought maybe it was my rear control arms so I replaced all 4 and it got a little better, but not solved. Ended up getting my wheels and tires balanced and then rotated because I was was having a vibration issue, and that resolved it for me
I was boggled but happy the problem was gone. Might be worth a shot!