Jason, don't think it's on your end.
I've used the adapter on a previous 4runner and noticed a bit extra flop using the adapter.
For my current 4runner I bought a 13" blade and carefully swapped just the rubber blade and both stainless steel stringers to the factory mount.
It's doable. Just...