The biggest drawback to swabar removal is on-road/higher speed handling. Many (self included) think that it may even be hazardous for all in an emergency maneuver. I would also argue the keen, aware, prepared driver may be fine with it. To me it's all about the optimal minimization of risk while still achieving the desired goal. If you do take it off make sure you and anyone else who drives your 4Runner is prepared/briefed.
With that 'disclaimer' out of the way: Many who run higher rate springs opt to remove the front swaybar. The stiffer spring offsets the lack of bodyroll control due to the loss of the swaybar. The advantage is a better ride and better articulation of the front suspension.
I have slight stiffer springs and drove mine for a while without bars (in different combination). In the case of the 4Runner the rear bar is pretty thin and doesn't harm the ride or articulation that much. The front bar on the other-hand is pretty stiff. The best combination of ride/handling etc with an off-road bias is no front bar with the back bar in place. With both bars removed my FJC springs couldn't keep up; it was way too rolly. With just the front bar removed the on-road handling was still tolerable and not scary provided you are a speed-limit +10 (or less) driver. If you push harder than that the excessive bodyroll becomes problematic.
I would say try it provided you drive easy and only careful, capable drivers use your rig. I know you'll love it off-road and on-road at lower speeds.
If you don't like it I think the swaybar relocation may fix your issue for very little $$$.
:cheers:
