I'm pretty sure I've seen some few 4th gens and a couple 5th gens that have done it, but I don't remember seeing them on the forum (I think they were all mainly offroad shops or professional wheelers, could be wrong though.)
The biggest issue is going to be the ECU and associated modules. Just off the top of my head I forsee issues with the following:
-ECM
-Skid Control
-Body Control Module
-4WD ECU
-Combination Meter
That's assuming you're talking about a turn key 4Runner, if you're talking about a newer '20+ that are almost all smart key only, you'll have to deal with the Smart Key/TPMS ECU, the Certification ECU, and Gateway module.
Much of the above can be bypassed, swapped out with FJ parts, and/or will need piggy back modules to "correct" problems. Though at that point, you'll be better served going standalone and making peace with losing some features.
It's doable, but it's a big job to do it right and retain as much of the factory feature set as possible. I mean someone managed to dump an entire 5.7L powertrain from a Tundra in a 5th gen, so I don't see what a manual couldn't be done. It's just a matter of money, time, and effort; all of which will be on the "very high" category.