I have tried to answer your question correctly , but I simply cannot remember all of it. First, what I am about to say is for an SR5 and I think the TE is the same. Limited is different. Not sure what you have.
Take off the back plastic door panel. Separate the plastic film from the door frame and look. I remember that you will have to take off the lock mechanism to get the garnishment off. I think there is an electrical connection to the the lock mech and it is "separate" from the garnishment. I am pretty sure you can replug in the lock mech, then temp attach it to the inside of the door panel. You will not be able to use your key, but you can lock and unlock using the key fob buttons or the inside door lock buttons.
What I can't remember, however, is the door handle. It's under the garnishment, but not sure if it came off with the garnishment or if you had to remove it to get the garnishment off or what. I simply forgot. If not, by simply plugging in the lock so the latch solenoids will work again, you should be good to go. If you have to remove the handle, then?
If you do a search here about removing the rear garnishment, I do remember multiple threads with some pretty good instructions. They might mention that handle.......just a thought. Be Safe.