Pretty sure the SR5 Premium has always been Softex material. Only trim to have leather is Limited AFAIK.
Unless I'm missing something from the '10-'13 5th gens, but it doesn't make sense to do (SR5) Leather -> (TRD) Softex -> (Limited) Leather from a production cost persepective.
In terms of the OPs question, for me it was pure personal preference. Wanted the hood scoop and silver trim on the front. Everything else is basically the same, powertrain wise.
See the vehicle you want and go get it. You can't go wrong with either. Who cares if you never use the rear diff lock!?
The '10-'13 models had leather in the SR5 and waterproof cloth in the Trail Edition. In '14, the SR5 and Trail Edition became more similar and all switched to Softex. I had a '10 with leather and currently have a '16 Trail Edition with Softex. I prefer the Softex. The leather on the SR5 had different grades depending if it was a high wear area or not. Some pieces were really hard and others were soft. I didn't care for it.