I build diffs for a living, if it was me I'd pop out the 3rd member, grab a ton of brake clean and spray everything out and if you have compressed air then blow it out. (Bearings etc.) As long as everything turns over fine you're good. Honestly if you really dont want to drop the diff then change the fluid every month untill shes clean. Diffs are fed lubricant from the ring gear, if there was a lack of lubricant due to low fluid there was more than likely metal transfer from the ring and pinion. This would explain your shavings. As long as it's not running loud aka you can hear abnormal noises from rear end while driving then the metal transfer wasnt bad enough to cause any type of crazy damage or high spots causing knocks. The other thing that can happen is when you initially put new fluid in it, it could have cleaned up the inside of the diff which already had a bunch of metal floating around because the previous owner never changed their fluid. What messes up a ring gear and a pinion is when you change the mating teeth. If you never remove the ring gear from a carrier and reinstall it to the same pinion a diff will last untill the bearings go or the metal goes enough to the point where a tooth chips, the toyota OEM gears are heat treated like any other but also have thin teeth on the ring gear, this allows more of the total metal on the tooth to be hardened. If the guy you bought it from took the ring gear out to inspect everything then re installed it there are now new teeth wearing at abnormal rates due to never being meshed before. Once they ware and the teeth become reaclimated to one another then they're fine. There is no true mileage limit on gears, OEM gear sets cannot be put into a mileage box. After markets for sure can be though. Diffs are one of the simplest and and bulletproof components of a vehicle. If you have no noise on the diff dont even worry about it. Just my two cents.
