You can try spraying lube in there and see if it knocks things loose. Key might not be turning far enough due to crap build up.
If that doesn’t help , I would pull the key lock assembly and inspect the mating end for damage.
You need a T-27 torx bit and plastic prying tool. Pry off the black grommet at the side of the door , next to the key lock assembly. there’s a torx screw under there that needs to be loosened. This screw does not come out, just loosen it as far as you can. Then pry at the cap on the outside and the assembly should wiggle out.
Here’s some visuals from my door handle tutorial that should help.
https://www.toyota-4runner.org/2729778-post5.html
__________________
2004 4Runner Limited V6 4WD ***SOLD***
385,000 Km
Titanium Metallic
Vehicles: '21 Highlander XSE V6 AWD | '21 Acura RDX SH-AWD Tech | '03 Honda CBR600F4i