So you have a VF3AM case. I'm sure someone will correct me if I'm wrong, but here is my understanding of it.

You have 3 position switches in the system, 2 in the case and 1 at the front diff. You also have the electronic actuator. Your 4wd and center lock buttons both send signals to the 4wd computer which then fires up the actuator either once for open center operation or twice for locked. The switches tell the computer when that actuator has moved the appropriate parts to where they're supposed to be for the selected mode of operation.

Your lever simply moves the internals from high to neutral and into low. There's another sensor internal I believe that signals 4 low.

Usually when my car is being picky, I go straight to low, then back to what I want. Think my actuator is on the way out. If it's slow no matter what, I would suspect the same for you.