Yes, it's the combination of worn contacts and plunger that caused the starter to stick on. The same thing happened to my truck and replacing the contacts and plunger fixed the issue.

Here's my theory of why, mainly worn contacts, causes the starter to stick on. There's a small and large copper contact in the solenoid that the plunger makes contact with. For whatever reason, the larger contact wears much faster than the small contact so there's a deeper grove created in the larger contact. When you turn the ignition, the plunger goes down to make a connection with the two contacts. Because they are at unequal levels due to the uneven wear, I believe the plunger can sometimes get wedged or cocked between the contacts and get stuck. It's the only thing I could come up with that made sense to me. Maybe somebody else has a theory on this but I do know replacing the contacts and plunger fixed the problem.

BTW, I now keep a channel locks inside the vehicle in case this ever happens again so I can quickly pop the hood and twist off the negative battery cable to stop the starter.

Here's a video I made for this repair. I hope it helps you out.