The oil film on metal starts to break down over time. Supposedly it takes about a month to fully break down inside a protected enclosure. But it's a sealed system, so it isn't like it starts rusting immediately. My wife is the primary driver of our 4Runner, and it sometimes goes months without 4WD (I honestly don't think she knows how to manually shift it). I try to run a few miles every now and then, and I do get to take her to play in the mud a few times a year, but rarely if ever do I get 10 miles/month.
The other issue is to keep the front diff locking actuator from getting stuck. Use-it-or-lose-it.
As for running on dry pavement, it's highly unlikely you'll break anything mechanical. Possible, usually because of some other undiagnosed problem, but unlikely. Your wheels will slip first. It does cause extra wear on joints and rubber, though.
The real issue is the fact that steering and handling are affected. No real danger driving slow in a subdivision or parking lot, or fast through straightaways or gentle curves on the highway. The danger comes if you're one of those people who drive a 4WD SUV like a sports car and try to take sharp curves as fast as you can. You go into the curve too fast, your tires start binding and slipping, and you hit the ditch (or a tree, or another car...).
