Sure it's a solid Toyota and a stock 4runner will take you pretty far off road, and pretty nicely on the road as well.
But, as a father of grown children, I have a good idea of what your dad is saying and why. Your in college, just starting out in life, your dad has already been there and done that so to speak. He is just trying to instill in you a sence of being practical, parents want their kids to learn from mistakes the parents have already made. I won't pretend to know your situation but most kids in college get some sort of finacial aid from their parents, either with school, room/board, a place to live in the summer, and/or, often with the car they drive and the insurance. I know from my own experience, when one of my kids was in college, the last thing I would want to see is them taking un-necessary risks to a vehicle that is needed for them to complete school, even if it was a sturdy 4runner. I'd feel the same weather I helped buy it for them or they did it on their own, your in college, finish it and get on with your life/career. You have plenty of time to wheel your truck after school if finished.
What your dad is saying is: don't let you brain write checks that your wallet can't cover. My job demanded that I have some sort of 4X4, I didn't have to climb rock piles, just get in and out of some rougher than normal territory. I had a 1977 Bronco slip into a lake, nothing crazy, it was just real muddy and steep. My 85 Toy PU got stuck so bad that I had to leave it, it froze and the truck had to stay out there until it thawed. My 92 runner had a CV boot ripped by a sharp corn stalk, and another by a piece of bailing wire wrapped around the axel. The passenger door got ripped by a piece of metal sticking out of the dirt on the side of a dirt road. There are a few more but none of this happend in real rough country, nor was I out there for fun, but they all cost $$. Take your truck off road enough and something is going to break and it will cost you money and down time.
If you can afford it then go have fun, if you can't then wait until you can and the 4runner is a great choice.
Doug