I judge by the amount of valve train rattle on startup. No rattle, no warm up; oil is flowing. Some rattle, some warm up; oil is slow. As long as oil pressures are up to snuff, I drive it immediately although conservatively.
More important to drive cautiously at start up. Coolant temps might rise in a minute or so of driving but oil/trans temps take WAY longer and won't rise much unless the power-train is under load.
I typically wait a minute or so then take off. Your engine should build up oil pressure and the valve train gets oil fairly quickly, if not you have bigger problems to worry about IMO. Strongly recommend staying on top of your preventative maintenance like the others are sayin.. my 4.7 has 205k on the clock and has had a lot of cold start ups in the Colorado winters. I may have left her running just long enough to scrap of the windshield
When it's cold (sub 25 ish?), I typically let it warm up for 1-2 minutes, if it's warmer than 50, key on and we're gone.