Hey Willy,
Sounds to me like a classic case of pulley's not being in alignment with one another. You might want to double and triple check that the pulleys align perfectly, or at higher RPM's the belts tend to fly off.
Make sure your alt, PS, A/C compressor, etc are all mounted properly with the brackets all in the right position. It doesn't take much sometimes, as you're finding out.
Another thought is make sure all the bolts are in place. Extra vibration from loose or missing bolts could do it also.
Stomper, new guy.