clear sign that ball joints need to be replaced. Stained, cracked, torn boots. Whoever told you that upper ball joints are uneccesary to replace was sadly mistaken. The lower ball joints see more abuse and damage then the uppers so the uppers still need to be looked at and replaced when needed.
Yes they are a pain because you have to remove the Upper control arm and then press them out, press the new ones in.