LOL! I went out and played with the different modes. Turns out I was in party mode all along. There is definitely a difference in my SR5. I most certainly do have sound coming from the front speakers in party mode. However, I wouldn't say that there is increased clarity using the party mode. Rather, I think the shift to party mode not only directs power away from the 2nd row speakers, but also appears to result in an increase in power output to the front and hatch speakers; thus an increase in volume output by those speakers. Loud speakers generally sound better. I found that, to my ears, use of the 2nd row speakers reduces the volume by 5-6 ticks of the meter. In other words, if you turn the volume up 5-6 ticks when switching to non-party mode, the volume output between the two is approximately the same. Try doing this and see which mode you prefer.
It would be interesting to see the schematics for the stock stereo, but I'd venture to bet that the sets of speakers (front, middle, and back) are not independently driven by the amp.