Removing the 2 cats in the y-pipe is definitely going to improve the performance a bit. I highly doubt there would be any noticeable sound difference if you keep the stock muffler.
For a reference point, I replaced the headers, up-pipe, and downpipe on my '05 Subaru Legacy GT (turbo 2.5L), and...